You can also create a Twitter account, link it to your phone, and get an SMS whenever someone reports that trains are late.
To subscribe for updates, make an account, link it to your phone, and then on the Twitter main page, look for a box marked "people", on the Twitter Caltrain page, click the "follow" button underneath the "user" name.
I'm just doing this now, and will see how well it works next time something goes wrong. I could have used it this morning.
And if this thing works, I'll work on figuring out how to make updates myself.