Last Saturday, I created a crontab to execute a long running script which supposed to run every 45 minutes. I went home and realized I got hundreds of error messages notifications in my mailbox. What's the problem? The script passed the test and everything was OK back then.
The next day, I had to go to myh office and investigated the situation. There were 5 instances of the script running at the same time. What? What the hell was going on with the shitty cron? The server which my script connects to does not allow multiple session of the same account. So, that was why I got a lot of error messages. But, why were there 5 running tasks? It was because the script needs more than 45 minutes to complete. And then after 45 minutes, when the first task hadden't finished yet, the second task was execute, and so on...
So, to solve this issue, I need to find a way to make sure that there is only one instance of the script running at a time. There are many ways to do that, such as: implement a lock mechanism in the script itself,. which are way complicated. Luckily, there is a great tool built-in Ubuntu called run-one. Run-One makes sure that the crontab only execute one instance of the same task at a time. So, my crontab will look like this:
*/45 * * * * run-one /path/to/myscript >> 2>&1
No more error! Greatness!
For more information about run-one: http://blog.dustinkirkland.com/2011/02/introducing-run-one-and-run-this-one.html