Automatic Running
In a web application there are many cases, when a certain piece of code or script needs to run automatically in the background to perform certain tasks, without the need of any user to start it. Like scraping, extracting or crawling data from web or perform some regular tasks like sending emails, system maintenance or administration, taking backups or whatever.
CronJobs
Scheduling a task to run automatically is really simple using a utility called "Cron". "Cron" is a utility (basically a program that runs as a background job/process) that can execute scheduled (time-based) commands. These commands are basically the jobs or tasks that are referred to as "Cron Jobs". Cron is available in Unix-like operating systems like Linux, Mac, FreeBSD, etc.
Now say, if a php script called regular.php has to be executed regularly every day or every month or on certain weekdays then Cron can be configured to execute the script at defined schedules and hence automation is achieved.
Cron can be configured from command line as well as cPanel on web hosts.
Let me show you how to manage Cron Jobs from terminal. The `crontab` command is basically used for the aforementioned purpose. It is used to maintain crontab files for individual users that are read by the Cron daemon to perform the jobs.
$ crontab --help crontab: invalid option -- '-' crontab: usage error: unrecognized option usage: crontab [-u user] file crontab [ -u user ] [ -i ] { -e | -l | -r } (default operation is replace, per 1003.2) -e (edit user's crontab) -l (list user's crontab) -r (delete user's crontab) -i (prompt before deleting user's crontab)
The command `crontab -l` will basically list/show the current user's crontab(what tasks have been scheduled to be executed).
Crontab consist of lines like this -
The 5 stars indicate the time to execute and the `command_to_execute` is the command that you want to execute at that point of time. If the PHP script regular.php, resides in the HOME of the user, then the command to execute that script will be -
The 5 stars indicate the following (exactly in that order) -
Minute Hour Day Month Day
Minute - 0-59
Hour - 0-23
Day - 1-31
Month - 1-12
Weekday 0-6 (0= Sunday)
* means any (or every)
So to execute a script everyday -
0 0 * * * - means At 00:00 anyday anymonth anyweekday
Hence, the cron job will be -
Now here comes something interesting. All your cron jobs can be written to a single file (say, crontasks.txt) and scheduled with a single command -
$ crontab crontasks.txt
Another line that can be added at the starting of the crontasks.txt file is -
[email protected]
Adding the above line will email the CronJobs output to the specified email address.
So the crontasks.txt file can look like this :
[email protected] 0 0 * * * php /home/username/www/regular.php
Although I showed you how to set the cronjobs using a file, you can also edit your crontab file directly using the following command -
$ crontab -e
If you are running the `crontab -e` command for the first time then your terminal might ask you to choose a command line text editor that you would like to use to edit the crontab. So, just make a selection like vim or nano and go ahead!
$ crontab -e no crontab for awakened - using an empty one Select an editor. To change later, run 'select-editor'. 1. /bin/ed 2. /bin/nano <---- easiest 3. /usr/bin/vim.basic 4. /usr/bin/vim.gnome 5. /usr/bin/vim.tiny Choose 1-5 [2]: 2
Now you can simple edit your crontab -> save -> exit. Thats it!
Before setting a new cronjob you may want to backup the previous set of cron tasks -
$ crontab -l > /home/username/cronjobs_backup.txt
Finally, Removing all cron jobs is very easy -
crontab -r
But I dont want the output of the commands executed by Cron !?
Right! you can simply redirect the output to /dev/null like this -
0 0 * * * php /home/username/www/regular.php >/dev/null 2>&1
So what does ">/dev/null 2>&1" do ?
">" is basically used to redirect the output of commands. ">/dev/null" redirects the STDOUT (Standard Output) of the commands to /dev/null which is basically like a blackhole. Its a special file that discards all streams of data written to it. "2>&1" redirects the STDERR (Standard Error) to STDOUT which was basically getting redirected to /dev/null.
2 is the built-in numbering for STDERR while 1 is for STDOUT. Hence, STDOUT and STDERR both gets redirected to /dev/null.
Similarly, if you want to log the output of the commands in a file, then you can replace ">/dev/null 2>&1" with ">/path/to/log/file 2>&1".
Conclusion
So I hope you learnt how easy it is to set cronjobs and how essential Cron can be in our web applications or system administrations.
Please let me know about your thoughts in the comments!
Try doing a chmod 777 filename first time and everytime the file is created or modified.
I am attempting to run a cron job and have run into some issues. The cron is a SOAP request to noaa.gov. The result is written to a file which is then saved to a cache directory. The script cycles through and writes the files just fine. The problem is when the script executes three hours later (it is set to run on three hour intervals) there is a permissions issue with opening the previously written files. The cron fails and the old files remain. If I manually delete the files, then it runs again just fine. Any thoughts on how to deal with this? is it MAC related?
Cheers,
Sam