/dev/pts/0' | at now + 1 minuteswarning: commands will be executed using /bin/shjob 2 at Sun May 26 06:30:00 2024After several seconds, a greeting should be printed to our terminal.bash...vagrant@ubuntu2204:~$ Greetings vagrant!It's not as common for this to be used to schedule one time tasks, but if youever needed to, now you have an idea of how this might work. In the next sectionwe'll learn about scheduling time-based tasks using cron and crontab.For a more in-depth exploration of scheduling things with at review therelevant articles in the further reading section below.Using crontab to schedule jobsIn Linux we use the crontab command to interact with tasks scheduled withthe cron daemon. Each user, including the root user, can schedule jobs that runas their user.Display your user's crontab with crontab -l.bashvagrant@ubuntu2204:~$ crontab -lno crontab for vagrantUnless you've already created a crontab for your user, you probably won't haveone yet. Let's create a simple cronjob to understand how it works.Using the crontab -e command, let's create our first cronjob. On Ubuntu, ifthis is you're first time editing a crontab you will be greeted with a menu tochoose your preferred editor.```bashvagrant@ubuntu2204:~$ crontab -eno crontab for vagrant - using an empty oneSelect an editor. To change later, run 'select-editor'. 1. /bin/nano <---- easiest 2. /usr/bin/vim.basic 3. /usr/bin/vim.tiny 4. /bin/edChoose 1-4 [1]: 2```Choose whatever your preferred editor is then press Enter.At the bottom of the file add the following cronjob and then save and quit thefile.bash* * * * * echo "Hello world!" > /dev/pts/0NOTE: Make sure that the /dev/pts/0 file path matches whatever was printedby your tty command above.Next, let's take a look at the crontab we just installed by running crontab -lagain. You should see the cronjob you created printed to your terminal.bashvagrant@ubuntu2204:~$ crontab -l* * * * * echo "Hello world!" > /dev/pts/0This cronjob will print the string Hello world! to your terminal every minute until we remove or update the cronjob. Wait a few minutes and see what it does.bashvagrant@ubuntu2204:~$ Hello world!Hello world!Hello world!...When you're ready uninstall the crontab you created with crontab -r.Understanding crontab syntaxThe basic crontab syntax is as follows:```* * * * * command to be executed| | | | || | | | ----- Day of week (0 - 7) (Sunday=0 or 7)| | | ------- Month (1 - 12)| | --------- Day of month (1 - 31)| ----------- Hour (0 - 23)------------- Minute (0 - 59)```Minute values can be from 0 to 59.Hour values can be from 0 to 23.Day of month values can be from 1 to 31.Month values can be from 1 to 12.Day of week values can be from 0 to 6, with 0 denoting Sunday.There are different operators that can be used as a short-hand to specify multiplevalues in each field:SymbolDescription*Wildcard, specifies every possible time interval,List multiple values separated by a comma.-Specify a range between two numbers, separated by a hyphen/Specify a periodicity/frequency using a slashThere's also a helpful site to check cron schedule expressions at crontab.guru.Use the crontab.guru site to play around with the different expressions to getan idea of how it works or click the random button to generate an expressionat random.Your Tasks TodaySchedule daily backups of user's home directoriesSchedule a task that looks for any backups that are more than 7 days old anddeletes themAutomating common system administration tasksOne common use-case that cronjobs are used for is scheduling backups of variousthings. As the root user, we're going to create a cronjob that creates acompressed archive of all of the user's home directories using the tar utility.Tar is short for "tape archive" and harkens back to earlier days of Unix andLinux when data was commonly archived on tape storage similar to cassette tapes.As a general rule, it's good to test your command or script before installing itas a cronjob. First we'll create a backup of /home by manually running aversion of our tar command.bashvagrant@ubuntu2204:~$ sudo tar -czvf /var/backups/home.tar.gz /home/tar: Removing leading `/' from member names/home//home/ubuntu//home/ubuntu/.profile/home/ubuntu/.bash_logout/home/ubuntu/.bashrc/home/ubuntu/.ssh//home/ubuntu/.ssh/authorized_keys...NOTE: We're passing the -v verbose flag to tar so that we can see betterwhat it's doing. -czf stand for "create", "gzip compress", and "file" inthat order. See man tar for further details.Let's also use the date command to allow us to insert the date of the backupinto the filename. Since we'll be taking daily backups, after this cronjobhas ran for a few days we will have a few days worth of backups each with it'sown archive tagged with the date.bashvagrant@ubuntu2204:~$ dateSun May 26 04:12:13 UTC 2024The default string printed by the date command isn't that useful. Let's outputthe date in ISO 8601 format, sometimes referred to as the "ISO date".bashvagrant@ubuntu2204:~$ date -I2024-05-26This is a more useful string that we can combine with our tar command tocreate an archive with today's date in it.bashvagrant@ubuntu2204:~$ sudo tar -czvf /var/backups/home.$(date -I).tar.gz /home/tar: Removing leading `/' from member names/home//home/ubuntu/...Let's look at the backups we've created to understand how this date command isbeing inserted into our filename.bashvagrant@ubuntu2204:~$ ls -l /var/backupstotal 16-rw-r--r-- 1 root root 8205 May 26 04:16 home.2024-05-26.tar.gz-rw-r--r-- 1 root root 3873 May 26 04:07 home.tar.gzNOTE: These .tar.gz files are often called tarballsby sysadmins.Create and edit a crontab for root with sudo crontab -e and add the followingcronjob.bash0 5 * * * tar -zcf /var/backups/home.$(date -I).tar.gz /home/This cronjob will run every day at 05:00. After a few days there will be severalbackups of user's home directories in /var/backups.If we were to let this cronjob run indefinitely, after a while we would end upwith a lot of backups in /var/backups. Over time this will cause the diskspace being used to grow and could fill our disk. It's probably bestthat we don't let that happen. To mitigate this risk, we'll setup anothercronjob that runs everyday and cleans up old backups that we don't need to store.The find command is like a swiss army knife for finding files based on allkinds of criteria and listing them or doing other things to them, such asdeleting them. We're going to craft a find command that finds all of thebackups we created and deletes any that are older than 7 days.First let's get an idea of how the find command works by finding all of ourbackups and listing them.bashvagrant@ubuntu2204:~$ sudo find /var/backups -name "home.*.tar.gz"/var/backups/home.2024-05-26.tar.gz...What this command is doing is looking for all of the files in /var/backupsthat start with home. and end with .tar.gz. The * is a wildcard characterthat matches any string.In our case we need to create a scheduled task that will find all of the filesolder than 7 days in /var/backups and delete them. Run sudo crontab -e andinstall the following cronjob.bash30 5 * * * find /var/backups -name "home.*.tar.gz" -mtime +7 -deleteNOTE: The -mtime flag is short for "modified time" and in our case find islooking for files that were modified more than 7 days ago, that's what the+7 indicates. The find command will be covered in greater detail on[Day 11 - Finding things...](11.md).By now, our crontab should look something like this:```bashvagrant@ubuntu2204:~$ sudo crontab -lDaily user dirs backup0 5 * * * tar -zcf /var/backups/home.$(date -I).tar.gz /home/Retain 7 days of homedir backups30 5 * * * find /var/backups -name "home.*.tar.gz" -mtime +7 -delete```Setting up cronjobs using the find ... -delete syntax is fairly idiomatic ofscheduled tasks a system administrator might use to manage files and removeold files that are no longer needed to prevent disks from getting full. It's notuncommon to see more sophisticated cron scripts that use a combination of toolslike tar, find, and rsync to manage backups incrementally or on a scheduleand implement a more sophisticated retention policy based on real-world use-cases.System crontabThere’s also a system-wide crontab defined in /etc/crontab. Let's take a lookat this file.```bashvagrant@ubuntu2204:~$ cat /etc/crontab/etc/crontab: system-wide crontabUnlike any other crontab you don't have to run the `crontab'command to install the new version when you edit this fileand files in /etc/cron.d. These files also have username fields,that none of the other crontabs do.SHELL=/bin/shYou can also override PATH, but by default, newer versions inherit it from the environmentPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binExample of job definition:.---------------- minute (0 - 59)| .------------- hour (0 - 23)| | .---------- day of month (1 - 31)| | | .------- month (1 - 12) OR jan,feb,mar,apr ...| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat| | | | |* * * * * user-name command to be executed17 * * * * root cd / && run-parts --report /etc/cron.hourly25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )```By now the basic syntax should be familiar to you, but you'll notice an extrafield user-name. This specifies the user that runs the task and is unique tothe system crontab at /etc/crontab.It's not common for system administrators to use /etc/crontab anymore andinstead user's are encouraged to install their own crontab for their user, evenfor the root user. User crontab's are all located in /var/spool/cron. Theexact subdirectory tends to vary depending on the distribution.bashvagrant@ubuntu2204:~$ sudo ls -l /var/spool/cron/crontabstotal 8-rw------- 1 root crontab 392 May 26 04:45 root-rw------- 1 vagrant crontab 1108 May 26 05:45 vagrantEach user has their own crontab with their user as the filename.Note that the system crontab shown above also manages cronjobs that run daily,weekly, and monthly as scripts in the /etc/cron.* directories. Let's lookat an example.bashvagrant@ubuntu2204:~$ ls -l /etc/cron.dailytotal 20-rwxr-xr-x 1 root root 376 Nov 11 2019 apport-rwxr-xr-x 1 root root 1478 Apr 8 2022 apt-compat-rwxr-xr-x 1 root root 123 Dec 5 2021 dpkg-rwxr-xr-x 1 root root 377 Jan 24 2022 logrotate-rwxr-xr-x 1 root root 1330 Mar 17 2022 man-dbEach of these files is a script or a shortcut to a script to do some regulartask and they're run in alphabetic order by run-parts. So in this caseapport will run first. Use less or cat to view some of the scripts on yoursystem - many will look very complex and are best left well alone, but othersmay be just a few lines of simple commands.```bashvagrant@ubuntu2204:~$ cat /etc/cron.daily/dpkg !/bin/shSkip if systemd is running.if [ -d /run/systemd/system ]; then exit 0fi/usr/libexec/dpkg/dpkg-db-backup```As an alternative to scheduling jobs with crontab you may also create a scriptand put it into one of the /etc/cron.{daily,weekly,monthly} directories andit will get ran at the desired interval.A note about systemd timersAll major Linux distributions now include "systemd". As well as starting andstopping services, this can also be used to run tasks at specific times via"timers". See which ones are already configured on your server with:bashsystemctl list-timersUse the links in the further reading section to read up about how these timers work.Further readingfreeCodeCamp: Job Scheduling in RHEL – cron and at Explained with ExamplesnixCraft: How To Add Jobs To cron Under Linux or UNIXArabesque: Cron best practicesLinode: Using Cron to Schedule Tasks for Certain Times or IntervalsArch Wiki: A good overview of systemd/TimersHow to Use Systemd Timers as a Cron ReplacementLicensePREVIOUS DAY'S LESSONDay 9 - Diving into networkingSome rights reserved. Check the license termshere" title="Day 10 - Scheduling tasks">full image - Repost: Day 10 - Scheduling tasks (from Reddit.com, Day 10 - Scheduling tasks)
Mining:
Exchanges:
Donations:
Complementary videoIntroductionLinux has a rich set of features for running scheduled tasks. One of the keyattributes of a good sysadmin is getting the computer to do your work for you(sometimes misrepresented as laziness!) - and a well configured set ofscheduled tasks is key to keeping your server running well.The time-based job scheduler cron(8) is theone most commonly used by Linux sysadmins. It's been around more or less in it'scurrent form since Unix System V and uses a standardized syntax that's inwidespread use.Using at to schedule oneshot tasksIf you're on Ubuntu, you will likely need to install the at package first.bashsudo apt updatesudo apt install atWe'll use the at command to schedule a one time task to be ran at some point in the future.Next, let's print the filename of the terminal connected to standard input (inLinux everything is a file, including your terminal!). We're going to echosomething to our terminal at some point in the future to get an ideaof how scheduling future tasks with at works.bashvagrant@ubuntu2204:~$ tty/dev/pts/0Now we'll schedule a command to echo a greeting to our terminal 1 minute in thefuture.bashvagrant@ubuntu2204:~$ echo 'echo "Greetings $USER!" > /dev/pts/0' | at now + 1 minuteswarning: commands will be executed using /bin/shjob 2 at Sun May 26 06:30:00 2024After several seconds, a greeting should be printed to our terminal.bash...vagrant@ubuntu2204:~$ Greetings vagrant!It's not as common for this to be used to schedule one time tasks, but if youever needed to, now you have an idea of how this might work. In the next sectionwe'll learn about scheduling time-based tasks using cron and crontab.For a more in-depth exploration of scheduling things with at review therelevant articles in the further reading section below.Using crontab to schedule jobsIn Linux we use the crontab command to interact with tasks scheduled withthe cron daemon. Each user, including the root user, can schedule jobs that runas their user.Display your user's crontab with crontab -l.bashvagrant@ubuntu2204:~$ crontab -lno crontab for vagrantUnless you've already created a crontab for your user, you probably won't haveone yet. Let's create a simple cronjob to understand how it works.Using the crontab -e command, let's create our first cronjob. On Ubuntu, ifthis is you're first time editing a crontab you will be greeted with a menu tochoose your preferred editor.```bashvagrant@ubuntu2204:~$ crontab -eno crontab for vagrant - using an empty oneSelect an editor. To change later, run 'select-editor'. 1. /bin/nano <---- easiest 2. /usr/bin/vim.basic 3. /usr/bin/vim.tiny 4. /bin/edChoose 1-4 [1]: 2```Choose whatever your preferred editor is then press Enter.At the bottom of the file add the following cronjob and then save and quit thefile.bash* * * * * echo "Hello world!" > /dev/pts/0NOTE: Make sure that the /dev/pts/0 file path matches whatever was printedby your tty command above.Next, let's take a look at the crontab we just installed by running crontab -lagain. You should see the cronjob you created printed to your terminal.bashvagrant@ubuntu2204:~$ crontab -l* * * * * echo "Hello world!" > /dev/pts/0This cronjob will print the string Hello world! to your terminal every minute until we remove or update the cronjob. Wait a few minutes and see what it does.bashvagrant@ubuntu2204:~$ Hello world!Hello world!Hello world!...When you're ready uninstall the crontab you created with crontab -r.Understanding crontab syntaxThe basic crontab syntax is as follows:```* * * * * command to be executed| | | | || | | | ----- Day of week (0 - 7) (Sunday=0 or 7)| | | ------- Month (1 - 12)| | --------- Day of month (1 - 31)| ----------- Hour (0 - 23)------------- Minute (0 - 59)```Minute values can be from 0 to 59.Hour values can be from 0 to 23.Day of month values can be from 1 to 31.Month values can be from 1 to 12.Day of week values can be from 0 to 6, with 0 denoting Sunday.There are different operators that can be used as a short-hand to specify multiplevalues in each field:SymbolDescription*Wildcard, specifies every possible time interval,List multiple values separated by a comma.-Specify a range between two numbers, separated by a hyphen/Specify a periodicity/frequency using a slashThere's also a helpful site to check cron schedule expressions at crontab.guru.Use the crontab.guru site to play around with the different expressions to getan idea of how it works or click the random button to generate an expressionat random.Your Tasks TodaySchedule daily backups of user's home directoriesSchedule a task that looks for any backups that are more than 7 days old anddeletes themAutomating common system administration tasksOne common use-case that cronjobs are used for is scheduling backups of variousthings. As the root user, we're going to create a cronjob that creates acompressed archive of all of the user's home directories using the tar utility.Tar is short for "tape archive" and harkens back to earlier days of Unix andLinux when data was commonly archived on tape storage similar to cassette tapes.As a general rule, it's good to test your command or script before installing itas a cronjob. First we'll create a backup of /home by manually running aversion of our tar command.bashvagrant@ubuntu2204:~$ sudo tar -czvf /var/backups/home.tar.gz /home/tar: Removing leading `/' from member names/home//home/ubuntu//home/ubuntu/.profile/home/ubuntu/.bash_logout/home/ubuntu/.bashrc/home/ubuntu/.ssh//home/ubuntu/.ssh/authorized_keys...NOTE: We're passing the -v verbose flag to tar so that we can see betterwhat it's doing. -czf stand for "create", "gzip compress", and "file" inthat order. See man tar for further details.Let's also use the date command to allow us to insert the date of the backupinto the filename. Since we'll be taking daily backups, after this cronjobhas ran for a few days we will have a few days worth of backups each with it'sown archive tagged with the date.bashvagrant@ubuntu2204:~$ dateSun May 26 04:12:13 UTC 2024The default string printed by the date command isn't that useful. Let's outputthe date in ISO 8601 format, sometimes referred to as the "ISO date".bashvagrant@ubuntu2204:~$ date -I2024-05-26This is a more useful string that we can combine with our tar command tocreate an archive with today's date in it.bashvagrant@ubuntu2204:~$ sudo tar -czvf /var/backups/home.$(date -I).tar.gz /home/tar: Removing leading `/' from member names/home//home/ubuntu/...Let's look at the backups we've created to understand how this date command isbeing inserted into our filename.bashvagrant@ubuntu2204:~$ ls -l /var/backupstotal 16-rw-r--r-- 1 root root 8205 May 26 04:16 home.2024-05-26.tar.gz-rw-r--r-- 1 root root 3873 May 26 04:07 home.tar.gzNOTE: These .tar.gz files are often called tarballsby sysadmins.Create and edit a crontab for root with sudo crontab -e and add the followingcronjob.bash0 5 * * * tar -zcf /var/backups/home.$(date -I).tar.gz /home/This cronjob will run every day at 05:00. After a few days there will be severalbackups of user's home directories in /var/backups.If we were to let this cronjob run indefinitely, after a while we would end upwith a lot of backups in /var/backups. Over time this will cause the diskspace being used to grow and could fill our disk. It's probably bestthat we don't let that happen. To mitigate this risk, we'll setup anothercronjob that runs everyday and cleans up old backups that we don't need to store.The find command is like a swiss army knife for finding files based on allkinds of criteria and listing them or doing other things to them, such asdeleting them. We're going to craft a find command that finds all of thebackups we created and deletes any that are older than 7 days.First let's get an idea of how the find command works by finding all of ourbackups and listing them.bashvagrant@ubuntu2204:~$ sudo find /var/backups -name "home.*.tar.gz"/var/backups/home.2024-05-26.tar.gz...What this command is doing is looking for all of the files in /var/backupsthat start with home. and end with .tar.gz. The * is a wildcard characterthat matches any string.In our case we need to create a scheduled task that will find all of the filesolder than 7 days in /var/backups and delete them. Run sudo crontab -e andinstall the following cronjob.bash30 5 * * * find /var/backups -name "home.*.tar.gz" -mtime +7 -deleteNOTE: The -mtime flag is short for "modified time" and in our case find islooking for files that were modified more than 7 days ago, that's what the+7 indicates. The find command will be covered in greater detail on[Day 11 - Finding things...](11.md).By now, our crontab should look something like this:```bashvagrant@ubuntu2204:~$ sudo crontab -lDaily user dirs backup0 5 * * * tar -zcf /var/backups/home.$(date -I).tar.gz /home/Retain 7 days of homedir backups30 5 * * * find /var/backups -name "home.*.tar.gz" -mtime +7 -delete```Setting up cronjobs using the find ... -delete syntax is fairly idiomatic ofscheduled tasks a system administrator might use to manage files and removeold files that are no longer needed to prevent disks from getting full. It's notuncommon to see more sophisticated cron scripts that use a combination of toolslike tar, find, and rsync to manage backups incrementally or on a scheduleand implement a more sophisticated retention policy based on real-world use-cases.System crontabThere’s also a system-wide crontab defined in /etc/crontab. Let's take a lookat this file.```bashvagrant@ubuntu2204:~$ cat /etc/crontab/etc/crontab: system-wide crontabUnlike any other crontab you don't have to run the `crontab'command to install the new version when you edit this fileand files in /etc/cron.d. These files also have username fields,that none of the other crontabs do.SHELL=/bin/shYou can also override PATH, but by default, newer versions inherit it from the environmentPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binExample of job definition:.---------------- minute (0 - 59)| .------------- hour (0 - 23)| | .---------- day of month (1 - 31)| | | .------- month (1 - 12) OR jan,feb,mar,apr ...| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat| | | | |* * * * * user-name command to be executed17 * * * * root cd / && run-parts --report /etc/cron.hourly25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )```By now the basic syntax should be familiar to you, but you'll notice an extrafield user-name. This specifies the user that runs the task and is unique tothe system crontab at /etc/crontab.It's not common for system administrators to use /etc/crontab anymore andinstead user's are encouraged to install their own crontab for their user, evenfor the root user. User crontab's are all located in /var/spool/cron. Theexact subdirectory tends to vary depending on the distribution.bashvagrant@ubuntu2204:~$ sudo ls -l /var/spool/cron/crontabstotal 8-rw------- 1 root crontab 392 May 26 04:45 root-rw------- 1 vagrant crontab 1108 May 26 05:45 vagrantEach user has their own crontab with their user as the filename.Note that the system crontab shown above also manages cronjobs that run daily,weekly, and monthly as scripts in the /etc/cron.* directories. Let's lookat an example.bashvagrant@ubuntu2204:~$ ls -l /etc/cron.dailytotal 20-rwxr-xr-x 1 root root 376 Nov 11 2019 apport-rwxr-xr-x 1 root root 1478 Apr 8 2022 apt-compat-rwxr-xr-x 1 root root 123 Dec 5 2021 dpkg-rwxr-xr-x 1 root root 377 Jan 24 2022 logrotate-rwxr-xr-x 1 root root 1330 Mar 17 2022 man-dbEach of these files is a script or a shortcut to a script to do some regulartask and they're run in alphabetic order by run-parts. So in this caseapport will run first. Use less or cat to view some of the scripts on yoursystem - many will look very complex and are best left well alone, but othersmay be just a few lines of simple commands.```bashvagrant@ubuntu2204:~$ cat /etc/cron.daily/dpkg !/bin/shSkip if systemd is running.if [ -d /run/systemd/system ]; then exit 0fi/usr/libexec/dpkg/dpkg-db-backup```As an alternative to scheduling jobs with crontab you may also create a scriptand put it into one of the /etc/cron.{daily,weekly,monthly} directories andit will get ran at the desired interval.A note about systemd timersAll major Linux distributions now include "systemd". As well as starting andstopping services, this can also be used to run tasks at specific times via"timers". See which ones are already configured on your server with:bashsystemctl list-timersUse the links in the further reading section to read up about how these timers work.Further readingfreeCodeCamp: Job Scheduling in RHEL – cron and at Explained with ExamplesnixCraft: How To Add Jobs To cron Under Linux or UNIXArabesque: Cron best practicesLinode: Using Cron to Schedule Tasks for Certain Times or IntervalsArch Wiki: A good overview of systemd/TimersHow to Use Systemd Timers as a Cron ReplacementLicensePREVIOUS DAY'S LESSONDay 9 - Diving into networkingSome rights reserved. Check the license termshere
Social Media Icons