I’ve set up a cron job to run a Python script to scrape some web pages.
/etc/crontab
JavaScript
x
GNU nano 2.3.1 File: crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=my_email_address@domain.com
# For details see man 4 crontabs
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
*/2 * * * * root /usr/bin scrapy crawl mycrawler
However, the emails are informing me that…
/bin/bash: /usr/bin: Is a directory
When I manually run the script, it pipes data into my database, but when the cron job executes the script, nothing…
What does the /bin/bash: /usr/bin: Is a directory
message allude to?!
Advertisement
Answer
As discussed in comments the very initial error is that the entry places /usr/bin
where the executable should be:
JavaScript
*/2 * * * * root /usr/bin scrapy crawl mycrawler
^^^^^^^^
command
Once fixed to be scapy
, the ultimate issue is that scrapy
is in /usr/local/bin
which is not in your PATH
. To change this:
JavaScript
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin/
And then you should be able to just do:
JavaScript
*/2 * * * * root cd <project dir> && scrapy crawl mycrawler