I’ve set up a cron job to run a Python script to scrape some web pages.
/etc/crontab
GNU nano 2.3.1 File: crontab SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=my_email_address@domain.com # For details see man 4 crontabs # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed */2 * * * * root /usr/bin scrapy crawl mycrawler
However, the emails are informing me that…
/bin/bash: /usr/bin: Is a directory
When I manually run the script, it pipes data into my database, but when the cron job executes the script, nothing…
What does the /bin/bash: /usr/bin: Is a directory
message allude to?!
Advertisement
Answer
As discussed in comments the very initial error is that the entry places /usr/bin
where the executable should be:
*/2 * * * * root /usr/bin scrapy crawl mycrawler ^^^^^^^^ command
Once fixed to be scapy
, the ultimate issue is that scrapy
is in /usr/local/bin
which is not in your PATH
. To change this:
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin/
And then you should be able to just do:
*/2 * * * * root cd <project dir> && scrapy crawl mycrawler