Skip to content
Advertisement

Tag: scrapy

How to store screen output of Scrapy into a file in Linux

I often use to debug Scrapy project. However, when there are lot of items crawled, those log information will be embedded inside a lot of non-useful information. What I want is to store the screen output into a file, so that I can do search, do text editing, etc. I tried: but it doesn’t work Answer It seems that your

Installing scrapy (openSSL)

I’m using scrapy for couple of months now. I’ve been using it on the couple of machines and never had trouble with it. I used it on Windows xp and 7, I also used it on arch linux and never had trouble installing. Pretty much it was only pip install scrapy and done. Now I’m trying to set it up

Advertisement