Scrapy: open source crawling and web scraping framework

Scrapy is a fast high-level web open source crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. Requires Python 2.7 or Python 3.4+.

Composer: proc_open(): fork failed errors

If composer shows proc_open() fork failed on some commands: PHP Fatal error: Uncaught exception ‘ErrorException’ with message ‘proc_open(): fork failed – Cannot allocate memory’ in phar This could be happening because the VPS runs out of memory and has no Swap space enabled. free -m   total used free shared buffers cached Mem: 2048 357
Continue reading Composer: proc_open(): fork failed errors