Home

Awesome

STW-Collection

Scrap The Web Collection; blog posts.

This repo contains Scrapy sample code to scrap the following kind of websites:

  1. Do you want to learn Scrapy? ScrapScrapy is gonna be your first scrapy project in that case.
  2. If you want to scrap a simple website without any javascript or AJAX calls,you can have a look at this project. This uses CrawlSpider.
  3. If you want to use selenium with scrapy, have a look at this project.
  4. You can refer this project, if you want to save to Django DB as you scrap.