Awesome

WebBot tutorials

This repository contains tutorials for parsing search engine results scraped with WebBot.

Parsing with Python

You can parse search result pages saved from Google, DuckDuckGo, etc. using our Python package WebBotParser for further analysis. To install it, use pip:

pip install git+https://github.com/gesiscss/WebBotParser

For examplary usage, see example.ipynb.

Parsing with R

Refer to webbotparseR, a similar R package.

Alternative

WebSearcher is a Python package that facilitates obtaining and parsing search results from Google text search. Compared to webbotparser, it supports parsing more diverse results (ads, knowledge boxes, etc.), but only Google text results (for now). websearcher.ipynb illustrates how to utilize WebSearcher's parsing capabilities on search result pages obtained using WebBot.