Awesome

Fess Crawler

Overview

Fess Crawler is a crawler library for crawling a web site and a file system.