Awesome

crawlerdetect

About

crawlerdetect is a Go version of PHP class @CrawlerDetect.

It helps to detect bots/crawlers/spiders via the user agent and other HTTP-headers. Currently able to detect 1,000's of bots/spiders/crawlers.

Installation

go get github.com/x-way/crawlerdetect

Basic Usage

import "fmt"
import "github.com/x-way/crawlerdetect"

func main() {
    uastring := "curl/7.54.0"
    if crawlerdetect.IsCrawler(uastring) {
        fmt.Println("Found a crawler")
    }
}

Contributing

The patterns and testcases are synced from the PHP repo. If you find a bot/spider/crawler user agent that crawlerdetect fails to detect, please submit a pull request with the regex pattern and a testcase to the upstream PHP repo.