Home

Awesome

JsonGenius

JsonGenius is a self-hosted scraping API that extracts structured data described by a JSON Schema. Provide any URL and a desired JSON Schema, and JsonGenius will return the structured data from the website.

Demo

image

Prerequisites

Usage

Docker Compose (recommended)

git clone https://github.com/semanser/jsongenius
cd jsongenius
export OPEN_AI_KEY=<your key here>
docker compose up

The API will be available at http://localhost:3001. You can change the port by editing the docker-compose.yml file.

Compile from source

git clone https://github.com/semanser/jsongenius
cd jsongenius
export OPEN_AI_KEY=<your key here>
go build .
./jsongenius

API

POST /lookup

This endpoint accepts a JSON body with the following fields:

Example

curl -X POST -H "Content-Type: application/json" -d '{
  "url": "https://www.amazon.com/s?k=gaming+headsets",
  "schema": {
    "type": "object",
    "properties": {
      "products": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": {
              "type": "string",
              "description": "The product name"
            },
            "price": {
              "type": "number",
              "description": "The price of the product in USD"
            }
          }
        }
      }
    }
  }
}' http://localhost:3001/lookup

FAQ