Awesome

AutoAgora

AutoAgora is beta software. Please use responsibly, monitor it (Prometheus metrics) , write issues and contribute!

An Agora cost model automation tool for The Graph indexers:

Automates the creation of relative prices for commonly seen query skeletons. (Off by default).
Continuously tunes a per-subgraph absolute price multiplier, using continuous online reinforcement learning.

Indexer's guide

⚠️ WARNING

Note that, as of as of indexer v0.20.20, AutoAgora works only with The Graph's protocol on Ethereum mainnet. This is due to the indexer stack not yet supporting cost models on Arbitrum (as of v0.20.20).

As such, --indexer-service-metrics-endpoint should only point to the indexer-service instances operating on the Ethereum mainnet protocol, as it would otherwise make decisions based on wrong observations.

Installation

We recommend using the AutoAgora container at ghcr.io/semiotic-ai/autoagora.

Build

If you'd like to build the container yourself, we recommend that you download the repo with git rather than downloading a tarball:

git clone https://github.com/semiotic-ai/autoagora.git

So that the build process can read the git metadata from the repo:

cd autoagora
docker build -t autoagora .

Usage

For AutoAgora's relative query costs generator, you will also need to set up:

AutoAgora will continuously:

Watch for the indexer's current allocations by querying the indexer-agent's management GraphQL endpoint.
Analyze the query logs stored in a PostgreSQL database (relative query costs generator) -- logs that were previously processed by the AutoAgora indexer-service wrapper and the AutoAgora Processor.
Gather query metrics from the indexer-service's prometheus metrics endpoint.
Update the allocated subgraph's cost models by sending mutations to the indexer-agent's management GraphQL endpoint.

Therefore, only a single instance of AutoAgora should be running against an indexer-agent.

Configuration:

usage: autoagora [-h] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--json-logs JSON_LOGS]
                 --postgres-host POSTGRES_HOST [--postgres-port POSTGRES_PORT]
                 [--postgres-database POSTGRES_DATABASE] --postgres-username POSTGRES_USERNAME
                 --postgres-password POSTGRES_PASSWORD
                 [--postgres-max-connections POSTGRES_MAX_CONNECTIONS]
                 --indexer-agent-mgmt-endpoint INDEXER_AGENT_MGMT_ENDPOINT
                 [--indexer-agent-protocol-network INDEXER_AGENT_PROTOCOL_NETWORK]
                 (--indexer-service-metrics-endpoint INDEXER_SERVICE_METRICS_ENDPOINT | --indexer-service-metrics-k8s-service INDEXER_SERVICE_METRICS_K8S_SERVICE)
                 [--qps-observation-duration QPS_OBSERVATION_DURATION] [--relative-query-costs]
                 [--relative-query-costs-exclude-subgraphs RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS]
                 [--relative-query-costs-refresh-interval RELATIVE_QUERY_COSTS_REFRESH_INTERVAL]
                 [--manual-entry-path MANUAL_ENTRY_PATH]

optional arguments:
  -h, --help            show this help message and exit
  --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        [env var: LOG_LEVEL] (default: WARNING)
  --json-logs JSON_LOGS
                        Output logs in JSON format. Compatible with GKE. [env var: JSON_LOGS]
                        (default: False)
  --indexer-agent-mgmt-endpoint INDEXER_AGENT_MGMT_ENDPOINT
                        URL to the indexer-agent management GraphQL endpoint. [env var:
                        INDEXER_AGENT_MGMT_ENDPOINT] (default: None)
  --indexer-agent-protocol-network INDEXER_AGENT_PROTOCOL_NETWORK
                        Network identifier of the Graph network to operate on. Uses the network
                        identifier format expected by the indexer-agent. [env var:
                        INDEXER_AGENT_PROTOCOL_NETWORK] (default: eip155:1)
  --qps-observation-duration QPS_OBSERVATION_DURATION
                        Duration of the measurement period of the query-per-second after a price
                        multiplier update. [env var: QPS_OBSERVATION_DURATION] (default: 60)
  --manual-entry-path MANUAL_ENTRY_PATH
                        Path to find manual agora entries, this expects Agora model files named
                        {subgraph_hash}.agora [env var: MANUAL_ENTRY_PATH] (default: None)

Database settings:
  Must be the same database as AutoAgora Processor's if the relative costs models generator is
  enabled.

  --postgres-host POSTGRES_HOST
                        Host of the postgres instance to be used by AutoAgora. [env var:
                        POSTGRES_HOST] (default: None)
  --postgres-port POSTGRES_PORT
                        Port of the postgres instance to be used by AutoAgora. [env var:
                        POSTGRES_PORT] (default: 5432)
  --postgres-database POSTGRES_DATABASE
                        Name of the database to be used by AutoAgora. [env var:
                        POSTGRES_DATABASE] (default: autoagora)
  --postgres-username POSTGRES_USERNAME
                        Username for the database to be used by AutoAgora. [env var:
                        POSTGRES_USERNAME] (default: None)
  --postgres-password POSTGRES_PASSWORD
                        Password for the database to be used by AutoAgora. [env var:
                        POSTGRES_PASSWORD] (default: None)
  --postgres-max-connections POSTGRES_MAX_CONNECTIONS
                        Maximum postgres connections (internal pool). [env var:
                        POSTGRES_MAX_CONNECTIONS] (default: 1)

Indexer-service metrics endpoint. Exactly one argument required:
  --indexer-service-metrics-endpoint INDEXER_SERVICE_METRICS_ENDPOINT
                        HTTP endpoint for the indexer-service metrics. Can be a comma-separated
                        for multiple endpoints. [env var: INDEXER_SERVICE_METRICS_ENDPOINT]
                        (default: None)
  --indexer-service-metrics-k8s-service INDEXER_SERVICE_METRICS_K8S_SERVICE
                        Kubernetes service name for the indexer-service and pod port serving its
                        metrics. Will watch the service's endpoint IPs continuously for changes.
                        Format: <scheme>://<service_name>:<pod_metrics_port>/<path>. [env var:
                        INDEXER_SERVICE_METRICS_K8S_SERVICE] (default: None)

Relative query costs generator settings:
  --relative-query-costs
                        (EXPERIMENTAL) Enables the relative query cost generator. Otherwise only
                        builds a default query pricing model with automated market price
                        discovery. [env var: RELATIVE_QUERY_COSTS] (default: False)
  --relative-query-costs-exclude-subgraphs RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS
                        Comma delimited list of subgraphs (ipfs hash) to exclude from the
                        relative query costs model generator. [env var:
                        RELATIVE_QUERY_COSTS_EXCLUDE_SUBGRAPHS] (default: None)
  --relative-query-costs-refresh-interval RELATIVE_QUERY_COSTS_REFRESH_INTERVAL
                        (Seconds) Interval between rebuilds of the relative query costs models.
                        [env var: RELATIVE_QUERY_COSTS_REFRESH_INTERVAL] (default: 3600)

 If an arg is specified in more than one place, then commandline values override environment
variables which override defaults.

AutoAgora also exposes metrics for Prometheus on port 8000 (Example values):

# HELP bandit_reward Reward of the bandit training: queries_per_second * price_multiplier.
# TYPE bandit_reward gauge
bandit_reward{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 1.577651313168855e-07
# HELP bandit_price_multiplier Price multiplier sampled from the Gaussian model.
# TYPE bandit_price_multiplier gauge
bandit_price_multiplier{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 2.60150080442184e-07
# HELP bandit_stddev Standard deviation of the Gaussian price multiplier model.
# TYPE bandit_stddev gauge
bandit_stddev{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 1.843469500541687
# HELP bandit_mean Mean of the Gaussian price multiplier model.
# TYPE bandit_mean gauge
bandit_mean{subgraph="QmRDGLp6BHwiH9HAE2NYEE3f7LrKuRqziHBv76trT4etgU"} 3.653126148672616e-05

Where "bandit" refers to the reinforcement learning method (Continuum-armed bandit) used to track the market price for each subgraph.

Developer's guide

If you would like to contribute, please consult CONTRIBUTING.md.

Installation directly from the source code

To install AutoAgora directly from the source code please clone the repository and install package in the virtual environment using poetry:

git clone https://github.com/semiotic-ai/autoagora.git
cd autoagora
poetry install

Running the AutoAgora code

All scripts should be executed in the virtual environment managed by poetry.

Running the test suite

poetry run python -m pytest