Awesome

spring-ai-integration-tests

This repository contains workflows that define the integration test automation for the Spring AI project.

Adding Secrets

All environment variables are stored in GitHub secrets.

To add or change a secret:

1. Open spring-projects/spring-ai-integration-tests in your browser.
2. If you have permissions, you will see "Settings" in the toolbar on that page.
3. Navigate to Settings -> Security -> Secrets and variables -> Actions.
4. Click New repository secret to add a new environment variable for the integration test pipeline.

Workflow Schedule

The integration tests run automatically:

• At 4:00 UTC on weekdays (Monday through Friday)
• At 10:00 UTC on weekdays (Monday through Friday)

The workflow can also be triggered manually using the workflow_dispatch event.

Job Types

Basic Integration Test Job

Example of a basic integration test job for testing AI models:

test-mistral-ai:
  runs-on: ubuntu-latest
  env:
    MISTRAL_AI_API_KEY: ${{ secrets.MISTRAL_AI_API_KEY }}
  steps:
    - name: Check secrets
      id: secret_check
      if: ${{ env.MISTRAL_AI_API_KEY != '' }}
      run: echo "Secrets exist"

    - name: Checkout the action
      uses: actions/checkout@v4

    - name: Integration Test
      if: steps.secret_check.conclusion == 'success'
      uses: ./.github/actions/do-integration-test
      with:
        model-name: mistral-ai

The standard steps:

• Define required environment variables using GitHub secrets
• Check if required secrets exist and have values
• Checkout the repository
• Run the integration test only if all secrets are available

The do-integration-test action handles:

• Checking environment variables
• Checking out the Spring AI source code
• Setting up JDK 17 with Maven caching
• Installing the Spring AI code without running tests
• Executing the integration tests with retry capability for failing tests

The only customization needed is the model name. The action assumes models/spring-ai- prefix, so for the above example with model-name mistral-ai, models/spring-ai-mistral-ai will be integration tested.

Multi-Module Test Job

For testing multiple modules in a single job:

test-vectorstores:
  runs-on: ubuntu-latest
  env:
    DOCKER_QUIET: 1              # Suppresses Docker CLI progress output
    TESTCONTAINERS_QUIET: true   # Additional quieting for testcontainers
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    MISTRAL_AI_API_KEY: ${{ secrets.MISTRAL_AI_API_KEY }}
    OLLAMA_TESTS_ENABLED: true
  steps:
    - name: Check secrets
      id: secret_check
      if: ${{ env.OPENAI_API_KEY != '' && env.MISTRAL_AI_API_KEY != '' }}
      run: echo "Secrets exist"

    - uses: actions/checkout@v4

    - name: Configure Testcontainers
      run: |
        mkdir -p $HOME
        echo "testcontainers.reuse.enable = true" >> $HOME/.testcontainers.properties

    - name: Integration Test
      uses: ./.github/actions/do-multi-module-test
      with:
        modules: vector-stores/spring-ai-module1,vector-stores/spring-ai-module2

The do-multi-module-test action is similar to do-integration-test but allows testing multiple modules in a single job.

Autoconfiguration Test

The autoconfiguration test job runs for Spring Boot autoconfiguration testing. Environment variables in the env section control which tests are enabled:

test-autoconfigure:
  env:
    AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
    AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
    # Additional keys can be uncommented when available
  runs-on: ubuntu-latest

Required Environment Variables

The following environment variables are needed for various tests:

Vector Store Tests

OPENAI_API_KEY
MISTRAL_AI_API_KEY
OLLAMA_TESTS_ENABLED

AI Model Tests

AZURE_OPENAI_API_KEY
AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_TRANSCRIPTION_API_KEY
AZURE_OPENAI_TRANSCRIPTION_ENDPOINT
AZURE_OPENAI_IMAGE_API_KEY
AZURE_OPENAI_IMAGE_ENDPOINT
MISTRAL_AI_API_KEY

Docker and TestContainers Configuration

For jobs using Docker or TestContainers, the following environment variables are used to reduce log verbosity:

env:
  DOCKER_QUIET: 1              # Suppresses Docker CLI progress output
  TESTCONTAINERS_QUIET: true   # Additional quieting for testcontainers

TestContainers reuse is enabled with:

echo "testcontainers.reuse.enable = true" >> $HOME/.testcontainers.properties

Additional Tests

The workflow also includes tests for:

• Spring AI Integration Tests
• Docker Compose Support
• TestContainers Support
• Spring Cloud Bindings

Each of these tests uses the multi-module test action with appropriate module paths.