Home

Awesome

Terraform Provider for Elastic Cloud

Go Acceptance Status

Terraform provider for the Elastic Cloud API, including:

Model changes might be introduced between minors until version 1.0.0 is released. Such changes and the expected impact will be detailed in the change log and the individual release notes.

Terraform provider scope

The goal for a Terraform provider is to orchestrate lifecycle for deployments via common set of APIs across ESS, ESSP and ECE (see https://www.elastic.co/guide/en/cloud/current/ec-restful-api.html for API examples)

Things which are out of scope for provider:

We now have Terraform provider for Elastic Stack https://github.com/elastic/terraform-provider-elasticstack which should be used for any operations on Elastic Stack products.

Version guidance

It is strongly recommended to consistently utilize the latest versions of both the Elastic Cloud terraform provider and Terraform CLI. Doing so not only mitigates the risk of encountering known issues but also enhances overall user experience.

Support

We welcome questions on how to use the Elastic providers. The providers are supported by Elastic. General questions, bugs and product issues should be raised in their corresponding repositories, either for the Elastic Stack provider, or the Elastic Cloud one. Questions can also be directed to the discuss forum. https://discuss.elastic.co/c/orchestration.

We will not, however, fix bugs upon customer demand, as we have to prioritize all pending bugs and features, as part of the product's backlog and release cycles.

Support tickets severity

Support tickets related to the Terraform provider can be opened with Elastic, however since the provider is just a client of the underlying product API's, we will not be able to treat provider related support requests as a Severity-1 (Immedediate time frame). Urgent, production-related Terraform issues can be resolved via direct interaction with the underlying project API or UI. We will ask customers to resort to these methods to resolve downtime or urgent issues.

Example usage

These examples are forward looking and might use an unreleased version, for a current view of working examples, please refer to the Terraform registry documentation.

terraform {
  required_version = ">= 0.12.29"

  required_providers {
    ec = {
      source  = "elastic/ec"
      version = "0.12.2"
    }
  }
}

provider "ec" {
  # ECE installation endpoint
  endpoint = "https://my.ece-environment.corp"

  # If the ECE installation has a self-signed certificate
  # setting "insecure" to true is required.
  insecure = true

  # APIKey is the recommended authentication mechanism. When
  # Targeting the Elasticsearch Service, APIKeys are the only
  # valid authentication mechanism.
  apikey = "my-apikey"

  # When targeting ECE installations, username and password
  # authentication is allowed.
  username = "my-username"
  password = "my-password"
}

data "ec_stack" "latest" {
  version_regex = "latest"
  region        = "us-east-1"
}

# Create an Elastic Cloud deployment
resource "ec_deployment" "example_minimal" {
  # Optional name.
  name = "my_example_deployment"

  # Mandatory fields
  region                 = "us-east-1"
  version                = data.ec_stack.latest.version
  deployment_template_id = "aws-io-optimized-v2"

  # Use the deployment template defaults
  elasticsearch = {
    hot = {
      autoscaling = {}
    }

    ml = {
       autoscaling = {
          autoscale = true
       }
    }

  }

  kibana = {
    topology = {}
  }
}

Developer Requirements

Installing the provider via the source code

Clone the repository to a folder on your machine and run make install:

$ mkdir -p ~/development; cd ~/development
$ git clone https://github.com/elastic/terraform-provider-ec
$ cd terraform-provider-ec
$ make install

Generating an Elasticsearch Service (ESS) API Key

To generate an API key, follow these steps:

  1. Open your browser and navigate to https://cloud.elastic.co/login.
  2. Log in with your email and password.
  3. Click on Elasticsearch Service.
  4. Navigate to Features > API Keys and click on Generate API Key.
  5. Choose a name for your API key.
  6. Save your API key somewhere safe.

Using your API Key on the Elastic Cloud terraform provider

After you've generated your API Key, you can make it available to the Terraform provider by exporting it as an environment variable:

$ export EC_API_KEY="<apikey value>"

After doing so, you can navigate to any of our examples in ./examples and try one.

Moving to TF Framework and schema change for ec_deployment resource.

v0.6.0 contains migration to TF Plugin Framework and intoduces new schema for ec_deployment resource:

resource "ec_deployment" "defaults" {
  name                   = "example"
  region                 = "us-east-1"
  version                = data.ec_stack.latest.version
  deployment_template_id = "aws-io-optimized-v2"

  elasticsearch = {
    hot = {
      autoscaling = {}
    }
  }

  kibana = {
    topology = {}
  }

  enterprise_search = {
    zone_count = 1
  }
}

  elasticsearch {
    topology {
      id         = "hot_content"
      size       = "1g"
      autoscaling {
        max_size = "8g"
      }
    }
    topology {
      id         = "warm"
      size       = "2g"
      autoscaling {
        max_size = "15g"
      }
    }
  }

has to be converted to

  elasticsearch = {
    hot = {
      size = "1g"
      autoscaling = {
        max_size = "8g"
      }
    }

    warm = {
      size = "2g"
      autoscaling = {
        max_size = "15g"
      }
    }
  }


resource "ec_deployment" "defaults" {
  name                   = "example"
  region                 = "us-east-1"
  version                = data.ec_stack.latest.version
  deployment_template_id = "aws-io-optimized-v2"

  elasticsearch = {
    hot = {
      autoscaling = {}
    }
  }
}

Please note that the snippet explicitly mentions hot tier with autoscaling attribute even despite the fact that they are empty.

Moving to the provider v0.6.0.

The schema modifications means that a current TF state cannot work as is with the provider version 0.6.0 and higher.

There are 2 ways to tackle this

Currently the state upgrade functionality is not implemented so importing existing resources is the recommended way to deal with existing TF states. Please mind the fact that state import doesn't import user passwords and secret tokens that can be the case if your TF modules make use of them. State upgrade doesn't have this limitation.

Known issues of moving to the provider v0.6.0