Onboard observability data

Use an OpenTelemetry Collector in gateway mode so applications send to a customer-operated collector that can batch, retry, and control egress. This procedure applies only when the deployment's Elasticsearch version supports the OTLP/HTTP endpoint.

Applications send telemetry to a customer-operated Collector. The Collector uses the deployment-provided HTTPS endpoint; telemetry remains in the customer's Azure-hosted managed platform.

Choose the supported intake path

Do not treat all Elastic OpenTelemetry endpoints as interchangeable:

Path	When to use it
Direct Elasticsearch OTLP/HTTP	Use this tutorial only after confirming the deployed Elasticsearch version exposes /_otlp/v1/logs, /_otlp/v1/metrics, and /_otlp/v1/traces
Customer-operated Collector	Recommended gateway for batching, retry, egress control, and limiting application connection churn
APM Server OTLP intake	Use only when the deployment explicitly exposes and supports an APM Server endpoint; its protocol and paths differ
Elastic Agent and Fleet	Use when Fleet and the required integration are exposed and compatible with the deployed Elastic version

The direct Elasticsearch endpoint accepts OTLP/HTTP with protobuf encoding. It does not accept OTLP/gRPC. Do not send OTLP to Kibana, omit the /_otlp base path, or guess that a Cloud-hosted or APM endpoint exists.

Before you begin

Confirm:

• the deployment-derived Elasticsearch URL;
• OTLP/HTTP support for the deployed version;
• TCP 443, DNS, and TLS trust from the collector;
• an administrator who can create a scoped API key;
• a non-production source for the first validation;
• ownership for collector availability, buffering, and upgrades.

Create a least-privilege API key

Set the endpoint without credentials:

export ELASTICSEARCH_URL="https://<deployment-elasticsearch-hostname>"

Using an approved administrator credential, create an expiring onboarding key:

curl --fail-with-body --user "<api-key-administrator>" \
  --request POST "${ELASTICSEARCH_URL}/_security/api_key" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "otel-gateway",
    "expiration": "30d",
    "role_descriptors": {
      "otel_writer": {
        "indices": [
          {
            "names": ["logs-*", "metrics-*", "traces-*"],
            "privileges": ["auto_configure", "create_doc"]
          }
        ]
      }
    }
  }'

Enter the administrator password through the client prompt. Store the returned encoded API key in the approved secret store; do not place it in a file, screenshot, shell history, or ticket.

Configure the Collector

Use environment substitution for the endpoint and secret:

receivers:
  otlp:
    protocols:
      http:

processors:
  batch: {}
  memory_limiter:
    check_interval: 1s
    limit_mib: 512

exporters:
  otlphttp/elastic:
    endpoint: ${env:ELASTICSEARCH_URL}/_otlp
    headers:
      Authorization: "ApiKey ${env:ELASTIC_API_KEY}"

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/elastic]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/elastic]
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/elastic]

Pin a Collector version supported by your organization. Add persistent buffering, high availability, resource limits, and telemetry for the Collector before production use.

The exporter appends /v1/logs, /v1/metrics, or /v1/traces to the /_otlp base path. Follow the Elastic Elasticsearch OTLP/HTTP endpoint guidance for the deployed version rather than copying configuration for Elastic Cloud Managed OTLP or APM Server.

Verify and hand over

1. Send one synthetic log, metric, and trace from a test source.
2. Confirm the Collector reports successful export without retry or TLS errors.
3. In Kibana, confirm logs-*, metrics-*, and traces-* data streams appear only for the signals sent.
4. Verify timestamps, service identity, environment, and sensitive-field handling.
5. Test Collector restart and destination unavailability behavior.
6. Record the API-key owner and rotation date.

Revoke the test key if the onboarding is abandoned. For endpoint or TLS failures, follow connectivity troubleshooting.