Skip to content

SIEM push recipes (Datadog, Grafana, Sentinel)

Point LumenFlow's per-organisation OpenTelemetry collector push at Datadog, Grafana Cloud, or Microsoft Sentinel — one OTLP destination, three destination recipes.

What this configures#

LumenFlow can push governed-action evidence spans to an OpenTelemetry collector URL your organisation configures. Spans are posted as OTLP/HTTP JSON to an HTTPS /v1/traces endpoint you choose. Delivery retries on transient failures and lands exhausted attempts in a dead-letter queue that a scheduled drain retries later. Destination URLs must be public HTTPS (private and loopback addresses are refused). A destination outside your organisation's pinned residency region is refused, and a compliance pack that forbids evidence egress blocks configuration entirely.

Organisation admins configure the destination with:

PUT /api/v1/orgs/<orgId>/evidence-egress

{
  "kind": "otlp",
  "endpointUrl": "https://collector.example/v1/traces",
  "headers": { "Authorization": "…" },
  "destinationRegion": "eu_west",
  "status": "active"
}

destinationRegion must be one of eu_west, us_east, us_west, or apac. Outbound auth headers are stored encrypted and never returned in list responses. List destinations with GET on the same path; remove one with DELETE ?kind=otlp.

The payload is the closed evidence-span field set only — action name, honest-scope fields, and opaque identifiers such as receipt and workspace ids — never prompts, tool arguments, or credentials. For the standards-portable shape of that evidence, see Standards-portable evidence (OpenTelemetry).

A separate Splunk HTTP Event Collector destination kind is also available on the same API (kind: "splunk_hec"). This article covers OTLP destinations only.

Recipe: Datadog#

  1. In Datadog, create an API key and note which Datadog site your organisation uses.
  2. Use Datadog's OTLP traces intake URL for that site — for example https://otlp.datadoghq.com/v1/traces (US1) or https://otlp.datadoghq.eu/v1/traces (EU). Confirm the exact URL in Datadog's current OTLP intake documentation for your site.
  3. Configure LumenFlow with kind: "otlp", that HTTPS URL as endpointUrl, header dd-api-key set to your Datadog API key, and a destinationRegion that matches where the intake lives.
  4. If Datadog's intake for your organisation requires protobuf-only or an allowlisted source header your key does not yet carry, place any OTLP/HTTP collector you control on a public HTTPS URL in front: point LumenFlow at that collector, and let the collector re-export to Datadog.

Recipe: Grafana Cloud#

  1. In Grafana Cloud, open your stack's OpenTelemetry tile and copy the OTLP gateway URL, numeric instance ID, and an access-policy token with traces write permission.
  2. Build the full traces URL as https://otlp-gateway-<region>.grafana.net/otlp/v1/traces (use the host Grafana shows for your stack — the region segment varies).
  3. Configure LumenFlow with kind: "otlp", that URL as endpointUrl, and an Authorization header using HTTP Basic auth: username = instance ID, password = the token (send Authorization: Basic <base64(instanceId:token)>).
  4. Set destinationRegion to the Grafana Cloud region that hosts the gateway. Grafana Cloud's OTLP gateway accepts OTLP/HTTP JSON on that path.

Recipe: Microsoft Sentinel#

  1. In Azure, enable OpenTelemetry intake for the Log Analytics workspace your Microsoft Sentinel instance uses (Application Insights OTLP connection info, or a Data Collection Endpoint plus Data Collection Rule with the Microsoft OTLP traces stream). Copy the traces OTLP URL Azure shows — it ends in /otlp/v1/traces.
  2. Azure Monitor authenticates OTLP with a short-lived Microsoft Entra bearer token. LumenFlow stores static outbound headers and does not refresh Entra tokens, so run a small OTLP/HTTP collector you control on a public HTTPS URL: that collector refreshes the Entra token and forwards to Azure's traces URL; LumenFlow points at your collector.
  3. Configure LumenFlow with kind: "otlp", your collector's HTTPS /v1/traces URL as endpointUrl, any headers your collector requires, and a destinationRegion matching the Azure region of the Data Collection Endpoint.
  4. In Microsoft Sentinel, query the connected workspace for the OpenTelemetry spans your collector forwarded — the exact table names follow Azure Monitor's OTLP intake (for example OTelSpans), not a LumenFlow-specific schema.

What this does not do#

  • It does not invent a native Datadog, Grafana, or Sentinel transport — every recipe uses the same OTLP destination kind.
  • It does not export message content, prompts, tool arguments, or secrets.
  • It does not bypass residency pins or a compliance pack that forbids evidence egress.

info See Standards-portable evidence (OpenTelemetry) for the portable evidence shape, Independently verifiable evidence for receipt verification, and Trust Centre for residency and compliance controls that gate this egress.