The issue has been identified and a fix is being implemented.
Posted Jan 18, 2025 - 00:16 UTC
Update
We are continuing to investigate this issue.
Posted Jan 17, 2025 - 19:56 UTC
Investigating
Users may receive internet write failures on collectors stating: "Exporting failed. Will retry the request after interval. {"kind": "exporter", "data_type": "traces", "name": "otlp/cloud", "error": "rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 502 (Bad Gateway)"
The agent/collector will automatically retry the write attempt and succeed, so there is no loss in data collection.
Posted Jan 17, 2025 - 19:56 UTC
This incident affects: Grafana Cloud: Tempo (AWS Australia - prod-au-southeast-1: Ingestion, AWS Brazil - prod-sa-east-1: Ingestion, AWS Canada - prod-ca-east-0: Ingestion, AWS Germany - prod-eu-west-2: Ingestion, AWS India - prod-ap-south-1: Ingestion, AWS Japan - prod-ap-northeast-0: Ingestion, AWS Singapore - prod-ap-southeast-1: Ingestion, AWS Sweden - prod-eu-north-0: Ingestion, AWS US East - prod-us-east-0: Ingestion, AWS US West - prod-us-west-0: Ingestion, Azure Germany - prod-eu-west-1: Ingestion, Azure Netherlands - prod-eu-west-3: Ingestion, Azure US Central - us-central2: Ingestion, GCP Australia - prod-au-southeast-0: Ingestion, GCP Belgium - prod-eu-west-0: Ingestion, GCP Brazil - prod-sa-east-0: Ingestion, GCP India - prod-ap-south-0: Ingestion, GCP Singapore - prod-ap-southeast-0: Ingestion, GCP UK - prod-gb-south-0: Ingestion, GCP US Central - prod-us-central-0: Ingestion).