Cortex read/write partial outage

Incident Report for Grafana Cloud

Resolved

This incident has been resolved.
Posted Sep 26, 2025 - 14:29 UTC

Monitoring

The system has recovered. We are continuing to monitor closely while we investigate the underlying cause.
Posted Sep 26, 2025 - 10:03 UTC

Update

We are continuing to investigate this issue.
Posted Sep 26, 2025 - 09:53 UTC

Update

Recording rules and alerts evaluation are also affected.
We are further investigating this
Posted Sep 26, 2025 - 09:51 UTC

Investigating

We are currently seeing a partial outage in cortex-prod-10 and cortex-prod-06, for on both reads / writes.
We are currently investigating this issue.
Posted Sep 26, 2025 - 09:34 UTC
This incident affected: Metrics (GCP US Central - prod-us-central-0: Querying, GCP US Central - prod-us-central-0: Ingestion) and Alerting and SLO (GCP US Central - prod-us-central-0: Rule Evaluation).