Rule Performance Degradation

Incident Report for Grafana Cloud

Resolved

There were no observed instances of this issue today. We're considering the incident resolved. Please contact our Support team if you believe you're impacted by this incident.
Posted Jul 22, 2024 - 21:46 UTC

Monitoring

Systems are stable again and we are continuing to monitor.

We will provide another status update if there are any changes.
Posted Jul 20, 2024 - 00:25 UTC

Update

If you are experiencing queries taking longer to complete than expected, please cancel and retry them.

We will continue to investigate and provide updates as more information becomes available.
Posted Jul 19, 2024 - 23:18 UTC

Update

Around 17:00 UTC we had noticed an impact that we are currently investigating with degraded performance with rule evaluation in the following clusters:
us-central-0/loki-prod
us-central-5/loki-prod-017
us-central-5/loki-prod3

We will continue to investigate and provide updates as more information becomes available.
Posted Jul 19, 2024 - 22:30 UTC

Investigating

We are currently investigating an issue with degraded performance with rule evaluation in us-central-0.loki.
We will continue to investigate and provide updates as more information becomes available.
Posted Jul 19, 2024 - 22:06 UTC
This incident affected: Grafana Cloud: Loki (GCP US Central - prod-us-central-0, GCP US Central - prod-us-central-5).