Ingestion problems - Hosted Metrics US East

Incident Report for Grafana Cloud

Resolved

One of four ingest nodes failed but wasn't correctly removed from service. This lead to 25% of requests to continue to be routed to the down node, resulting in a partial service disruption for sending and query metrics.
After identifying the issue, engineers manually marked the node as failed and service promptly returned to normal. The total service interruption time was 14 minutes.

Posted Mar 30, 2018 - 18:11 UTC

Investigating

We are currently investigating this issue.

Posted Mar 30, 2018 - 17:47 UTC