Airbyte - Sync Failures in EU region – Incident details

All systems operational

Sync Failures in EU region

Resolved
Partial outage
Started over 1 year agoLasted about 9 hours

Affected

Data Sync

Partial outage from 7:58 AM to 5:08 PM

Updates
  • Resolved
    Resolved

    This issue has been resolved. If your most recent sync failed, we expect the next sync to succeed.

    We apologize for the inconvenience and thank you for your patience as the team worked to resolve the issue.

  • Monitoring
    Monitoring

    We have updated our networking stack to workaround the flooded Paris GCP datacenter. Our EU syncs can now pull secrets from a healthy datacenter which should fix the issue with our EU syncs. We have verified that EU syncs are currently working. If you had a failed sync starting it again now should work. We are continuing to monitor the incident.

  • Update
    Update

    We are continuing to work on a fix for this incident.

  • Identified
    Identified

    We have identified the root cause and are working on the resolution.

    GCP’s Secret manager service powers part of the Airbyte EU Sync infrastructure. Because of the flooding in GCP’s Paris datacenter Airbyte’s EU Sync’s are unable to start.

    We are working on resolving those secrets from a functioning GCP EU region. Until we can set up that system syncs may be run successfully from the US for any customers not bound by GDPR.

  • Investigating
    Investigating

    Many syncs in the EU region are currently failing. They appear to be impacted by a general GCP outage in the Paris region https://status.cloud.google.com/incidents/dS9ps52MUnxQfyDGPfkY#73mBtVKKfeJGJ1yaY7hV

    We are currently monitoring the situation.