Long running syncs

May 26, 2023 at 11:02 PM
Resolved after 4 days


Partial outage
Data Sync
  • Resolved

    After monitoring the fix implemented we have resolved the issue.

    The issue occurred under certain conditions and caused syncs to to not be properly retried.

    Thank you for your patience as we investigated this issue and we apologize for any inconvenience.

  • Monitoring

    We believe we have found the root cause and have implemented a fix. We will be monitoring the results.

  • Investigating

    We are still seeing instances of these long running syncs and failures with "Setting attempt to FAILED because the temporal workflow for this connection was restarted, and existing job state was cleaned.". We are continuing our investigation and will add updates as we have them.

  • Monitoring

    We have been monitoring the results over the weekend and would like to continue monitoring throughout the day.

  • Monitoring

    We implemented a fix and are currently monitoring the result. Any sync affected will show as a failed sync but the next scheduled sync should run as expected.

  • Identified

    We have identified an issue that is causing syncs to run longer than expected. We are taking action to address these long running syncs.

    Thank you for your patience as we work to resolve the issue.