Skip to main content

Incident Report: Linea Sequencer Update

Date: 2024-01-22
Time: 15:44 (GMT+3)
Duration: Ongoing

Description​

An incident has been identified in Centurion, where the heartbeat exceeded for CAD/USD on the Linea chain, and ETH/USD has deviated, remaining unupdated.

Root Cause​

The root cause appears to be related to recent improvements/updates on the Linea chain itself. Linea transitioned their sequencer from geth to besu, deploying a new besu-powered sequencer. This change seems to have led to transaction issues, causing deviations and pending transactions on various contracts.

Impact​

Pending Airseeker transactions.

Timeline​

  • 15:44 - Abdel has noticed the deviation on ETH/USD and heartbeat exceedance on CAD/USD, both on Linea.
  • 15:58 - The investigation has been initiated by Aaron.
  • 16:04 - Observed multiple pending transactions.
  • 16:08 - Ugur and Bedirhan have started to investigate.
  • 16:51 - Contacted Linea for further clarification by Bedirhan and found out that the problem originates from their side.
  • 15:54 - Burak has been told that there are no updates yet.

Lessons Learned:​

Changes in the underlying infrastructure, such as sequencer transitions, can impact transaction processing.
Clear communication and coordination are crucial when making significant updates to the chain.

Actions Taken​

  • Increased gas prices for pending transactions.
  • Contacted Linea for clarification and assistance.
  • Monitored additional contracts for similar issues.
  • Ongoing investigation into potential sequencer blocking.

Escalation link.

Incident Reviewer(s)​

Abdel, Aaron, Ugur, Bedirhan, Burak

Last updated on by Warren Anderson