Flow Mainnet Forte Network Upgrade Retrospective
On October 22, 2025, the Flow Mainnet underwent a planned network upgrade. While the upgrade was completed successfully, the process took longer than expected due to several technical issues encountered during execution.
Below is a summary of the main findings and follow-up actions from the retrospective:
- Upgrade Timing and Sequencing: Flow Foundation Access nodes were stopped earlier than intended without checking their latest block height, leading to missing blocks that were required for the Flow Foundation’s EVM Gateways to serve historical queries. Future upgrades will follow on-chain height-coordinated shutdown procedure to ensure all nodes capture the complete block range before shutting down.
- Flow Foundation Automation Validation: Certain Flow Foundation automation scripts failed to execute as expected due to configuration inconsistencies. New checks will be added to ensure such configuration inconsistencies are easily identified and resolved. This includes checks for duplicate node entries when generating the genesis data - this error was flagged by the automation but it took the team some time to comprehend and resolve. Additionally, Flow engineering team will expand on the dry-run process which is executed during the days leading up to the network upgrade, to be more comprehensive by including all nodes and not just the foundation nodes.
- Partner Automation Improvements: Partner node startup issues were traced to missing or incorrect parameters in startup procedures and to incorrect Foundation issued standard operating procedure (SOP). Partner automation and SOP will be reviewed step-by-step and a dry-run will be performed before future network upgrades to ensure that nodes can be started successfully.
- Communication and Coordination: Parallel troubleshooting during the upgrade led to gaps in communication between teams. While partners can run separate channels for deep-diving into issues, a point-of-contact/incident commander will provide updates to the main coordination channel to ensure timely visibility into progress and issues.
The Flow network has been operating normally since the upgrade. These improvements will help ensure future network updates are more predictable and resilient.