Yesterday, at 18:35:29 UTC (2:35pm ET), our primary live feed handler for Nasdaq TotalView (XNAS.ITCH) had a network gap during a microburst on the feed. Recovery attempts failed under high load, causing missed packets and triggering the F_MAYBE_BAD_BOOK flag on the MBO schema.
Under normal circumstances, we would promote our secondary servers to handle new connections, and MBO users would've been able to reconnect with the snapshot=1 parameter to pick up the current state of the book from the new healthy server. However, a monitoring issue earlier in the day prevented us from bringing the secondary server online during trading hours. We've since fixed the monitoring gap and are addressing the root cause by allocating more resources to our recovery system. In the longer term, we’ve scheduled enhancements to the Nasdaq TotalView feed handler, similar to the OPRA improvements released last month.