TL;DR
- Sui added gRPC streaming as the primary data source in its indexing infrastructure, enabling real-time ingestion of finalized checkpoints.
- The system allows full nodes to push checkpoints to indexers at the moment of finalization, replacing the periodic polling-based model.
- Sui’s architecture combines gRPC streaming as the primary path with polling sources as a fallback, ensuring historical backfill and clean restarts.
Sui added gRPC streaming as the primary data source within its indexing infrastructure, enabling real-time checkpoint ingestion with minimal latency. The update is already live in production and is part of the network’s official data stack, used by both custom indexers and the General-Purpose Indexer.
This new system allows full nodes to send checkpoints to indexers at the exact moment they are finalized. This mechanism replaces the traditional model based exclusively on polling, where systems had to periodically query whether new data was available. With gRPC streaming, data delivery becomes push-based and continuous.
It is worth clarifying that the architecture adopted by Sui does not eliminate polling. The design combines streaming as the primary path with mandatory polling sources as a fallback. Streaming only delivers data from the moment a connection is established, so polling is used for historical backfill and for recovery after interruptions, restarts, or temporary network failures.
Sui Will Operate With a Hybrid Indexing Framework
This hybrid approach is integrated into the Custom Indexing Framework. The framework separates the ingestion layer from the checkpoint processing logic. Indexers consume checkpoints through a unified interface, without coupling to a specific data source. As a result, the same pipeline can operate with gRPC streaming, HTTP polling, or both sources simultaneously.
Adopting gRPC streaming does not require changes to existing indexer logic. It can be enabled through a configuration update that adds a streaming-url parameter alongside the remote-store-url already used for polling. The framework automatically manages switching between sources, preserving continuity in indexed data.
A Synchronized, Fault-Tolerant System
Sui’s General-Purpose Indexer uses gRPC streaming as its primary channel to stay synchronized with the chain head. At the same time, it retains polling sources to ensure clean restarts and full recovery in the event of failures. This model allows a single modular indexer to operate without sacrificing consistency or availability.
Sui’s infrastructure targets latency-sensitive workloads such as real-time monitoring, analytics, alerts, and systems that depend on finalized data without delays. Batch flows, offline processes, and less demanding applications can continue operating with polling-only configurations.
Sui stated that the streaming-first model is part of a broader transition toward structured, composable, event-driven data interfaces. The goal is to reduce indexer operational complexity and standardize access to onchain data across the network.
