Quickstart: Stream Arbitrum data with Firehose and Substreams

Firehose and SubstreamsCopy Anchor Link

Firehose and Substreams are open-source blockchain data technologies maintained by StreamingFast and Pinax.

Firehose extracts blocks once and persists them as flat files, then streams them to consumers over gRPC. Because the same files serve both historical and live reads, one integration covers backfill and tail — you don't pair a websocket subscription with batched eth_getLogs calls for history.

Substreams runs on top of Firehose. You write modules in Rust, compile them to WebAssembly, and the server executes them in parallel across historical block ranges, then switches to live as it reaches the chain head. Output can be streamed to a consumer, written to PostgreSQL or ClickHouse, or used as a subgraph data source.

The difference from a JSON-RPC indexing loop is where the work happens. Decoding runs server-side, next to the data, and parallelizes across ranges — so a backfill is bounded by how much parallelism the server allocates rather than by your request rate.

Arbitrum endpointsCopy Anchor Link

Both are gRPC endpoints on port 443 and require an API key. Arbitrum Nova is not currently served.

Blocks for both networks come from Nitro nodes running the Firehose execution tracer, so they carry full execution detail — call traces, internal transfers, balance changes, and storage diffs — rather than only what a JSON-RPC poller can observe. Modules that depend on state changes work as a result; the example below is one of them.

PrerequisitesCopy Anchor Link

• The Substreams CLI
• A free Pinax API key from app.pinax.network

AuthenticateCopy Anchor Link

Substreams reads its key from the SUBSTREAMS_API_KEY environment variable:

export SUBSTREAMS_API_KEY="<your-api-key>"

If you call Firehose directly over gRPC rather than through Substreams, pass the key in an X-Api-Key header instead.

Run a Substreams packageCopy Anchor Link

Substreams packages are distributed as .spkg files, so you can run one without writing any Rust. Inspect a package first to see its modules and the shape of their output:

substreams info https://github.com/streamingfast/substreams-erc20-balance-changes/releases/download/v1.4.0/erc20-balance-changes-v1.4.0.spkg

Then stream it against Arbitrum One. This follows ERC-20 balance changes from a thousand blocks behind the chain head:

substreams run \
  -e arbone.substreams.pinax.network:443 \
  https://github.com/streamingfast/substreams-erc20-balance-changes/releases/download/v1.4.0/erc20-balance-changes-v1.4.0.spkg \
  map_balance_changes \
  --start-block -1000

A negative --start-block is resolved by the server relative to the chain head. The command prints each block's decoded output as JSON, then follows the head until you stop it. An invalid or missing API key fails here, before any streaming starts.

To process a fixed historical range instead, give absolute bounds. With a positive start block, a + prefix on --stop-block means "relative to the start block":

substreams run \
  -e arbone.substreams.pinax.network:443 \
  https://github.com/streamingfast/substreams-erc20-balance-changes/releases/download/v1.4.0/erc20-balance-changes-v1.4.0.spkg \
  map_balance_changes \
  --start-block 250000000 \
  --stop-block +10000

Browse substreams.dev for other packages.

Build a module for your own contractCopy Anchor Link

To decode your own contract's events and calls, initialize a project:

substreams init

The CLI walks you through choosing a code generator, the network, and your contract address, and generates a Rust module from the contract's ABI. Build it and run it against the same endpoint:

substreams build
substreams run -e arbone.substreams.pinax.network:443 substreams.yaml map_events --start-block -1000

Use the module name that substreams init generated — substreams info substreams.yaml lists them.

The Substreams developer documentation covers module types, stores for stateful aggregation, and the Rust API in full.

Consume Firehose directlyCopy Anchor Link

If you want raw blocks rather than a module's output, stream Firehose over gRPC. Each block arrives as a protobuf message with transaction traces, logs, and balance changes already extracted:

grpcurl \
  -H "X-Api-Key: $PINAX_API_KEY" \
  -d '{"start_block_num": -10, "final_blocks_only": false}' \
  arbone.firehose.pinax.network:443 \
  sf.firehose.v2.Stream/Blocks

Client libraries are available in Go, Rust, and TypeScript. See the Firehose documentation for the block model and for cursors, which are how you resume a stream exactly where it stopped after a restart or a reorg.

Sink the outputCopy Anchor Link

Substreams output is designed to be written somewhere rather than read in a terminal:

• Substreams:SQL — PostgreSQL and ClickHouse
• Files — Parquet, JSONL, CSV
• Substreams:Stream — consume the stream directly from your own service

The Consuming Substreams overview lists the rest, including community sinks.

Next stepsCopy Anchor Link

• Substreams documentation — modules, stores, and the Rust API
• Firehose documentation — block model and cursors
• substreams.dev — the package registry
• Pinax documentation — endpoints, authentication, and limits