How to set up a high-availability sequencer

note

This documentation is for production sequencer deployments. If you want to set up a sequencer for testing or on a testnet, see How to run a testnet sequencer node.

Introduction

The sequencer is a critical component of your Arbitrum chain and is responsible for queuing transactions submitted to the network. It serves as the transaction ordering engine, accepting transactions forwarded from full nodes, queuing them, and returning feed messages to those full nodes. It subsequently sends queued transactions to batch posters for data posting.

If your sequencer goes offline, the network cannot process new transactions arriving at the child chain's RPC nodes, impacting the user experience. This guide provides detailed instructions for setting up a high-availability (HA) sequencer architecture to minimize downtime and ensure your Arbitrum chain remains operational even if individual components fail.

Prerequisites

Before you begin, ensure you have:

• Experience with Kubernetes and container orchestration
• Access to a Kubernetes cluster with multiple availability zones
• Understanding of Redis and cloud infrastructure
• A properly configured parent chain node or RPC node endpoint
• Sequencer keys and permissions
• Sufficient storage and compute resources

High-availability sequencer architecture

A high-availability sequencer deployment consists of seven key components:

1. CDN/Load Balancer: Manages traffic routing and bot detection
2. Nitro full nodes: Process read requests and forward write transactions
3. External Relays: Handle public feed traffic
4. Sequencer Relays: Combine feeds from all sequencers
5. Sequencers: Multiple redundant transaction queuing machines
6. Redis: Coordinates active sequencer selection and sequencer-related information sharing
7. Batch Poster: Posts transaction batches to the parent chain

Architecture diagrams

The architecture varies slightly depending on whether your full nodes use Redis to identify the active sequencer or forward transactions to a predefined endpoint.

Send to active enabled

In this configuration, full nodes query Redis to determine the active sequencer and forward transactions directly to it:

Send to active disabled

In this configuration, full nodes forward transactions to a predefined endpoint without checking which sequencer is active:

Using Helm charts for deployment

We recommend using the Offchain Labs community Helm charts to deploy your high-availability sequencer setup. These charts provide pre-configured templates for all the necessary components and make it easier to maintain your deployment. Base configuration values are provided in the examples below. However, you should adjust them to fit your needs and use values files for production deployments.

Detailed component setup

1. Load balancing (CDN)

We strongly recommend using a CDN for managing traffic and security concerns:

• Recommendation: Use Cloudflare or a similar CDN service
• Configuration:
  • Direct RPC traffic to the Nitro full node fleet
  • Direct feed traffic to public-facing relays
  • Implement rate limiting and bot detection as needed
• Benefits: Distributes load, improves security, and enhances availability

2. Relays setup

The architecture requires two types of relays:

Sequencer relays

• Deployment should have multiple replicas
• Configure to listen to feed outputs from all sequencers
• All other components connect to these relays instead of directly to the sequencers
• Minimize direct load on sequencer nodes

Deploy sequencer relays using the relay Helm chart:

helm install sequencer-relay offchainlabs/relay \
  --set replicaCount=2 \
  --set configmap.data.chain.id=<your-chain-id> \
  --set configmap.data.node.feed.input.url=ws://sequencer-nitro-0.sequencer-nitro:9642,ws://sequencer-nitro-1.sequencer-nitro:9642,ws://sequencer-nitro-2.sequencer-nitro:9642

Key configuration parameters:

• replicaCount: Number of relay replicas to deploy (recommend at least two for high availability)
• configmap.data.chain.id: Your chain ID
• configmap.data.node.feed.input.url: Comma-separated list of WebSocket URLs for all sequencer feed outputs. This relies on the perReplicaHeadlessService.enabled=true parameter in the sequencer deployment to create individual services for each sequencer replica.

External relays

• Connect to Sequencer Relays (not directly to sequencers)
• Handle all public feed requests
• Provide an additional layer of isolation for production sequencers
• Since these are public-facing, ensure they scale appropriately based on your traffic needs:

helm install external-relay offchainlabs/relay \
  --set replicaCount=2 \
  --set configmap.data.chain.id=<your-chain-id> \
  --set configmap.data.node.feed.input.url=ws://sequencer-relay:9642

You can check run a feed relay to see how to set up a relay node.

3. Nitro full node setup

helm install fullnode offchainlabs/nitro \
  --set replicaCount=2 \
  --set configmap.data.parent-chain.id=<parent-chain-id> \
  --set configmap.data.parent-chain.connection.url=<parent-node-url> \
  --set configmap.data.chain.id=<child-chain-id> \
  --set configmap.data.execution.forwarding-target=http://sequencer-nitro:8547

Send to active configuration (optional)

To enable Redis-based active sequencer discovery:

• Monitor Redis to identify the active sequencer
• Enable with: -execution.forwarder.redis-url=redis://<redis-url>:6379
• Ensure connectivity to individual sequencer services and Redis
• Test failover scenarios before production deployment

helm install fullnode offchainlabs/nitro \
  --set replicaCount=2 \
  --set configmap.data.parent-chain.id=<parent-chain-id> \
  --set configmap.data.parent-chain.connection.url=<parent-node-url> \
  --set configmap.data.chain.id=<child-chain-id> \
  --set configmap.data.execution.forwarder.redis-url=redis://<redis-url>:6379

Mutating-only endpoint (optional)

For high availability, we recommend routing mutating transactions to a fleet of precheckers, which forward them to the sequencer. To do this, set up a separate endpoint for mutating transactions and allow only calls such as eth_sendRawTransaction to route to the precheckers, which then route to the sequencer. This insulates the sequencer from unnecessary load and enables it to focus on transaction ordering. However, this configuration requires custom load-balancing logic and is out of scope for this guide.

4. Redis setup

Set up a highly available Redis cluster for sequencer coordination:

Deployment options

• Use a managed service like AWS ElastiCache (recommended)
• Deploy within Kubernetes using a StatefulSet with PersistentVolumeClaims

Requirements

• Minimum of three replicas across different availability zones (recommended)

• Secured access (only accessible within the Kubernetes cluster)

• Backups enabled

• Configuration:

  • Use a Redis cluster or Redis Sentinel for high availability
  • Secure the endpoint with proper network policies
  • Monitor Redis health as part of your overall monitoring strategy

5. Sequencer setup

Deploy multiple sequencer replicas with availability zone spread using the Nitro Helm chart (availability zone spread is not demonstrated in the example below):

helm install sequencer offchainlabs/nitro \
  --set replicaCount=3 \
  --set configmap.data.parent-chain.id=<parent-chain-id> \
  --set configmap.data.parent-chain.connection.url=<parent-node-url> \
  --set configmap.data.chain.id=<child-chain-id> \
  --set configmap.data.node.sequencer=true \
  --set configmap.data.node.delayed-sequencer.enable=true \
  --set configmap.data.node.seq-coordinator.enable=true \
  --set configmap.data.node.seq-coordinator.redis-url=<redis-url> \
  --set configmap.data.node.feed.output.enable=true \
  --set configmap.data.node.feed.output.port=9642 \
  --set configmap.data.execution.sequencer.enable=true \
  --set perReplicaHeadlessService.enabled=true

Critical sequencer coordinator parameters

The sequencer coordinator is the key component for high availability. These parameters are essential:

Parameter	Description	Recommended Value
node.seq-coordinator.enable	Enable sequencer coordinator	true
node.seq-coordinator.redis-url	Redis URL for coordination	Your Redis URL
node.seq-coordinator.my-url	URL for this sequencer	Unique per sequencer

For the full set of coordinator flags and their defaults, see Sequencer configuration reference.

Redis priority registration

Configuring redis-url and my-url is not enough to make a sequencer eligible. Each sequencer must also appear in the Redis priority list, stored under the key coordinator.priorities as a comma-separated list of URLs in priority order.

Nitro treats this key as read-only. No sequencer can write itself into it. You populate it with the Sequencer Coordination Manager (SQM), which is the only component that writes the key.

A sequencer that is not in the list starts cleanly, syncs, serves RPC, and broadcasts to the feed—but never becomes the active sequencer.

How the coordinator picks the active sequencer:

1. It reads coordinator.priorities from Redis.
2. It goes through the list in order and takes the first sequencer that has published a liveliness key at coordinator.liveliness.<my-url>. Each sequencer writes its own key when it is synced and ready.
3. That sequencer acquires the lockout, recorded at coordinator.chosen, and activates. Everything else forwards to it.

Two consequences follow from step 2:

• Order is priority. The first entry that is healthy and synced wins. Put your preferred sequencer first.
• my-url must match its list entry exactly. The comparison is a string match. A trailing slash, a different port form, or an IP address instead of a hostname all cause a silent mismatch.

Never register the batch poster

The batch poster runs with node.seq-coordinator.enable=true so it can follow coordination state, but it must stay out of coordinator.priorities. If it reaches the top of the list, it is selected as the active sequencer while having no sequencer to run, and the chain stops producing blocks. Nitro logs myurl main sequencer, but no sequencer exists.

Verifying registration

Read the key directly:

redis-cli -u <redis-url> GET coordinator.priorities

Compare each URL against the node.seq-coordinator.my-url of the sequencer it should match. Then check which sequencer currently holds the lockout, and which ones report themselves ready:

redis-cli -u <redis-url> GET coordinator.chosen
redis-cli -u <redis-url> --scan --pattern 'coordinator.liveliness.*'

A sequencer that appears in coordinator.priorities but has no matching coordinator.liveliness.<my-url> key is not eligible. It is either unsynced or its my-url does not match its list entry.

Two log messages tell you the priority list is the problem:

• sequencer priorities unset — the key does not exist. This is expected on a new Redis instance and means no sequencer can ever be chosen until you populate it.
• no sequencer appears to want the lockout on redis — the key exists, but nothing on it is eligible. The log includes the priorities value, so compare it against your running sequencers. This starts at DEBUG, escalates to WARN after 10 seconds and ERROR after 20.

For a full diagnostic path, see Sequencer troubleshooting.

Redis persistence

The priority list lives only in Redis. If you replace the Redis instance or lose its data, the key is gone and no sequencer becomes active until you repopulate it. Include coordinator.priorities in your backup and disaster recovery plan, and re-register after any Redis migration.

Setting the sequencer's self URL

A critical configuration for the sequencer coordinator is setting a unique URL for each sequencer instance. This configuration can be adjusted using Kubernetes environment variables:

extraEnv:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: NITRO_NODE_SEQ__COORDINATOR_MY__URL
    value: 'http://$(POD_NAME).<NAMESPACE>.svc.cluster.local:8547/rpc'

This configuration:

1. Gets the pod name from Kubernetes metadata
2. Uses it to create a unique URL for each sequencer instance
3. Sets this URL as the node.seq-coordinator.my-url parameter

Adjust the domain name (<NAMESPACE>.svc.cluster.local) to match your Kubernetes cluster's DNS configuration. This configuration requires Nitro to be configured to read environment variables beginning with NITRO_.

Individual sequencer services

You can enable the automatic creation of headless services for each sequencer replica by setting the perReplicaHeadlessService.enabled=true parameter in the Helm chart (as shown in the installation command above). This configuration creates individual services named <release-name>-nitro-<index> that allow direct access to each sequencer replica.

These individual services are critical for a proper high-availability setup because they allow components like sequencer relays to connect directly to specific sequencer instances. This direct connection is essential for:

• Proper failover: When the active sequencer changes, other components can address the new active sequencer directly
• Feed aggregation: Sequencer relays need to collect feeds from all sequencer instances to ensure no messages get lost during transitions

6. Sequencer Coordination Manager

To manage active sequencer selection, use the built-in sequencer coordinator UI:

• Follow detailed instructions at: How to run a Sequencer Coordination Manager (SQM)
• Use this interface to manually switch between sequencer replicas when needed
• Configure permissions and access controls appropriately

7. Batch poster setup

Deploy the batch poster using the Nitro Helm chart:

helm install batchposter offchainlabs/nitro \
  --set configmap.data.parent-chain.id=<parent-chain-id> \
  --set configmap.data.parent-chain.connection.url=<parent-node-url> \
  --set configmap.data.chain.id=<child-chain-id> \
  --set configmap.data.execution.forwarding-target=null \
  --set configmap.data.node.seq-coordinator.enable=true \
  --set configmap.data.node.seq-coordinator.redis-url=<redis-url> \
  --set configmap.data.node.batch-poster.enable=true \
  --set "configmap.data.node.batch-poster.parent-chain-wallet.private-key=<your-private-key>"

Important

Do not add the batch poster to the sequencer priority list in the Sequencer Coordination Manager (SQM) to prevent it from becoming the active sequencer unintentionally.

Monitoring and maintenance

Health checks

Implement comprehensive health checks for all components:

• Sequencer health: Monitor the sequencer's logs and metrics
• Redis connectivity: Ensure all components can access Redis
• Feed availability: Verify feed connectivity between components
• Transaction processing: Monitor end-to-end transaction flow

Troubleshooting

If you run into any issues, visit the node-running troubleshooting guide.

References

• How to run a full node
• How to run a Sequencer Coordination Manager (SQM)
• Kubernetes Documentation
• Offchain Labs Community Helm Charts