DEV Community

Cover image for The Complete Guide to Running a Full Midnight Node: Setup, Sync & Monitoring
Uroy Nwankwo
Uroy Nwankwo

Posted on • Edited on

The Complete Guide to Running a Full Midnight Node: Setup, Sync & Monitoring

#blockchain #docker #tutorial #web3

If you've been building on Midnight or want to participate in the network without trusting third-party infrastructure, running your own full node is the right move. This tutorial takes you from a bare server to a healthy, synced node using the official Midnight node binary and the supporting Cardano infrastructure — Mithril, Cardano-node, Cardano-db-sync, and PostgreSQL — all managed through systemd. Along the way you'll learn how to monitor your sync progress, verify node health, and work through the failure modes that catch most people: dropped peers, getting stuck on block 1, and stale database connections.

Target audience: Developers and node operators comfortable with the Linux command line.

Prerequisites:

  • Ubuntu 24.04 LTS (bare metal or VPS) — required; the node binary targets GLIBC 2.39+
  • A non-root user named midnight with sudo privileges
  • A stable internet connection with inbound TCP port 30333 open
  • Outbound HTTPS access

What you'll have by the end: A full node synced to the Midnight network, running via systemd services, with monitoring commands and a tested troubleshooting playbook for common failures.

A note on the Docker repository. Earlier versions of this guide used the midnight-node-docker repository. The official approach has since changed. The current documentation installs native binaries and manages services with systemd. The Docker repository is no longer the official recommended path for running a node.

Understanding the node types

Before spinning anything up, it helps to know what you're actually running. Midnight has four node types:

  • Full node: syncs the chain, validates transactions, and provides real-time state queries. It prunes historical state older than 256 blocks by default, making it efficient on disk. This is what most developers and node operators need.
  • Archive node: identical to a full node but retains the entire chain history. Required for block explorers, historical analytics, or services that query past state at arbitrary heights. Uses substantially more storage.
  • RPC node: exposes an HTTP/WebSocket API for DApps to interact with the chain programmatically.
  • Boot node: helps new nodes discover peers. You don't need to run one unless you're contributing to network bootstrapping.

This tutorial focuses on the full node. Where the archive node diverges, this guide calls out the single flag you need to change.

Step 1: Check resource requirements

Before installing anything, confirm your machine meets the minimums. Running an undersized node is the fastest way to end up stuck on block 1 with peers constantly dropping.

The requirements below come directly from the official Midnight documentation:

Requirement Preview/Preprod Testnet Cardano Mainnet
OS Ubuntu 24.04 LTS (64-bit) Ubuntu 24.04 LTS (64-bit)
Memory 16 GB or more 32 GB or more
CPU cores 4 or more 4 or more
Disk storage 40 GB NVMe SSD (minimum) 320 GB NVMe SSD
Disk IOPS 30,000 IOPS or better 60,000 IOPS or better
Network Stable 100 Mbps, port 30333 open Stable 100 Mbps, port 30333 open

A note on GLIBC. The node binary requires GLIBC 2.39 or greater. Ubuntu 24.04 ships with it. Ubuntu 22.04 does not. Check yours:

ldd --version
Enter fullscreen mode Exit fullscreen mode

Avoid HDDs and cheap burstable cloud volumes. Low-IOPS storage causes sync stalls, peer disconnections, and PostgreSQL failures under write load. If your node keeps falling behind the chain tip, check disk I/O before anything else:

iostat -x 1 5
Enter fullscreen mode Exit fullscreen mode

Look at the %util column for your storage device. Sustained values above 80% mean your disk is the bottleneck. Migrate to a faster SSD before continuing.

Step 2: Create the midnight user

You can optionally create a non-privileged user to run all node services. Not running blockchain software as root is a security best practice.

sudo adduser midnight
sudo usermod -aG sudo midnight
su - midnight
whoami  # should return: midnight
Enter fullscreen mode Exit fullscreen mode

All remaining steps in this tutorial assume you're operating as the midnight user. Paths like /home/midnight throughout this guide refer to this user's home directory.

Step 3: Set up Cardano node with Mithril

Midnight operates as a Cardano partner chain. Your Midnight node requires a persistent connection to a Cardano-db-sync instance to track relevant scripts on the Cardano blockchain. This is not optional. Your node will stall near block 1 without it.

Syncing Cardano from genesis takes days. This guide uses Mithril to download a cryptographically verified snapshot of the Cardano blockchain, reducing that to roughly 20 minutes.

Install Mithril

Create a working directory and install the Mithril client:

mkdir -p $HOME/tmp/mithril && cd $HOME/tmp/mithril

curl --proto '=https' --tlsv1.2 -sSf \
  https://raw.githubusercontent.com/input-output-hk/mithril/refs/heads/main/mithril-install.sh \
  | sh -s -- -c mithril-client -d unstable -p $(pwd)
Enter fullscreen mode Exit fullscreen mode

Configure Mithril for your network

Set the environment variables for Preview Testnet. For Preprod or Mainnet, use the corresponding values from the Mithril network configurations.

export CARDANO_NETWORK=preview
export AGGREGATOR_ENDPOINT=https://aggregator.pre-release-preview.api.mithril.network/aggregator
export GENESIS_VERIFICATION_KEY=$(wget -q -O - \
  https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/pre-release-preview/genesis.vkey)
export ANCILLARY_VERIFICATION_KEY=$(wget -q -O - \
  https://raw.githubusercontent.com/input-output-hk/mithril/main/mithril-infra/configuration/pre-release-preview/ancillary.vkey)
export SNAPSHOT_DIGEST=latest
Enter fullscreen mode Exit fullscreen mode

Download and verify the Cardano snapshot 

./mithril-client cardano-db snapshot list
./mithril-client cardano-db snapshot show $SNAPSHOT_DIGEST
./mithril-client cardano-db download --include-ancillary $SNAPSHOT_DIGEST
Enter fullscreen mode Exit fullscreen mode

The Mithril client downloads the snapshot into a db/ directory in the current working directory ($HOME/tmp/mithril/db). You'll move this into place in the next step.

Install Cardano node

Always check the official Cardano node release page and the Midnight compatibility matrix for the current required version. At the time of writing, the required version is 11.0.1.

VERSION="11.0.1"
ARCH="linux-amd64"
URL="https://github.com/IntersectMBO/cardano-node/releases/download/${VERSION}/cardano-node-${VERSION}-${ARCH}.tar.gz"

mkdir -p ~/.local/bin ~/.local/share

# Download binaries and configuration files
curl -L "$URL" | tar -xz -C ~/.local/bin --strip-components=2 ./bin
curl -L "$URL" | tar -xz -C ~/.local/share --strip-components=1 ./share

chmod +x ~/.local/bin/cardano-*
Enter fullscreen mode Exit fullscreen mode

Reload your shell and verify the binary:

source ~/.bashrc
cardano-node --version
# cardano-node 11.0.1 - linux-x86_64 - ghc-9.6
Enter fullscreen mode Exit fullscreen mode

You should see the ~/.local/share directory now contains network config folders: mainnet, preprod, and preview.

Inject the Mithril snapshot and run Cardano node

Create the Cardano data directory and move the snapshot in:

mkdir -p ~/cardano-data
mv ~/tmp/mithril/db/ ~/cardano-data/
ls ~/cardano-data/db  # verify the database exists
Enter fullscreen mode Exit fullscreen mode

Start the Cardano node interactively to confirm it works before creating the systemd service:

cardano-node run \
  --topology ~/.local/share/preview/topology.json \
  --database-path ~/cardano-data/db \
  --socket-path ~/cardano-data/db/node.socket \
  --host-addr 0.0.0.0 \
  --port 3001 \
  --config ~/.local/share/preview/config.json
Enter fullscreen mode Exit fullscreen mode

Once you see the node logging block events, press Ctrl+C and set it up as a systemd service. Create the service file:

sudo vim /etc/systemd/system/cardano-node.service
Enter fullscreen mode Exit fullscreen mode

Paste the following (for Preview; adjust the config and topology paths for Preprod or Mainnet):

[Unit]
Description=Cardano Relay Node
Wants=network-online.target
After=network-online.target

[Service]
User=midnight
Type=simple
WorkingDirectory=/home/midnight/cardano-data
ExecStart=/home/midnight/.local/bin/cardano-node run \
    --topology /home/midnight/.local/share/preview/topology.json \
    --database-path /home/midnight/cardano-data/db \
    --socket-path /home/midnight/cardano-data/db/node.socket \
    --host-addr 0.0.0.0 \
    --port 3001 \
    --config /home/midnight/.local/share/preview/config.json
KillSignal=SIGINT
Restart=always
RestartSec=5
LimitNOFILE=32768

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable cardano-node
sudo systemctl start cardano-node
sudo systemctl status cardano-node
Enter fullscreen mode Exit fullscreen mode

Follow the live logs:

journalctl -fu cardano-node
Enter fullscreen mode Exit fullscreen mode

Check sync progress using cardano-cli. The testnet magic number is 2 for Preview:

export CARDANO_NODE_SOCKET_PATH="$HOME/cardano-data/db/node.socket"
cardano-cli query tip --testnet-magic 2
Enter fullscreen mode Exit fullscreen mode

Example output:

{
    "block": 1253000,
    "epoch": 212,
    "era": "Conway",
    "syncProgress": "99.80"
}
Enter fullscreen mode Exit fullscreen mode

Wait for syncProgress to reach 100 before continuing.

Step 4: Set up PostgreSQL and Cardano-db-sync

Install PostgreSQL 17 

sudo apt install curl ca-certificates -y
sudo install -d /usr/share/postgresql-common/pgdg
sudo curl -s -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc \
  --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
sudo sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] \
  https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" \
  > /etc/apt/sources.list.d/pgdg.list'
sudo apt update && sudo apt -y install postgresql-17 postgresql-server-dev-17
Enter fullscreen mode Exit fullscreen mode

Configure roles and the database

Start the PostgreSQL shell as the admin user:

sudo -i -u postgres psql
Enter fullscreen mode Exit fullscreen mode

Create the midnight database user, grant it privileges, and create the cexplorer database:

CREATE USER midnight WITH PASSWORD 'your_actual_password';
ALTER ROLE midnight WITH SUPERUSER CREATEDB;
CREATE DATABASE cexplorer;
\q
Enter fullscreen mode Exit fullscreen mode

Replace your_actual_password with a strong, unique password only you know.

Configure passwordless authentication

Cardano-db-sync uses a .pgpass file for authentication:

export POSTGRES_PASSWORD='your_actual_password'
export PGPASSFILE="${HOME}/.pgpass"

echo "/var/run/postgresql:5432:cexplorer:midnight:$POSTGRES_PASSWORD" > "$PGPASSFILE"
chmod 0600 "$PGPASSFILE"
Enter fullscreen mode Exit fullscreen mode

Test the connection — it should succeed without a password prompt:

psql -h /var/run/postgresql -U midnight -d cexplorer -c "SELECT current_user; SELECT now();"
Enter fullscreen mode Exit fullscreen mode

PostgreSQL performance tuning

For Mainnet (and good practice on testnets with fast hardware), tune your PostgreSQL configuration. Edit /etc/postgresql/17/main/postgresql.conf and update or uncomment these lines:

shared_buffers = 16GB
maintenance_work_mem = 4GB
max_parallel_maintenance_workers = 4
effective_cache_size = 48GB
Enter fullscreen mode Exit fullscreen mode

Restart PostgreSQL after any config change:

sudo systemctl restart postgresql
Enter fullscreen mode Exit fullscreen mode

Install Cardano-db-sync

The required version is 13.7.1.0. Always verify the current required version against the compatibility matrix before downloading.

cd ~/tmp
curl -L -O https://github.com/IntersectMBO/cardano-db-sync/releases/download/13.7.1.0/cardano-db-sync-13.7.1.0-linux.tar.gz
tar -xzf cardano-db-sync-13.7.1.0-linux.tar.gz

mkdir -p ~/.local/bin
cp bin/* ~/.local/bin/
chmod +x ~/.local/bin/cardano-db-sync*

# Verify
cardano-db-sync --version
Enter fullscreen mode Exit fullscreen mode

Move the schema and download the db-sync config:

mkdir -p ~/cardano-data/
sudo mv ~/tmp/schema ~/cardano-data/

cd ~/cardano-data
curl -O https://book.world.dev.cardano.org/environments/preview/db-sync-config.json

# Update the config to point at the Cardano node's config file
sed -i 's|"NodeConfigFile": "config.json"|"NodeConfigFile": "/home/midnight/.local/share/preview/config.json"|' \
  ~/cardano-data/db-sync-config.json
Enter fullscreen mode Exit fullscreen mode

Start Cardano-db-sync

Test it interactively first:

export PGPASSFILE="${HOME}/.pgpass"

cardano-db-sync \
    --config /home/midnight/cardano-data/db-sync-config.json \
    --socket-path /home/midnight/cardano-data/db/node.socket \
    --schema-dir /home/midnight/cardano-data/schema \
    --state-dir /home/midnight/cardano-data/db-sync-state
Enter fullscreen mode Exit fullscreen mode

It may take 5–20 minutes to initialise. This is normal. Once you see it syncing, press Ctrl+C and create the systemd service:

sudo vim /etc/systemd/system/cardano-db-sync.service
Enter fullscreen mode Exit fullscreen mode 
[Unit]
Description=Cardano DB Sync
After=cardano-node.service
Requires=cardano-node.service

[Service]
User=midnight
Type=simple
Environment="PGPASSFILE=/home/midnight/.pgpass"
WorkingDirectory=/home/midnight/cardano-data
ExecStart=/home/midnight/.local/bin/cardano-db-sync \
    --config /home/midnight/cardano-data/db-sync-config.json \
    --socket-path /home/midnight/cardano-data/db/node.socket \
    --schema-dir /home/midnight/cardano-data/schema \
    --state-dir /home/midnight/cardano-data/db-sync-state
KillSignal=SIGINT
Restart=always
RestartSec=10
LimitNOFILE=32768

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable cardano-db-sync
sudo systemctl start cardano-db-sync
sudo systemctl status cardano-db-sync
Enter fullscreen mode Exit fullscreen mode

Monitor Cardano-db-sync sync progress

Follow the logs:

journalctl -u cardano-db-sync -f
Enter fullscreen mode Exit fullscreen mode

Query sync percentage from PostgreSQL:

psql -d cexplorer -c "
SELECT
    100 * (EXTRACT(epoch FROM (MAX(time) AT TIME ZONE 'UTC')) - EXTRACT(epoch FROM (MIN(time) AT TIME ZONE 'UTC')))
    / (EXTRACT(epoch FROM (NOW() AT TIME ZONE 'UTC')) - EXTRACT(epoch FROM (MIN(time) AT TIME ZONE 'UTC')))
AS sync_percent
FROM block;"
Enter fullscreen mode Exit fullscreen mode

You'll see a number between 0 and 100. Cardano-db-sync must reach 100 before your Midnight node can operate correctly. On Preview Testnet this typically takes several hours depending on your hardware and disk IOPS.

If you start the Midnight node before db-sync finishes, you'll see this in its logs:

Unable to author block in slot. Failure creating inherent data provider:
'No latest block on chain.' not found.
Possible causes: main chain follower configuration error, db-sync not synced fully,
or data not set on the main chain.
Enter fullscreen mode Exit fullscreen mode

This is the Cardano dependency telling you it's not ready. Wait for sync_percent to reach 100 before proceeding.

Step 5: Install and configure the Midnight node

Create directories and download the binary

Always check the compatibility matrix for the current node version before downloading. At the time of writing, the Preview network uses 1.0.0 (binary release node-0.22.5).

mkdir -p ~/data ~/res ~/.local/bin
mkdir -p ~/tmp && cd ~/tmp

curl -L -O https://github.com/midnightntwrk/midnight-node/releases/download/node-0.22.5/midnight-node-0.22.5-linux-amd64.tar.gz
tar -xvzf midnight-node-0.22.5-linux-amd64.tar.gz

mv ~/tmp/midnight-node ~/.local/bin/
mv ~/tmp/res ~/res

source ~/.bashrc
midnight-node --version
Enter fullscreen mode Exit fullscreen mode

Confirm the chain spec file for your network is present:

ls ~/res/preview/chain-spec-raw.json
Enter fullscreen mode Exit fullscreen mode

Set environment variables

Create an .env file for the PostgreSQL credentials:

cat > ~/.env << 'EOF'
export POSTGRES_HOST="localhost"
export POSTGRES_DB="cexplorer"
export POSTGRES_PORT="5432"
export POSTGRES_USER="midnight"
export POSTGRES_PASSWORD="your_actual_password"
export DB_SYNC_POSTGRES_CONNECTION_STRING="postgresql://midnight:your_actual_password@localhost:5432/cexplorer"
export NODE_NAME="your_node_name"
EOF

source ~/.env
echo $DB_SYNC_POSTGRES_CONNECTION_STRING
Enter fullscreen mode Exit fullscreen mode

Replace your_actual_password and your_node_name with your actual values.

Run the Midnight node

You can start the node interactively first to verify the configuration:

midnight-node \
  --chain /home/midnight/res/preview/chain-spec-raw.json \
  --base-path /home/midnight/data \
  --pool-limit 35 \
  --name $NODE_NAME \
  --no-private-ip
Enter fullscreen mode Exit fullscreen mode

Watch for three things in the output before proceeding:

  • Postgres connection established — confirms your DB_SYNC_POSTGRES_CONNECTION_STRING is correct.
  • Best: #0 ... beginning to increment — the node is pulling blocks.
  • Peer count appearing in the log output — if it stays at 0 peers for more than a few minutes, check your firewall on port 30333.

Once satisfied, press Ctrl+C and set it up as a systemd service.

Create the systemd service 

sudo vim /etc/systemd/system/midnight-node.service
Enter fullscreen mode Exit fullscreen mode 
[Unit]
Description=Midnight Full Node
Wants=network-online.target
After=network-online.target cardano-db-sync.service
Requires=cardano-db-sync.service

[Service]
User=midnight
Type=simple
EnvironmentFile=/home/midnight/.env
ExecStart=/home/midnight/.local/bin/midnight-node \
    --chain /home/midnight/res/preview/chain-spec-raw.json \
    --base-path /home/midnight/data \
    --pool-limit 35 \
    --name ${NODE_NAME} \
    --no-private-ip
KillSignal=SIGINT
Restart=always
RestartSec=10
LimitNOFILE=32768

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable midnight-node
sudo systemctl start midnight-node
sudo systemctl status midnight-node
Enter fullscreen mode Exit fullscreen mode

Follow the logs:

journalctl -fu midnight-node
Enter fullscreen mode Exit fullscreen mode

Step 6: Monitor the initial sync

Reading the sync log output

During a healthy sync, you'll see lines like this:

2026-06-16 12:00:01 ⚙️  Syncing 45.2 bps, target=#187432 (12 peers), best: #8492 (0x1a2b…c3d4), finalized #8448 (0x5e6f…7a8b)
Enter fullscreen mode Exit fullscreen mode

Here's what each field means:

  • bps: blocks per second. Several dozen during initial sync is healthy. Single-digit bps suggests your disk or CPU is the bottleneck.
  • target=#XXXXXX: the current chain tip your peers are reporting.
  • best: #XXXXXX: your local best block. Should climb steadily.
  • finalized #XXXXXX: the last finalised block, trailing your best by a small number.
  • (N peers): connected peers. You want 5 or more.

When your node finishes syncing, the log line changes from Syncing to Idle:

2026-06-16 14:22:01 💤 Idle (14 peers), best: #187432 (0x1a2b…c3d4), finalized #187428 (0x5e6f…7a8b)
Enter fullscreen mode Exit fullscreen mode

Idle means you're at the tip and waiting for new blocks.

Check sync state via RPC

The Midnight node exposes a JSON-RPC interface on port 9944. Query sync state directly:

curl -s -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"system_syncState","params":[],"id":1}' \
  http://localhost:9944 | jq '.result'
Enter fullscreen mode Exit fullscreen mode

Example output:

{
  "startingBlock": 0,
  "currentBlock": 142560,
  "highestBlock": 187432
}
Enter fullscreen mode Exit fullscreen mode

When currentBlock equals highestBlock, you're synced. Check peer connectivity at the same time:

curl -s -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"system_health","params":[],"id":1}' \
  http://localhost:9944 | jq '.result'
Enter fullscreen mode Exit fullscreen mode

A healthy, synced node returns:

{
  "peers": 14,
  "isSyncing": false,
  "shouldHavePeers": true
}
Enter fullscreen mode Exit fullscreen mode

isSyncing: false with peers greater than zero means you're at the tip and connected. If peers is persistently 0 or 1, see the troubleshooting section.

Monitor block height on a loop

Save this as monitor-height.sh for reuse throughout operations:

cat > ~/monitor-height.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail

RPC="${RPC:-http://localhost:9944}"
INTERVAL="${INTERVAL:-30}"

while true; do
  result=$(curl -fsS \
    -H "Content-Type: application/json" \
    -d '{"jsonrpc":"2.0","method":"system_syncState","params":[],"id":1}' \
    "$RPC" 2>/dev/null | jq -r '
      .result |
      "current=\(.currentBlock)  target=\(.highestBlock)  lag=\(.highestBlock - .currentBlock)"
    ') || result="RPC unreachable"
  printf '%s  %s\n' "$(date -u +"%H:%M:%SZ")" "$result"
  sleep "$INTERVAL"
done
EOF
chmod +x ~/monitor-height.sh
Enter fullscreen mode Exit fullscreen mode

Run it:

~/monitor-height.sh
Enter fullscreen mode Exit fullscreen mode

A healthy syncing node shows a rising current and a shrinking lag. A fully synced node shows lag=0 and holds there.

Step 7: Verify your node is synced and healthy

Once the node reports Idle or lag=0, run through this verification checklist.

All services running:

sudo systemctl status cardano-node cardano-db-sync midnight-node
Enter fullscreen mode Exit fullscreen mode

Node has peers and is not syncing:

curl -s -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"system_health","params":[],"id":1}' \
  http://localhost:9944 | jq '.result'
Enter fullscreen mode Exit fullscreen mode

P2P port is listening:

ss -ltnp | grep ':30333'
Enter fullscreen mode Exit fullscreen mode

No repeated errors in the last 30 minutes:

journalctl -u midnight-node --since "30 minutes ago" \
  | grep -Ei 'error|panic|disconnect' \
  || echo "No errors found."
Enter fullscreen mode Exit fullscreen mode

Confirm binary version:

midnight-node --version
Enter fullscreen mode Exit fullscreen mode

A clean bill of health is: all three services active, isSyncing: false, peer count above zero, and no repeated panics or database errors.

Step 8: Troubleshooting common issues

Node is stuck on block 1 (or a very low block number)

This is the most commonly reported issue. There are four causes, in order of likelihood.

1. Cardano-db-sync hasn't finished syncing

Your Midnight node requires a fully synced Cardano-db-sync instance. Run the sync percentage query from Step 4. If it's below 100, wait. The "Unable to author block" log message in your Midnight node's journal is the confirmation of this state.

2. PostgreSQL connection string is wrong

A common mistake is using localhost in the connection string when PostgreSQL is configured to use a Unix socket. The db-sync setup in this guide uses the socket path /var/run/postgresql. Confirm your connection string is correct:

echo $DB_SYNC_POSTGRES_CONNECTION_STRING
Enter fullscreen mode Exit fullscreen mode

Test the connection directly:

psql -h /var/run/postgresql -U midnight -d cexplorer -c 'SELECT COUNT(*) FROM block;'
Enter fullscreen mode Exit fullscreen mode

If this command fails, fix your PostgreSQL configuration before restarting the Midnight node. Also make sure the .env file is loading correctly:

source ~/.env
echo $DB_SYNC_POSTGRES_CONNECTION_STRING
Enter fullscreen mode Exit fullscreen mode

3. Chain spec file is missing or wrong

The --chain flag must point at the correct chain spec for your network. Confirm the file exists:

ls -lh ~/res/preview/chain-spec-raw.json
Enter fullscreen mode Exit fullscreen mode

If the file is missing, the binary extraction in Step 5 may have failed. Re-run the extraction and verify ~/res contains the expected directories.

4. Node binary is outdated

Node versions update frequently, and an outdated binary gets rejected by current network peers. Always cross-reference the version you're running against the compatibility matrix. If a newer version is listed, follow the upgrade procedure at the end of this tutorial.

Peers keep disconnecting

If peer count climbs then immediately drops back to 0, work through these checks in order.

1. Port 30333 is not publicly reachable

This is the single most common cause. Even if the port is open locally, your cloud security group may be blocking inbound connections from external peers.

# Check UFW
sudo ufw status

# Open if missing
sudo ufw allow 30333/tcp
sudo ufw reload
Enter fullscreen mode Exit fullscreen mode

If you're on a cloud VM (AWS, GCP, DigitalOcean), also check your provider's inbound security group rules in the web console. The operating-system firewall and the cloud-level firewall are independent.

2. Clock drift

Substrate-based nodes reject connections from peers whose system clocks differ significantly. Check that NTP is active:

timedatectl status
Enter fullscreen mode Exit fullscreen mode

Look for NTP service: active and System clock synchronized: yes. If NTP is inactive:

sudo timedatectl set-ntp true
Enter fullscreen mode Exit fullscreen mode

3. Disk I/O bottleneck

High disk utilisation causes the node to fall behind processing blocks. Peers consider it unresponsive and drop the connection. Run iostat -x 1 10 and check %util. If it's consistently above 80%, the disk is your problem.

4. NAT or double-NAT

If your server is behind NAT, peers can't initiate inbound connections to you. Either configure port forwarding at the router level or use a VPS with a dedicated public IP.

Node crashes or restarts frequently

Check the last error before the most recent restart:

journalctl -u midnight-node -n 50 | grep -E "ERROR|WARN|panic"
Enter fullscreen mode Exit fullscreen mode

Check whether the system OOM-killed the process:

journalctl -k | grep -i "oom\|killed process"
Enter fullscreen mode Exit fullscreen mode

If OOM kills appear, the node is running out of memory. Add more RAM, or as a temporary measure on an undersized machine:

sudo fallocate -l 8G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
Enter fullscreen mode Exit fullscreen mode

For suspected chain data corruption after a hard shutdown, clear the Midnight node's base path while leaving your Cardano-db-sync database intact — this avoids re-running the hours-long db-sync:

sudo systemctl stop midnight-node
rm -rf ~/data
sudo systemctl start midnight-node
Enter fullscreen mode Exit fullscreen mode

If you also need to reset Cardano-db-sync (only do this if you're certain the issue is there; it adds hours of re-sync time):

sudo systemctl stop cardano-db-sync midnight-node
# Drop and recreate the cexplorer database in psql, then restart
sudo systemctl start cardano-db-sync midnight-node
Enter fullscreen mode Exit fullscreen mode

IPC socket errors

After an unclean shutdown you may see Cardano-db-sync logs complaining about a missing or stale socket:

No such file or directory: .../db/node.socket
Enter fullscreen mode Exit fullscreen mode

The socket file is created by Cardano-node on startup. If Cardano-node hasn't fully initialised yet, or if it crashed and left a stale socket, db-sync can't connect.

# Check whether the socket exists
ls -la ~/cardano-data/db/node.socket

# If the Cardano node is running, wait a moment and check again
sudo systemctl status cardano-node
Enter fullscreen mode Exit fullscreen mode

If Cardano-node is running but the socket is missing, restart the service and watch the logs for its creation:

sudo systemctl restart cardano-node
watch -n 2 "ls -la ~/cardano-data/db/node.socket"
Enter fullscreen mode Exit fullscreen mode

Once the socket appears, db-sync will reconnect automatically.

Port conflicts

If a service fails to start with address already in use, something else on your host is already bound to that port. The full port map:

Port Service
9944 Midnight node RPC
30333 Midnight node P2P
9615 Midnight node Prometheus metrics
3001 Cardano node P2P
5432 PostgreSQL

Identify the conflicting process:

sudo ss -tlnp | grep -E '9944|30333|9615|3001|5432'
Enter fullscreen mode Exit fullscreen mode

Stop the conflicting process or remap the port in the affected service's flags.

Keeping your node up to date

Midnight is in active development. Node versions update frequently, and an outdated binary will stop syncing or get rejected by peers. Watch the Midnight Discord and the release notes for new versions.

When a new version is released:

  1. Check the compatibility matrix to confirm the new binary version and any updated component versions.
  2. Download the new binary: 
cd ~/tmp
curl -L -O https://github.com/midnightntwrk/midnight-node/releases/download/<NEW_VERSION>/midnight-node-<NEW_VERSION>-linux-amd64.tar.gz
tar -xvzf midnight-node-<NEW_VERSION>-linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode
  1. Stop the node, replace the binary, and restart: 
sudo systemctl stop midnight-node
mv ~/tmp/midnight-node ~/.local/bin/midnight-node
sudo systemctl start midnight-node
Enter fullscreen mode Exit fullscreen mode
  1. Confirm the new version is running: 
midnight-node --version
Enter fullscreen mode Exit fullscreen mode
  1. Watch the logs for a clean startup and verify peers reconnect: 
journalctl -fu midnight-node
Enter fullscreen mode Exit fullscreen mode

Before every upgrade, back up your configuration. Chain data can be re-synced from genesis, but your env file and service configuration cannot be recovered if lost:

tar -czf "midnight-config-backup-$(date -u +%Y%m%d).tar.gz" \
  ~/.env \
  /etc/systemd/system/midnight-node.service \
  /etc/systemd/system/cardano-node.service \
  /etc/systemd/system/cardano-db-sync.service
Enter fullscreen mode Exit fullscreen mode

What to do next

You now have a full node running on the official stack, synced to the Midnight network, with monitoring commands and a tested troubleshooting playbook.

  • Set up an RPC node: if you want your DApp to talk directly to your own infrastructure instead of a public endpoint, follow the RPC node setup guide.
  • Explore the DApp quickstart: start building a privacy-preserving application on top of the network you're now running: Getting started.
  • Add persistent monitoring: schedule ~/monitor-height.sh via a systemd timer or cron, and set up Prometheus alerting when peer count drops to zero.
  • Join the community: the Midnight developer forum and Discord are the best places to ask questions, report issues, and follow network updates.
  • Share your work: tag @midnightntwrk on X and use #MidnightforDevs.

References

This tutorial targets Midnight Preview using the current official binary installation method. Always check the compatibility matrix before starting a node or upgrading an existing one.

Top comments (0)