Skip to main content
Before proceeding, make sure you have compatible hardware.
This guide will walk you through setting up and running an Inference.net node using Docker on Linux.

​
Prerequisites

Before you can run an Inference.net node, you’ll need the following dependencies installed and configured:

​
Required Dependencies

  • Docker Engine - Container runtime for running the Inference node
  • NVIDIA Drivers - GPU drivers compatible with your hardware
  • NVIDIA Container Toolkit - Enables GPU support in Docker containers
Please ensure all dependencies are properly installed before proceeding with node setup.

​
Verify Installation

Before proceeding, verify that all components are properly installed: 
# Check Docker installation
docker --version
docker info

# Verify NVIDIA drivers
nvidia-smi

# Test NVIDIA Docker support
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi
If all commands execute successfully and the last command shows your GPU information, you’re ready to proceed.

​
Setting Up Your Account

  1. Register an account at https://inference.net/register
  2. Contact the Inference.net team to verify your account. You will not be able to start a node until your account is verified.

​
Creating and Running Your First Node

​
Step 1: Create a Worker

  1. Navigate to the Workers tab in your dashboard
  2. Click Create Worker in the top-right corner
  3. Enter a descriptive name for your worker (e.g., “8x-h200-1”)
  4. Ensure Docker is selected as the deployment method
  5. Click Create Worker

​
Step 2: Launch Your Worker

  1. On the Worker Details page, click Launch Worker
  2. You’ll see a Docker command with your unique worker code. It will look like this: 
    docker run \
  --pull=always \
  --restart=always \
  --runtime=nvidia \
  --gpus all \
  -v ~/.inference:/root/.inference \
  inferencecloud/amd64-nvidia-inference-node:latest \
  --code <your-worker-code>
  3. Copy and run this command in your terminal. Make sure to replace <your-worker-code> with the worker code on the launch worker modal.

​
Step 3: Monitor Initialization

Once started, your node will enter the “Initializing” state on the dashboard. This initialization phase:
  • Typically takes 1-2 minutes
  • May take up to 10 minutes depending on your GPU and network speed
  • Downloads necessary model files and prepares the inference environment
You can monitor the progress by checking:
  • The dashboard status
  • Docker logs: docker logs -f $(docker ps -lq)

​
Understanding the Docker Command

Let’s break down what each parameter does:
  • --pull=always: Always pulls the latest image version
  • --restart=always: Automatically restarts the container if it stops or the system reboots
  • --runtime=nvidia: Enables NVIDIA GPU support
  • --gpus all: Grants access to all available GPUs
  • -v ~/.inference:/root/.inference: Persists data between container restarts
  • --code <your-worker-code>: Your unique worker authentication code

​
Managing Your Node

​
Viewing Logs

To monitor your node’s activity: 
# View recent logs
docker logs $(docker ps -lq)

# Follow logs in real-time
docker logs -f $(docker ps -lq)

​
Checking Status

# List running containers
docker ps

# View resource usage
docker stats

​
Stopping Your Node

# Stop the container gracefully
docker stop $(docker ps -lq)

# Or stop by container ID
docker stop <container-id>

​
Troubleshooting

​
Common Issues

Container won’t start:
  • Check Docker daemon: sudo systemctl status docker
  • Restart Docker: sudo systemctl restart docker
  • View Docker logs: sudo journalctl -fu docker
GPU not detected:
  • Verify NVIDIA runtime: docker info | grep nvidia
  • Check GPU availability: nvidia-smi
  • Test GPU access: docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi
Node stuck in “Initializing”:
  • Check container logs for errors: docker logs $(docker ps -lq)
  • Ensure you have sufficient disk space for model downloads
  • Verify your internet connection is stable

​
Next Steps

Hardware Requirements

Check out the list of officially supported hardware