> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need. # Worker A worker is the service that does Amp's work: it runs the [jobs](/managing-amp/jobs) the [controller](/running-amp/controller) assigns it. Most jobs build a dataset's tables, which Amp calls materializing them: * **Raw datasets**: the worker extracts blocks from a chain through its [provider](/providers), and writes them as tables. See [Raw Datasets](/datasets/raw-datasets). * **Derived datasets**: the worker runs the dataset's SQL over other datasets. See [Derived Datasets](/datasets/derived-datasets). * **Exports**: the worker exports tables to another database. See [Exports](/managing-amp/pipelines). * **Garbage collection**: the worker deletes the files that tables no longer use. See [Garbage Collection](/running-amp/storage/garbage-collection). It writes the tables to [storage](/running-amp/storage) as Parquet files, records them in the [metadata database](/running-amp/metadata-database), and [compacts](/running-amp/storage/compaction) them as it goes. Solo mode runs one worker in its process, with the ID `worker`. In [distributed mode](/running-amp/distributed-mode), you run as many as you need, with `ampd worker`. ## Prerequisites * A [configuration file](/running-amp/configuration) with the metadata database's URL and your storage locations. * A migrated metadata database, unless the configuration lets the worker migrate it ([`auto_migrate`](/running-amp/metadata-database#auto_migrate)). * Network access to the providers of the datasets it extracts. ## Start a worker ```bash ampd --config config.toml worker --node-id worker-01 ``` The worker logs its jobs as it runs them. Check that the controller sees it with `ampctl worker list`, or its [health check](/running-amp/worker#health-check). ## Node IDs Every worker has an ID, which you give it with `--node-id` or the `AMP_NODE_ID` environment variable. The controller assigns jobs to IDs, so: * **Make it unique** among the workers that run at the same time. * **Keep it across restarts.** A worker that starts again with the same ID resumes the jobs assigned to it. * **Make it descriptive**, like `eu-west-1a-worker`. You can [send a job](/managing-amp/workers) to a group of workers with a pattern, like `eu-west-*`. An ID starts with a letter, and contains only letters, digits, `_`, `-`, and `.`. `controller` is reserved. Otherwise, the worker doesn't start: ``` Exiting with error: Invalid worker node ID | Caused by: Invalid worker ID '1worker': must start with a letter, got '1' ``` ## How a worker gets its jobs 1. When it starts, the worker registers its ID in the metadata database, and resumes the jobs assigned to it. 2. Every second, it writes a heartbeat there. The controller considers a worker active while its heartbeat is less than 5 seconds old, and only assigns new jobs to active workers. 3. The metadata database notifies it when the controller assigns it a job or stops one. Every 60 seconds, it also checks for changes it may have missed. A job stays on the worker it was first assigned to, through its retries and periodic runs. Starting more workers doesn't move existing jobs: only new ones go to them. See [Job assignment](/running-amp/controller#job-assignment). ## Settings The worker reads the [shared settings](/running-amp/configuration#per-service-settings) and the `[worker]` section. The settings marked with an asterisk can also be set once at the top level, for workers and query servers both: see [Global defaults](/running-amp/configuration#global-defaults). | Setting | Default | Description | | ------------------------------------------------------------------------------------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------- | | [`health_addr`](/running-amp/configuration-reference#workerhealth_addr) | `0.0.0.0:1620` | The address the [health check](/running-amp/worker#health-check) listens on. | | [`poll_interval_secs`](/running-amp/configuration-reference#workerpoll_interval_secs) | `1.0` | How often a raw dataset job asks its provider for new blocks once it reaches the chain's head, in seconds. | | [`microbatch_max_interval`](/running-amp/configuration-reference#workermicrobatch_max_interval)\* | `100000` | The most blocks a derived dataset job processes at once. | | [`max_mem_mb`](/running-amp/configuration-reference#workermax_mem_mb)\* | `0` | The memory the SQL of all derived dataset and export jobs uses together, in MiB. `0` is unlimited. | | [`query_max_mem_mb`](/running-amp/configuration-reference#workerquery_max_mem_mb)\* | `0` | The memory each step of that SQL uses, in MiB. `0` is unlimited. | | [`spill_location`](/running-amp/configuration-reference#workerspill_location)\* | `[]` | Directories where that SQL writes temporary files when it runs out of memory. | | [`metadata_fetch_concurrency`](/running-amp/configuration-reference#workermetadata_fetch_concurrency)\* | `32` | How many requests for Parquet metadata that SQL sends the metadata database at once, per table. | | [`[worker.events]`](/running-amp/worker-events) | | Sends job progress to Kafka. (Experimental) | | [`[worker.metadb]`](/running-amp/metadata-database#per-service-pools) | | The worker's own connection pool settings. Workers write the most, so they often need more connections. | How workers write and compact files is set in the shared [`[writer]`](/running-amp/storage#parquet-files) and [`[writer.compactor]`](/running-amp/storage/compaction#settings) sections, which apply to every job on the worker, including how long a raw dataset job writes to the same file before closing it: [`writer.segment_flush_interval_secs`](/running-amp/configuration-reference#writersegment_flush_interval_secs). ## Health check `GET /healthz` on port `1620` returns: * `200 OK` once the worker has started its jobs, and while it receives notifications from the metadata database. * `503 Service Unavailable` while it starts, and while it has lost its connection for notifications. Use it as a liveness and readiness check in your orchestrator, like a Kubernetes probe. When the health check itself stops, the worker exits, so the orchestrator restarts it. ## Stopping On `SIGINT` or `SIGTERM`, the worker stops its jobs and exits: ``` 2026-10-30T17:07:39.818817Z INFO amp_worker_service::service: Shutdown signal received, stopping worker node_id=worker-01 ``` Its jobs stay assigned to it, and resume when a worker with the same ID starts again. ## Troubleshooting To find out why a worker isn't running its jobs, see [Troubleshooting Workers](/managing-amp/workers/troubleshooting). ### `the following required arguments were not provided` ``` error: the following required arguments were not provided: --node-id ``` Give the worker an ID, with `--node-id` or `AMP_NODE_ID`. ### `failed to bind health server` ``` Exiting with error: Worker command failed | Caused by: Worker initialization failed -> failed to bind health server to 0.0.0.0:1620 -> Address already in use (os error 48) ``` Another program, often another worker on the same machine, uses the port. Give each worker its own [`health_addr`](/running-amp/configuration-reference#workerhealth_addr), like `AMP_CONFIG_WORKER__HEALTH_ADDR=0.0.0.0:1621`.