> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need. # Architecture Amp is three services, two stores, and the clients that talk to them. This page explains what each one does, and how data moves from a chain to your queries. ```text Providers ──▶ Workers ──▶ Parquet files in storage ──▶ Query servers ──▶ Clients │ ▲ │ └──────── PostgreSQL metadata database ─────┘ ▲ │ Controller (Admin API) ``` ## How data flows 1. **You define the data.** With `ampctl` or an SDK, you send the controller a [provider](/get-started/concepts/providers), which says where to get a chain's data, and a [dataset](/get-started/concepts/datasets), which says what to extract or compute. The controller records them. 2. **You deploy the dataset.** The controller schedules a [job](/get-started/concepts/jobs-and-workers), and assigns it to a worker. 3. **A worker extracts the data.** It fetches blocks through the provider, or runs a derived dataset's SQL over other datasets. It writes the results to storage as Parquet files, and records each file in the metadata database. 4. **A query server answers queries.** It finds the files a query needs in the metadata database, reads them from storage, and sends the results back over Arrow Flight. For a streaming query, it keeps sending new results as workers write new files. ## Services All three services are the same program, `ampd`. In [solo mode](/running-amp/solo-mode), one `ampd solo` process runs all of them, with one worker. In [distributed mode](/running-amp/distributed-mode), each runs as its own process: | Command | Runs | | ----------------- | -------------------------------------------------------------------------------- | | `ampd solo` | The query server, the controller, and one worker, in one process | | `ampd server` | A query server | | `ampd controller` | The controller | | `ampd worker` | A worker | | `ampd migrate` | Nothing: it updates the metadata database's tables for a new release, then exits | ### Query server The query server answers SQL queries over [Arrow Flight](/querying/arrow-flight), on port 1602 by default. It runs [batch queries](/querying/batch-queries), which return a complete result once, and [streaming queries](/querying/streaming-queries), which keep sending new rows as the data grows. Query servers don't write data: the metadata database tells them which files hold a table's data, and they read those files from storage. So you can run several, behind a load balancer you provide. ### Controller The controller serves the [Admin API](/managing-amp/admin-api), an HTTP API on port 1610 by default, which `ampctl` and the SDKs' admin clients use to manage Amp: register providers and datasets, deploy datasets, and watch jobs and workers. It also schedules jobs: it records each one in the metadata database, and assigns it to a worker. ### Workers Workers do the work: they run the jobs the controller assigns them, like extracting a raw dataset from its provider, or running a derived dataset's SQL. They write Parquet files to storage, and record them in the metadata database. Each worker has a node ID, a name unique among running workers. Workers report that they're alive to the metadata database every second, and serve a health check at `/healthz`, on port 1620 by default. Add workers to extract more data at once. [Jobs and Workers](/get-started/concepts/jobs-and-workers) explains how they share work. ## Stores The services don't talk to each other directly: they share two stores. ### Metadata database A PostgreSQL database, version 16 or newer, holds what Amp knows about its data: datasets and their versions, jobs and their progress, workers, and which files make up each table. The controller writes jobs to it, workers pick them up from it, and query servers look up files in it. In solo mode, Amp can run this database for you. ### Storage Amp stores three things, each in a location you configure: * **Data**: each table's [Parquet](/get-started/glossary#parquet) files, organized by dataset, table, and [table revision](/get-started/concepts/tables#table-revisions). * **Manifests**: the JSON definitions of datasets. * **Providers**: the [providers](/get-started/concepts/providers) you register, as TOML files. See [Provider files](/get-started/concepts/providers#provider-files). Each location is a directory or an object store: Amazon S3 (or a store compatible with it, with an `s3://` URL), Google Cloud Storage (`gs://`), or Azure Blob Storage (`az://`). Data files are written once and never changed, so any number of query servers can read them while workers write new ones. In distributed mode, every service must reach the same locations. [Storage](/running-amp/storage) explains how to configure them. ## Interfaces | Interface | Default address | Protocol | Used by | | -------------------------------------- | --------------- | ----------- | --------------------------------------------------------------------- | | [Arrow Flight](/querying/arrow-flight) | `0.0.0.0:1602` | gRPC | `ampsql`, `ampsync`, the SDKs, and any Arrow Flight SQL client | | [Admin API](/managing-amp/admin-api) | `0.0.0.0:1610` | HTTP (REST) | `ampctl`, and the SDKs' admin clients | | Worker health check (`/healthz`) | `0.0.0.0:1620` | HTTP | Your monitoring, like a load balancer's or Kubernetes's health checks | Amp doesn't authenticate requests or encrypt connections (TLS). In production, keep the controller and workers on a private network, and put the query servers behind your own authentication, TLS, and load balancing. [Choose How to Run Amp](/get-started/choose-how-to-run-amp#distributed-mode) has an example.