> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need. # ampctl manifest `ampctl manifest` works with [manifests](/datasets/manifests): the JSON documents that define datasets, with their kind, their tables and columns, and, for [derived datasets](/datasets/derived-datasets), the SQL that fills them. It does two jobs: * [`generate`](/clis/ampctl/manifest#ampctl-manifest-generate) writes the manifest of a [raw dataset](/datasets/raw-datasets), a dataset that extracts a chain through a [provider](/providers), so you don't write one by hand. * The other commands manage the manifests Amp stores. Amp keeps each manifest once, under its [hash](/clis/ampctl/manifest#hashes), and datasets point to it. [Manifest Storage](/managing-amp/manifests) explains how. You don't need these commands to publish a dataset: [`ampctl dataset register`](/clis/ampctl/dataset) stores the manifest and registers the dataset in one step. Use them to store a manifest before a dataset uses it, to read the manifest behind a hash, and to delete manifests no dataset uses. `ampctl manifests` is the same command. ## Prerequisites * [`ampctl` installed](/clis/ampctl#install). That's all [`generate`](/clis/ampctl/manifest#ampctl-manifest-generate) needs: it runs without Amp. * For the other commands, a running Amp, like the one the [Quickstart](/get-started/quickstart) starts with [`ampd solo`](/running-amp/solo-mode), and `ampctl` [connected to it](/clis/ampctl#connect-to-amp). The examples' output includes the Quickstart's `acme/eth_mainnet@1.0.0` and [`acme/eth_transfers@1.0.0`](/datasets/derived-datasets#the-erc-20-transfers-dataset), a derived dataset you create on top of it. Every command but `generate` takes `ampctl`'s [global flags](/clis/ampctl#global-flags): `--admin-url`, `--auth-token`, and `--json`, which prints [JSON](/clis/ampctl#json-output) instead of text. `generate` takes none of them, and fails with `error: unexpected argument '--json' found`. ## Hashes Amp identifies each manifest by its hash: the SHA-256 of the manifest's canonical form, 64 hexadecimal characters, like `b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b`. Amp builds the canonical form when you register a manifest: it checks the manifest, fills in the defaults of the fields you left out, drops the fields it doesn't know, and puts the keys in a fixed order. So two files that define the same dataset get the same hash, whatever their whitespace, key order, or default fields, and registering a manifest again stores nothing new. `inspect` and `rm` take a hash. Anything but 64 hexadecimal characters fails before anything is sent. Write it in lowercase, as Amp prints it: a hash in uppercase passes that check, but matches no manifest. When the Admin API requires authentication, [`list`](/clis/ampctl/manifest#ampctl-manifest-list) only shows the manifests that datasets in your token's namespaces use, and counts only those datasets, so it never shows a manifest no dataset uses. The other commands take a hash, and find any manifest by it, whichever datasets use it. [Solo mode](/running-amp/solo-mode) doesn't require authentication, so `list` shows every manifest too. ## `ampctl manifest generate` Writes the manifest of a raw dataset for a provider kind, with every table Amp extracts for that kind. ``` ampctl manifest generate [OPTIONS] --kind KIND --network NETWORK ``` | Flag | Type | Default | Description | | ----------------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--kind ` | string | required | The kind of provider the dataset extracts through, like `--kind evm-rpc`. See [Kinds](/clis/ampctl/manifest#kinds). | | `--network ` | string | required | The network the dataset extracts, like `--network mainnet`: lowercase letters, digits, and hyphens. Use the `network` of the [provider](/clis/ampctl/provider) you registered for it. | | `-o`, `--out ` | path | stdout | Where to write the manifest, like `--out eth_mainnet.json`. A directory that exists gets a file named after the kind, like `evm-rpc.json`. An existing file is overwritten. | | `--start-block ` | integer | `0` | The first block the dataset covers, like `--start-block 24000000`. Amp never extracts the blocks below it. See [Raw Datasets](/datasets/raw-datasets). | | `--finalized-blocks-only` | flag | off | Extracts only the blocks the chain has finalized. The dataset trails the chain's head, but never holds a block that a [reorg](/get-started/concepts/reorgs) replaces. | The kind and network are how a raw dataset finds its providers: when its job starts, Amp extracts through every provider registered with the dataset's kind and network. A network no provider serves fails the job with `No provider found for kind 'evm-rpc' and network 'mainnet'`. Without `--out`, `generate` prints the manifest. Each table lists its columns, so the EVM RPC manifest is about 600 lines long, shortened here: :::terminal ```bash ampctl manifest generate --kind evm-rpc --network mainnet --start-block 24000000 ``` ```json { "kind": "evm-rpc", "network": "mainnet", "start_block": 24000000, "finalized_blocks_only": false, "tables": { "blocks": { "schema": { "arrow": { "fields": [ { "name": "_block_num", "type": "UInt64", "nullable": false }, … { "name": "slot_number", "type": "UInt64", "nullable": true } ] } }, "network": "mainnet", "sorted_by": [ "block_num", "timestamp" ], "bloom_filter_columns": [], "segment_selection_policy": "hash-linked" }, "logs": { … }, "transactions": { … } } } ``` ::: With `--out`, it writes the file and prints nothing. Register the dataset from it, as the [Quickstart](/get-started/quickstart) does: ```bash ampctl manifest generate --kind evm-rpc --network mainnet --start-block 24000000 --out eth_mainnet.json ampctl dataset register acme/eth_mainnet ./eth_mainnet.json --tag 1.0.0 ``` Before registering it, you can edit the file, like to add [bloom filters](/datasets/bloom-filters) to a table's `bloom_filter_columns`. [Manifest Reference](/datasets/manifest-reference) documents every field. ### Kinds `generate` writes manifests for these kinds. Each kind's page lists its tables' columns. | Kind | Tables | Notes | | ------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | `evm-rpc` | [`blocks`, `logs`, `transactions`](/datasets/tables/evm-rpc) | | | `solana` | [`block_headers`, `block_rewards`, `instructions`, `messages`, `transactions`](/datasets/tables/solana) | Leave out `--start-block`. Solana skips slots, and a dataset whose start block is a skipped slot never reads as synced. | | `tempo` | [`blocks`, `logs`, `transactions`](/datasets/tables/tempo) | | | `bittensor` | [`blocks`, `digest_logs`, `events`, `extrinsics`, `runtime_metadata`, `runtime_symbols`, `storage_entries`](/datasets/tables/bittensor) | `storage_entries` stays empty until you add `state_snapshots` selectors to the manifest. See [Bittensor](/providers/bittensor). | | `bitcoin-rpc` | [`blocks`, `inputs`, `outputs`, `transactions`](/datasets/tables/bitcoin-rpc) | | | `eth-beacon` | [`attestations`, `blocks`, `execution_payloads`, `validators`](/datasets/tables/eth-beacon) | Always extracts only finalized slots, with or without `--finalized-blocks-only`. | | `stellar` | [`events`, `ledgers`, `transactions`](/datasets/tables/stellar) | | Every flag applies to every kind, except as the notes say. Kinds are lowercase: `--kind EVM-RPC` fails. `generate` doesn't write other kinds of manifests. You write a derived dataset's manifest yourself, since it holds your SQL (its kind is `manifest`), and the same goes for a [static dataset](/datasets/static-datasets): ``` ✗ Derived datasets don't support dataset generation ``` ``` ✗ Unsupported dataset kind 'static' ``` ### Errors * `error: invalid value 'eth_mainnet' for '--network ': network id must be kebab-case (lowercase alphanumeric and hyphens): 'eth_mainnet'`: the network has a character other than a lowercase letter, a digit, or a hyphen. Network names use hyphens, like `arbitrum-one`. * `✗ Unsupported dataset kind 'evm_rpc'`: the kind isn't one of the [kinds](/clis/ampctl/manifest#kinds) above. Check its spelling. * `✗ Failed to write output`, then `→ No such file or directory (os error 2)`: the directory in `--out` doesn't exist. Create it first. ## `ampctl manifest register` Stores a manifest without linking it to a dataset, and prints its [hash](/clis/ampctl/manifest#hashes). ``` ampctl manifest register [OPTIONS] PATH ``` | Argument | Type | Default | Description | | -------- | ----------- | -------- | ---------------------------------------------------------------------------------------------------------------------- | | `PATH` | path or URL | required | The manifest file, in JSON, like `./eth_mainnet.json`: a local path, or a `file://`, `s3://`, `gs://`, or `az://` URL. | `register` takes any kind of manifest: raw, derived, or static. It reads a file, not standard input, so save the output of [`generate`](/clis/ampctl/manifest#ampctl-manifest-generate) with `--out` first. For a file in object storage, `ampctl` reads it itself, with the credentials in its environment, like `AWS_ACCESS_KEY_ID`, `GOOGLE_APPLICATION_CREDENTIALS`, or `AZURE_STORAGE_ACCOUNT_NAME`. :::terminal ```bash ampctl manifest register ./eth_mainnet.json ``` ``` ✓ Manifest registered successfully → Manifest hash: b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b ``` ::: Amp checks the manifest before storing it: its JSON, its fields, that each table's network is the manifest's, and, for a derived dataset, that the datasets it depends on exist. Registering a manifest that's already stored succeeds and prints the same hash. With `--json`, it prints the hash: :::terminal ```bash ampctl manifest register ./eth_mainnet.json --json ``` ```json { "hash": "b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b" } ``` ::: Pass the hash to `ampctl dataset register` instead of the file, and Amp links the stored manifest without receiving it again: ```bash HASH=$(ampctl manifest register ./eth_mainnet.json --json | jq -r .hash) ampctl dataset register acme/eth_mainnet "$HASH" --tag 1.0.0 ``` A manifest no dataset uses is orphaned, and [`prune`](/clis/ampctl/manifest#ampctl-manifest-prune) deletes it, so link it soon after you register it. ### Errors * `error: invalid value './eth_mainet.json' for '': invalid manifest file path: ./eth_mainet.json`: the local file doesn't exist. Check the path. * `✗ manifest registration failed`, then `→ invalid manifest JSON`: the file isn't JSON, or has a syntax error. The next line says where, like `→ EOF while parsing a value at line 2 column 0`. A TOML file fails this way too. * `→ invalid manifest`, then a line like ``→ INVALID_MANIFEST: invalid manifest: missing field `kind` at line 1 column 33``: a field is missing or has the wrong type. See [Manifest Reference](/datasets/manifest-reference). * `→ dependency validation error`, then a line starting with `→ MANIFEST_VALIDATION_ERROR:`: the manifest doesn't hold together. The line says why, like `table 'blocks' belongs to network 'base', but dataset network is 'mainnet'`, `raw dataset manifest must declare at least one table`, or, for a derived dataset, `Dependency 'eth' not found (acme/eth_polygon@1.0.0)`. Register the dependency first. ## `ampctl manifest list` Lists every stored manifest, sorted by hash, with how many datasets use it. ``` ampctl manifest list [OPTIONS] ``` It takes no arguments. `ampctl manifest ls` is the same command. :::terminal ```bash ampctl manifest list ``` ``` Manifests: 24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1 - 0 datasets b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b - 1 dataset fceb18fb7f3d2be31ea324a0d8760ff3e3440c212a7bea0a0b65016ff613af26 - 1 dataset ``` ::: The count is of datasets, not versions: a manifest that several versions of `acme/eth_mainnet` use counts once. A manifest with `0 datasets` is orphaned, like the first one here, registered with [`register`](/clis/ampctl/manifest#ampctl-manifest-register) and never used. With no manifests stored, `list` prints `No manifests found`. With `--json`, each manifest has its `hash` and `dataset_count`, so you can list the orphaned ones: :::terminal ```bash ampctl manifest list --json | jq -r '.manifests[] | select(.dataset_count == 0) | .hash' ``` ``` 24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1 ``` ::: ```json { "manifests": [ { "hash": "24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1", "dataset_count": 0 }, { "hash": "b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b", "dataset_count": 1 }, { "hash": "fceb18fb7f3d2be31ea324a0d8760ff3e3440c212a7bea0a0b65016ff613af26", "dataset_count": 1 } ] } ``` ## `ampctl manifest inspect` Prints a stored manifest, in its canonical form. ``` ampctl manifest inspect [OPTIONS] HASH ``` | Argument | Type | Default | Description | | -------- | ------ | -------- | -------------------------------------------------------------------------------------------------------- | | `HASH` | string | required | The manifest's [hash](/clis/ampctl/manifest#hashes), like `b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b`. | `ampctl manifest get` is the same command. `inspect` prints the manifest as indented JSON, like [`generate`](/clis/ampctl/manifest#ampctl-manifest-generate). Use `jq` to read parts of it: :::terminal ```bash ampctl manifest inspect b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b | jq '{kind, network, start_block, tables: (.tables | keys)}' ``` ```json { "kind": "evm-rpc", "network": "mainnet", "start_block": 24000000, "tables": ["blocks", "logs", "transactions"] } ``` ::: To read the manifest of a dataset by its name instead, use [`ampctl dataset manifest`](/clis/ampctl/dataset), like `ampctl dataset manifest acme/eth_mainnet@1.0.0`. With `--json`, `inspect` prints the manifest on one line, exactly as Amp hashed it. To check that a manifest matches its hash, hash that line without its final newline: :::terminal ```bash ampctl manifest inspect b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b --json | tr -d '\n' | shasum -a 256 ``` ``` b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b - ``` ::: A hash that matches no manifest fails, with exit code 1. ## `ampctl manifest rm` Deletes a manifest that no dataset uses. ``` ampctl manifest rm [OPTIONS] HASH ``` | Argument | Type | Default | Description | | -------- | ------ | -------- | -------------------------------------------------------------------------------------------------------- | | `HASH` | string | required | The manifest's [hash](/clis/ampctl/manifest#hashes), like `24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1`. | `ampctl manifest remove` is the same command. It deletes without asking first. ```bash ampctl manifest rm 24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1 ``` Deleting a manifest that doesn't exist succeeds too, so you can run it again safely. With `--json`, it prints the hash: `{"hash":"24bd2593439e7759a303d115461ac47eee3b272e81e069b147ae1c83e58ccaf1"}`. A manifest a dataset uses can't be deleted, and `rm` fails with exit code 1: ``` ✗ API error: [MANIFEST_LINKED] manifest 'b9e2ebd92c0752ba8a61a233f9cc5d3f4f8b601b1b789a882a5dd8ece9bca56b' is linked to datasets and cannot be deleted ``` To delete it, remove the datasets that use it with [`ampctl dataset remove`](/clis/ampctl/dataset). That deletes the manifests no other dataset uses, so you don't need `rm` afterwards. ## `ampctl manifest prune` Deletes every manifest that no dataset uses. ``` ampctl manifest prune [OPTIONS] ``` It takes no arguments, and deletes without asking first. That includes the manifests you registered with [`register`](/clis/ampctl/manifest#ampctl-manifest-register) and haven't used yet. To see what it deletes, list the orphaned manifests with [`list`](/clis/ampctl/manifest#ampctl-manifest-list) first. :::terminal ```bash ampctl manifest prune ``` ``` ✓ Pruned 2 orphaned manifest(s) ``` ::: If Amp fails to delete a manifest, `prune` goes on with the others, and doesn't count it. Run it again to retry. With nothing to delete, it prints `✓ Pruned 0 orphaned manifest(s)`, so it's safe to run on a schedule. With `--json`, it prints the count: `{"deleted_count":2}`.