> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need. # Manifests A manifest is a JSON document that defines a [dataset](/get-started/concepts/datasets): what kind of data it holds, its tables, and each table's columns. You register a manifest to publish a dataset, and Amp uses it to fill the tables and to answer queries about them. Every field is listed in the [Manifest Reference](/datasets/manifest-reference). ## What a manifest contains Every manifest has a `kind` and its `tables`. The rest depends on the kind: | `kind` | Dataset | Its tables come from | | ---------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------ | | `evm-rpc`, `solana`, `tempo`, `bittensor`, `bitcoin-rpc`, `eth-beacon`, `stellar`, or `phaser` | [Raw](/datasets/raw-datasets) | A chain, through a [provider](/providers) of the same kind and `network` | | `manifest` | [Derived](/datasets/derived-datasets) | SQL over other datasets, its `dependencies` | | `static` | [Static](/datasets/static-datasets) | CSV files in object storage | This manifest defines a derived dataset with one table, `block_gas`, which copies three columns of another dataset's `blocks` table: ```json [block_gas.json] { "kind": "manifest", "dependencies": { "eth": "acme/eth_mainnet@1.0.0" }, "tables": { "block_gas": { "input": { "sql": "SELECT block_num, gas_used, gas_limit FROM eth.blocks" }, "schema": { "arrow": { "fields": [ { "name": "_block_num", "type": "UInt64", "nullable": false }, { "name": "_ts", "type": { "Timestamp": ["Nanosecond", "+00:00"] }, "nullable": false }, { "name": "block_num", "type": "UInt64", "nullable": false }, { "name": "gas_used", "type": "UInt64", "nullable": false }, { "name": "gas_limit", "type": "UInt64", "nullable": false } ] } } } } } ``` You rarely write a whole manifest by hand. `ampctl manifest generate` writes a raw dataset's, and the Admin API infers a derived table's columns from its SQL (see [Derived Datasets](/datasets/derived-datasets#get-the-tables-schema)). ## Tables `tables` maps each table's name to its definition. A table's name is how you query it, like `"acme/block_gas".block_gas`. It's 1 to 63 characters long, starts with a letter or an underscore, and holds only letters, digits, underscores, and `$`. Each table has a `schema`, which lists its columns in order. Each column has a `name`, a `type`, and whether it's `nullable`, that is, whether it can be `NULL`: ```json { "name": "base_fee_per_gas", "type": { "Decimal128": [38, 0] }, "nullable": true } ``` ### Reserved columns Amp adds two columns to raw tables, and derived tables carry them over from the tables they read: * `_block_num`: the number of the block the row comes from. Amp uses it to order the data, split it into files, and track which blocks a table has. * `_ts`: the block's time, for chains that timestamp their blocks. A table that reads several networks orders its data by `_ts` instead of `_block_num`, since block numbers can't be compared across networks. Which ones a derived table's `schema` needs depends on how many networks the table reads: * **One network**: `_block_num`, and `_ts` if the tables it reads have it, or a `timestamp` column in its place. Without `_block_num`, the job that fills the table fails. Without `_ts`, the table still fills, but you can't select `_ts` from it, and a table that reads it across networks can't fill. * **Several networks**: `_ts`. Amp doesn't carry `_block_num` over, so the schema only has it if the query selects it. Without `_ts`, or with a `_block_num` the query doesn't select, the job can't fill the table. The schema [the Admin API returns](/datasets/derived-datasets#get-the-tables-schema) has the right ones, so use it. [Errors](/datasets/derived-datasets#errors) lists what a job says otherwise. Static tables have neither column, since their data isn't tied to blocks. ## Data types Column types are [Apache Arrow](https://arrow.apache.org/) types, the format Amp stores and returns data in. A type without parameters is a string, like `"UInt64"`. A type with parameters is an object with the type's name as its only key, like `{ "FixedSizeBinary": 32 }`. The types chain data uses most: | Type in a manifest | What it holds | Example columns | | ------------------------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------- | | `"Boolean"` | `true` or `false` | `status` of an EVM transaction | | `"UInt8"` … `"UInt64"` | Whole numbers from 0, in 8 to 64 bits | `block_num`, `gas_used` | | `"Int8"` … `"Int64"` | Whole numbers, negative too | `lamports` of a Solana reward | | `"Float32"`, `"Float64"` | Numbers with decimals, approximated | | | `{ "Decimal128": [38, 0] }` | Exact numbers of up to 38 digits, here with 0 after the decimal point | `gas_price`, `base_fee_per_gas` | | `"Utf8"` | Text | `block_hash` of a Solana block | | `"Binary"` | Bytes, any number of them | `input` of an EVM transaction | | `{ "FixedSizeBinary": 32 }` | Exactly this many bytes, like a 32-byte hash or a 20-byte address | `hash`, `miner` | | `{ "Timestamp": ["Nanosecond", "+00:00"] }` | A point in time, to the nanosecond, in UTC | `timestamp`, `_ts` | | `{ "List": { … } }` | A list of values of one type, given as a column (`name`, `data_type`, `nullable`, and others) | `access_list`, `blob_versioned_hashes` | | `{ "Struct": [ … ] }` | A group of named values, given as a list of columns | the items of an EVM `access_list` | The [table schema pages](/datasets/tables/evm-rpc) list every raw table's columns with their types. Types are written the way [Arrow's Rust library](https://docs.rs/arrow-schema/latest/arrow_schema/enum.DataType.html) names them: a string like `"Decimal128(38, 0)"`, as some tools print types, doesn't work in a manifest. ## Manifest hashes Amp identifies a manifest by its hash: the SHA-256 hash of its JSON, after Amp puts the JSON in a canonical form. Two manifests with the same content have the same hash, whatever their formatting, and Amp stores them once. Changing anything in a manifest, even a raw dataset's start block, gives it a new hash. A hash always names the same manifest, so refer to a dataset by its manifest hash to be sure its definition never changes. A [version](/datasets/versioning), like `1.0.0`, points to one manifest hash at a time, but registering another manifest with the same version moves it to that manifest. ## Register a manifest Register a manifest under a dataset's name with `ampctl dataset register`. Amp checks it before storing it: its JSON, its kind's fields, its tables' SQL for derived datasets, and its bloom filter columns. ```bash ampctl dataset register acme/block_gas ./block_gas.json --tag 1.0.0 ``` [Managing Datasets](/managing-amp/datasets) covers registering, deploying, and removing datasets, and [Manifest Storage](/managing-amp/manifests) covers storing manifests on their own, by hash. ## Errors ### `INVALID_MANIFEST: invalid manifest: unknown variant` A column's `type` isn't an Arrow type Amp knows, like `"Timestamp(Nanosecond, Some(\"+00:00\"))"`. The message lists the type names Amp accepts. Write types with parameters as objects, like `{ "Timestamp": ["Nanosecond", "+00:00"] }` (see [Data types](/datasets/manifests#data-types)). ### `MANIFEST_VALIDATION_ERROR` Part of the manifest is valid JSON but doesn't make sense, like a [bloom filter](/datasets/bloom-filters) on a column the table doesn't have. The message names the table and the field.