> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need.

# Metadata Caching

Before a query reads any data, Amp plans it: it finds the Parquet files of each table the query reads, and uses each
file's metadata to skip the files and parts of files that can't match. Fetching that information from the
[metadata database](/running-amp/metadata-database) for every query would slow queries down and load the database, so
each service keeps two caches in memory:

* **[Parquet metadata](/running-amp/storage/metadata-caching#parquet-metadata)**: each file's schema, structure, and statistics.
* **[File lists](/running-amp/storage/metadata-caching#file-lists)**: which files each table has.

Both work on their own, and their defaults suit most deployments. You only need this page to size them, or to read
their [metrics](/running-amp/storage/metadata-caching#monitor-the-caches).

Every service has its own caches: query servers use them to plan queries, workers to read the datasets they transform
and compact, and the controller to answer the [Admin API](/managing-amp/admin-api). They start empty when the service
starts, and fill as it works. By default, each can grow to 1 GiB, so count up to 2 GiB per service when you size its
memory.

## Parquet metadata

A Parquet file ends with a footer: the file's schema, how its rows are split into row groups, and statistics like each
column's smallest and largest values in each row group. A query uses the statistics to skip the row groups that can't
match its filters, like the blocks outside the range it asks for.

When a worker writes a file, it records the footer in the metadata database, along with the file. So to plan a query, a
service never reads footers from [storage](/running-amp/storage), only from the database, and only once:

1. It looks up the file in its cache.
2. On a miss, it fetches the footer from the metadata database, reads it, and adds it to the cache.

Files never change once they're written, so a cached footer never goes out of date. Queries still read the files' rows
from storage.

When a query misses many files at once, the service fetches their footers on at most half of its database connections
(its [`pool_size`](/running-amp/metadata-database#pool_size), divided by 2), so they don't take the connections that
record new files. Requests for the same file share one fetch.

### Parquet metadata settings

They're in the `[writer]` section, and apply to every service:

| Setting                                                                                                | Default | Description                                                                                          |
| ------------------------------------------------------------------------------------------------------ | ------- | ---------------------------------------------------------------------------------------------------- |
| [`writer.cache_size_mb`](/running-amp/configuration-reference#writercache_size_mb)                     | `1024`  | The cache's size, in MiB. When it's full, the cache drops the entries least likely to be used again. |
| [`writer.cache_time_to_idle_secs`](/running-amp/configuration-reference#writercache_time_to_idle_secs) | `120`   | How long, in seconds, an entry stays in the cache without being used.                                |

```toml [config.toml]
[writer]
cache_size_mb = 4096
cache_time_to_idle_secs = 600
```

A larger cache keeps more footers, so fewer queries wait for the database. A smaller one saves memory, at the cost of
more fetches.

## File lists

A table's files change as workers write and [compact](/running-amp/storage/compaction) them, and a large table has
many. Rather than reading the whole list for every query, each service caches the list of every table it reads, and
keeps it current with what changed since:

* **The metadata database numbers each change.** Each table's data, a
  [revision](/managing-amp/table-revisions), has a version that goes up whenever files are added to it or removed. A
  change log records which files each version added and removed.
* **A service checks the version before it uses a cached list.** When it hasn't changed, the service uses the list as
  it is. When it has, the service applies the changes logged since, rather than reading the whole list.

A service reads a table's whole list when it hasn't cached it yet, when its list is more than
[`max_delta_versions`](/running-amp/storage/metadata-caching#file-list-settings) versions behind, or when its list no longer matches the database. The
controller prunes the change log every hour, keeping the last `max_delta_versions` versions of each revision.

### File list settings

They're in the `[location_membership]` section:

| Setting                                                                                                      | Default | Description                                                                                                                         |
| ------------------------------------------------------------------------------------------------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| [`cache_size_mb`](/running-amp/configuration-reference#location_membershipcache_size_mb)                     | `1024`  | The cache's size, in MiB.                                                                                                           |
| [`cache_time_to_idle_secs`](/running-amp/configuration-reference#location_membershipcache_time_to_idle_secs) | `120`   | How long, in seconds, a list stays in the cache without being used.                                                                 |
| [`max_delta_versions`](/running-amp/configuration-reference#location_membershipmax_delta_versions)           | `10000` | How many versions a service catches up on with the change log, and how many the log keeps for each revision. At least `1`.          |
| [`log_prune_interval_secs`](/running-amp/configuration-reference#location_membershiplog_prune_interval_secs) | `3600`  | How often, in seconds, the controller prunes the change log. The log keeps `max_delta_versions` versions however often it's pruned. |

```toml [config.toml]
[location_membership]
cache_size_mb = 2048
```

The cache counts each list at a fixed cost per file, above what it really uses, so it uses less memory than its size.

Raising `max_delta_versions` lets services that fall far behind catch up from the log, but makes the log larger.
Lowering it keeps the log small, but has those services read whole lists. Either way, queries see the same files.

## Monitor the caches

Each cache has [metrics](/running-amp/metrics#data-store-metrics):

* **Parquet metadata**: `parquet_footer_cache_hits_total` and `parquet_footer_cache_misses_total` count lookups, and
  `parquet_footer_cache_usage_bytes` is the cache's size.
* **File lists**: `membership_resolve_total` counts how each list was found, in its `outcome` label, and
  `membership_cache_usage_bytes` is the cache's size.

A cache whose size stays at its setting (`cache_size_mb` times 1,048,576 bytes) drops entries to make room. If its
misses, or `membership_resolve_total{outcome="reread_not_cached"}`, rise at the same time, give it more memory. Misses
right after a service starts are expected: its caches start empty.

## Troubleshooting

### `timed out waiting 5s for footer query admission`

A query needed more footers than the service could fetch within its database
[`acquire_timeout_secs`](/running-amp/metadata-database#acquire_timeout_secs). Raise the service's
[`pool_size`](/running-amp/metadata-database#per-service-pools), which also raises how many footers it fetches at once,
or `writer.cache_size_mb`, so fewer queries miss.