> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://docs.ampdb.io/api/mcp` to find what you need.

# Redis Loader

The Redis loader writes query results into Redis, an in-memory key-value store. Choose it to serve recent chain data
to an application with low latency, like the latest blocks or a leaderboard of token holders. It stores each row as one
Redis value, in the data structure you choose: a hash per row by default, or a JSON string, or entries of a stream, a
set, a sorted set, or a list.

## Prerequisites

* **The Python SDK**, [installed](/sdks/python#installation), and the Redis client it uses, `redis`:

  :::code-group
  ```bash [pip]
  pip install redis
  ```

  ```bash [uv]
  uv add redis
  ```
  :::

* **A running Amp with the Quickstart's dataset**, `acme/eth_mainnet`. [Quickstart](/get-started/quickstart) sets it up.

* **A Redis server you can write to.** To run one locally in Docker:

  ```bash
  docker run --name amp-loader-redis -p 6379:6379 -d redis:7
  ```

## Load data

This script names a connection to your Redis server, `local_redis`, then loads the Quickstart's 100 blocks, one hash per
block:

```python [load_redis.py]
from amp import Client

client = Client('grpc://localhost:1602')

client.configure_connection(
    'local_redis',
    'redis',
    {'host': 'localhost', 'key_pattern': '{table}:{block_num}'},
)

result = client.sql(
    'SELECT block_num, evm_decode_hex(hash) AS hash, timestamp, gas_used FROM "acme/eth_mainnet".blocks'
).load(connection='local_redis', destination='blocks', read_all=True)

print(result)
```

```bash
python load_redis.py
```

* `destination` names the data, like a table: `blocks`. The key pattern uses it.
* `key_pattern` builds each row's key from `{table}`, the destination, and `{block_num}`, the row's `block_num`. Block
  24,000,099's hash is at `blocks:24000099`.
* `evm_decode_hex(hash)` turns the binary block hash into the familiar `0x…` text (see
  [EVM functions](/querying/functions/evm)). Without it, Redis stores the hash's raw bytes.
* `read_all=True` loads the whole result at once and returns one `LoadResult`. [Loaders](/sdks/python/loaders) covers
  the other ways to load, and the options `load()` takes.

Read a block's hash:

:::terminal
```bash
docker exec amp-loader-redis redis-cli HGETALL blocks:24000099
```

```
gas_used
10751338
hash
0x6ed4b3511195052eea47925fe5035f5e3fc528440eed62649ba82058a1b70e40
timestamp
2025-12-13 00:25:59+00:00
block_num
24000099
```
:::

Redis has no tables or columns, so the loader creates nothing in advance.

## Keys

For hashes, JSON strings, and RedisJSON documents, each row gets its own key, built from `key_pattern`:

* `{table}` is replaced with the destination.
* `{column}`, for any column of your query, is replaced with the row's value in that column. A binary value is replaced
  with the first 8 hexadecimal characters of its MD5 hash, to keep keys short. A null value is replaced with `null`.

The default pattern, `{table}:{id}`, needs an `id` column: rename one in your query, like `SELECT block_num AS id, …`,
or set a pattern with your own columns. Pick columns that identify each row: rows with the same key overwrite each
other.

The other data structures store every row under one key: `{destination}:stream`, `{destination}:set`,
`{destination}:zset`, or `{destination}:list`.

## Data structures

Set `data_structure` to store rows another way:

| `data_structure` | Key                    | What each row becomes                                                                                                       |
| ---------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `hash` (default) | From `key_pattern`     | A hash with a field per non-null column. Binary values are stored as raw bytes, and other values as text.                   |
| `string`         | From `key_pattern`     | A JSON object, with binary values as hexadecimal text, without `0x`.                                                        |
| `json`           | From `key_pattern`     | A RedisJSON document. Needs the RedisJSON module, and fails on timestamp columns (see [Troubleshooting](/sdks/python/loaders/redis#troubleshooting)). |
| `stream`         | `{destination}:stream` | An entry of a Redis stream, with a field per non-null column, added with an ID Redis generates.                             |
| `set`            | `{destination}:set`    | A member of a set: the row's value in `unique_field`, or else the whole row as JSON.                                        |
| `sorted_set`     | `{destination}:zset`   | A member of a sorted set, the whole row as JSON, scored with its value in `score_field`.                                    |
| `list`           | `{destination}:list`   | An element appended to a list, the whole row as JSON.                                                                       |

For example, to rank blocks by the gas they used:

```python
client.configure_connection(
    'gas_ranking',
    'redis',
    {'host': 'localhost', 'data_structure': 'sorted_set', 'score_field': 'gas_used'},
)
```

Without `score_field`, a sorted set scores each row with its position in its batch, which isn't meaningful across
batches.

## Configuration

The third argument of `configure_connection()`:

| Key                      | Type    | Default          | Description                                                                                                                          |
| ------------------------ | ------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `host`                   | string  | `'localhost'`    | The server's host name or IP address.                                                                                                |
| `port`                   | integer | `6379`           | The server's port, like `6380`.                                                                                                      |
| `db`                     | integer | `0`              | The Redis database number, like `1`.                                                                                                 |
| `password`               | string  | none             | The server's password, if it has one. Read it from an environment variable, like `os.environ['REDIS_PASSWORD']`.                     |
| `data_structure`         | string  | `'hash'`         | How to store rows: `hash`, `string`, `json`, `stream`, `set`, `sorted_set`, or `list` (see [Data structures](/sdks/python/loaders/redis#data-structures)).     |
| `key_pattern`            | string  | `'{table}:{id}'` | How to build each row's key, for `hash`, `string`, and `json` (see [Keys](/sdks/python/loaders/redis#keys)), like `'{table}:{block_num}'`.                     |
| `ttl`                    | integer | none             | How many seconds the keys the loader writes live before Redis deletes them, like `3600`. Without it, they never expire.              |
| `score_field`            | string  | none             | For `sorted_set`: the column whose value scores each row, like `'gas_used'`.                                                         |
| `unique_field`           | string  | none             | For `set`: the column whose value is the set's member, like `'hash'`. Without it, the member is the whole row as JSON.               |
| `pipeline_size`          | integer | `1000`           | How many commands the loader sends to Redis at once, like `5000`.                                                                    |
| `max_connections`        | integer | `20`             | The most connections the loader opens to the server at once.                                                                         |
| `socket_connect_timeout` | integer | `5`              | How many seconds to wait for a connection to the server.                                                                             |
| `socket_timeout`         | integer | `5`              | How many seconds to wait for the server to answer a command.                                                                         |
| `decode_responses`       | boolean | `False`          | Whether the loader's Redis client decodes the server's answers to text. Leave it off: binary values need it off.                     |
| `connection_params`      | dict    | `{}`             | Other arguments for the [redis-py](https://redis.readthedocs.io) connection pool, passed as they are, like `{'username': 'loader'}`. |
| `batch_size`             | integer | `1000`           | Not used. Pass `batch_size` to `load()` instead (see [Loaders](/sdks/python/loaders)).                                               |

The `state` and `resilience` keys, which every loader takes, are on [Loaders](/sdks/python/loaders).

With `ttl`, the loader sets the expiry each time it writes a key, or, for the data structures that store every row under
one key, each time it writes a batch.

## Load modes

Pass `mode` to `load()` ([Loaders](/sdks/python/loaders#load-modes) explains the modes every loader shares):

* **`append`** (the default) writes the rows, keeping existing keys. A row whose key already exists updates it.
* **`overwrite`** deletes the destination's existing data, then writes the rows. For `hash`, `string`, and `json`, it
  deletes the keys matching `key_pattern`, with `{table}` replaced and `{id}`, `{block_num}`, `{block_hash}`, and
  `{user_id}` replaced with `*`. Other column placeholders aren't replaced, so their keys aren't deleted. For the other
  data structures, it deletes the destination's one key.

  Use `overwrite` only with `read_all=True`. Without it, the loader deletes the data before each batch, so only the
  last batch stays, and the last batch is often empty.

`upsert` and `merge` aren't supported: they fail with `Unsupported mode LoadMode.UPSERT`.

## Streaming and reorgs (Unstable)

With `stream=True`, `load()` runs a [streaming query](/querying/streaming-queries) and keeps loading new rows as Amp
extracts new blocks (see [Streaming](/sdks/python/streaming)). Streamed rows get one more field, `_amp_batch_id`, which
identifies the block range each row came from:

```python
results = client.sql(
    'SELECT block_num, evm_decode_hex(hash) AS hash, timestamp, gas_used FROM "acme/eth_mainnet".blocks'
).load(connection='local_redis', destination='blocks', stream=True, with_reorg_detection=False)
for result in results:
    print(result)
```

`with_reorg_detection=False` works around a problem with reorg detection, explained in
[Known issues](/sdks/python/streaming#known-issues).

When Amp reports a [reorg](/get-started/concepts/reorgs), the loader looks up the batches it loaded from the reorged
blocks onward, then scans every key starting with `{destination}:` and deletes those whose `_amp_batch_id` matches. That
scan reads every key of the destination, so it takes longer the more keys you have. It only works for `hash` and
`string`, and only when `key_pattern` starts with `{table}:`: for the other data structures, reorged rows stay.

The loader remembers which batches it loaded in memory, for as long as the `load()` call runs. When a stream restarts,
it starts from the beginning of the query, rewriting the same keys, and can't undo reorgs of the batches it loaded
before.

## Troubleshooting

### `Key pattern contains {id} placeholder but no 'id' field found in data`

The default key pattern, `{table}:{id}`, needs an `id` column. Set `key_pattern` with your query's columns, like
`'{table}:{block_num}'`, or add an `id` column, like `SELECT block_num AS id, …`.

### `ID value is None at row 0. Redis keys require non-null IDs.`

A row's `id` is null. Filter those rows out in your query, like `WHERE id IS NOT NULL`, or use another column in the key
pattern.

### `Object of type datetime is not JSON serializable`

The `json` data structure can't store timestamp columns. Use `string`, which stores the same JSON object as text, or
convert timestamps to text in your query, like `CAST(timestamp AS VARCHAR) AS timestamp`.

### `unknown command 'JSON.SET'`

The `json` data structure needs the [RedisJSON](https://redis.io/docs/latest/develop/data-types/json/) module, which
your Redis server doesn't have. Use `string`, or a Redis with the module, like Redis 8 or Redis Stack.

### `Error … connecting to localhost:6379. Connection refused.`

Redis isn't running, or listens on another port. Start it, or set `port` to the port it listens on.