# Batch Testing

> How to configure and debug batch consumer scenarios in KTestify.

Source: https://docs.ktestify.xyz/docs/write-tests/advanced/batch-testing

KTestify supports a **batch consumer mode** where a single assertion step collects a configurable number of records before running assertions. This is useful for pipelines that produce multiple output records from a single input (fan-out) or for asserting ordering guarantees.

---

## How batch mode works

When `expectedRecordsCount > 1` in a batch step, `KafkaRecordFetcher` switches to batch mode:

1. Polls continuously until `batchSize` (= `expectedRecordsCount`) records have been collected.
2. Applies deduplication, no two steps in the same scenario can match the same physical record (topic + partition + offset).
3. If the timeout expires before enough records are collected, fails with:
   > `Timed out after Nms waiting for 4 record(s), only 2 collected.`

---

## Using batch steps

### Raw batch

```gherkin
Then expected records from files
  | topicAlias | expectedRecordsCount | files                              | consumerReadTimeout |
  | orders-out | 3                    | order-1.json,order-2.json,order-3.json | 60              |
```

### Avro batch

```gherkin
Then expected records from files based on schema
  | topicAlias | expectedRecordsCount | files                 | excludedKeys | batchMatchStrategy | consumerReadTimeout |
  | orders-out | 2                    | exp-1.json,exp-2.json | ts           | lazy               | 60                  |
```

---

## Batch matching strategies: positional vs lazy

By default (`batchMatchStrategy = positional`), records and files are matched **by position** in the order they are fetched from Kafka (`record[0]` against `file[0]`, `record[1]` against `file[1]`, and so on).

When a topic has multiple partitions or an upstream pipeline emits records out of order, set `batchMatchStrategy` to `lazy` (aliases: `fuzzy`, `unordered`, `any-order`):

- **Positional (`positional`, default):** requires a single-partition topic or deterministic emission order. Record `i` must match `files[i]` (and `expectedRecordKeys[i]` if provided).
- **Lazy (`lazy`):** performs order-independent 1-to-1 matching without replacement. Each consumed record is compared against all unclaimed expected files (and corresponding `expectedRecordKeys` entries if provided) and claims the first one that matches. Duplicate records will not match the same expected file twice.

Batch steps also support `expectedRecordKeys` (comma-separated, one key per expected file in `files`), `expectedRecordKey` (single filter for all records), and `keyMatchStrategy` so the consumer only collects batch records whose keys match and asserts each record's key alongside its value.

---

## Deduplication across steps

The deduplication registry (`KafkaRecordFetcher.MATCHED_RECORDS`) is a static `ConcurrentHashSet` shared across all consumer instances within the same JVM run. It is cleared in the `@Before` hook at the start of each scenario.

This means:
- One step cannot accidentally consume a record that was already matched by a previous step in the **same scenario**.
- Records from a **previous scenario** are always cleared, no bleed-through.

---

## Recommended topic setup for batch tests

```
docker/init-topics.sh creates:  ktestify.raw-batch  (3 partitions, 1 replication factor)
```

Single-partition topics give deterministic ordering. Three-partition topics test ordering-agnostic scenarios.

---

## See also

- [Batch assertions →](../assertions/batch-assertions)
- [Timeout tuning →](timeout-tuning)
- [Multi-row DataTables →](multi-row-datatables)