Skip to main content

Batch Assertions

Batch steps consume multiple records from an output topic in a single step and match each one against a corresponding expected file. Records and files are matched positionally (record[0] → file[0], etc.).

Raw batch : Then expected records from files​

Then expected records from files
  | topicAlias | expectedRecordsCount | files                            | consumerReadTimeout |
  | orders-out | 3                    | exp-1.json,exp-2.json,exp-3.json | 60                  |
ColumnTypeRequiredDescription
topicAliasstring✅Alias of a declared output topic
expectedRecordsCountinteger✅Exact number of records to collect before asserting
filesstring✅Comma-separated list of expected file paths (relative to assets dir), one per record
consumerReadTimeoutinteger❌Max seconds to wait for all records
consumerDeltaTimeinteger❌How far back (seconds) to seek

Avro batch : Then expected records from files based on schema​

Then expected records from files based on schema
  | topicAlias | expectedRecordsCount | files                    | excludedKeys |
  | orders-out | 2                    | exp-1.json,exp-2.json    | ts,eventId   |
ColumnTypeRequiredDescription
topicAliasstring✅Alias of a declared output topic
expectedRecordsCountinteger✅Exact number of records to collect
filesstring✅Comma-separated list of expected JSON file paths
excludedKeysstring❌Comma-separated Avro field names to exclude from all records
consumerReadTimeoutinteger❌Max seconds to wait for all records
consumerDeltaTimeinteger❌How far back (seconds) to seek

Timeout behaviour​

  • If the consumer collects fewer than expectedRecordsCount records before the timeout, the step fails with:

    Timed out after Nms waiting for 4 record(s), only 2 collected.

  • Increase consumerReadTimeout if your pipeline produces output slowly.

Single-row only​

Unlike single-record assertion steps (see Multi-row DataTables), batch steps accept exactly one DataTable row. That one row already describes an any-to-all match across expectedRecordsCount records collected from a single consumer call, adding more rows would be ambiguous, so a multi-row DataTable here is rejected immediately:

This step only supports a single DataTable row but 2 rows were provided.

If you need several independent batches, or several unrelated single records against the same topic, use separate step invocations, or the multi-row single-record assertion steps (raw matchers →, Avro matchers →) instead.

Ordering caveat​

Records are matched positionally in the order they are fetched from Kafka partitions. If your pipeline does not guarantee ordering (e.g. multiple partitions), use individual assertion steps per record with expectedRecordKey filtering instead.

See also​