Azure Blob Storage Plugin

ktestify-plugin-azureblob is a first-party KTestify plugin that integrates Azure Blob Storage into your Kafka testing workflows. It allows you to upload blobs as test fixtures before producing Kafka messages, and assert that your pipeline correctly wrote output blobs.

Work in progress

This plugin is actively being developed. The API surface, step definitions, and configuration keys may evolve before the first stable release.

When to Use

Use the Azure Blob plugin when your system,

Reads input files from Azure Blob Storage before processing Kafka messages
Writes result files to Azure Blob Storage after consuming Kafka topics
Needs to validate blob content as part of end-to-end test scenarios
Must test Azure Storage and Kafka integration together

Installation

Maven Dependency

<dependency>
    <groupId>io.github.ktestify</groupId>
    <artifactId>ktestify-plugin-azureblob</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <scope>test</scope>
</dependency>

With ktestify-cucumber (Docker)

The Azure Blob plugin is considered a first-party plugin and is bundled in the ktestify-cucumber fat JAR. No additional installation is required.

Configuration

The plugin reads settings from ktestify.plugins.azure-blob in your HOCON configuration file. Place your configuration in application.conf or override via environment variables.

Configuration Keys

ktestify.plugins.azure-blob {
  # Connection string (primary, recommended)
  # Takes precedence over SAS token or account key approaches
  connection-string = "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net"
  connection-string = ${?AZURE_STORAGE_CONNECTION_STRING}

  # Alternatively: SAS token (when connection-string is not set)
  sas-token = ""
  sas-token = ${?AZURE_STORAGE_SAS_TOKEN}

  # Storage account endpoint (required when using SAS or managed identity)
  endpoint = "https://myaccount.blob.core.windows.net"
  endpoint = ${?AZURE_STORAGE_ENDPOINT}

  # Timeout for individual blob operations (upload, download, list)
  request-timeout = 30s
  request-timeout = ${?KTESTIFY_AZURE_BLOB_REQUEST_TIMEOUT}

  # Automatically create containers if they don't exist
  auto-create-containers = true
  auto-create-containers = ${?KTESTIFY_AZURE_BLOB_AUTO_CREATE}
}

Configuration Priority

System properties (e.g. -Dktestify.plugins.azure-blob.connection-string=...)
Environment variables (e.g. AZURE_STORAGE_CONNECTION_STRING)
application.conf in classpath
Plugin's reference.conf (defaults)

Gherkin Steps

Background, Register a Blob Container

Before using the Azure Blob steps, register a container with an alias in your scenario background,

Background:
  Given Azure Blob container
    | containerName  | containerAlias |
    | test-inputs    | input-blobs    |
    | test-outputs   | output-blobs   |

Column	Description
`containerName`	Actual Azure container name
`containerAlias`	Alias used in later steps (e.g. `input-blobs`)

Action, Upload a Blob from File

Upload a local file to Azure Blob Storage before running your action,

When blob from file is uploaded
  | containerAlias | blobName         | file            | tags             |
  | input-blobs    | orders/order.json| order.json      | env=test,type=order |

Column	Description	Optional
`containerAlias`	The registered container alias	No
`blobName`	Target blob path in container	No
`file`	Local file path (relative to assets dir)	No
`tags`	Blob tags as key=value pairs (comma-separated)	Yes

Validation, Assert Blob Content

Validate that a blob exists and matches expected content,

Then expected blob from file
  | containerAlias | blobName              | file              | readTimeout |
  | output-blobs   | orders/order.json     | expected.json     | 30          |

Column	Description	Optional
`containerAlias`	The registered container alias	No
`blobName`	Blob path to validate	No
`file`	Expected file path (relative to assets dir)	No
`readTimeout`	How long to wait for blob to appear (seconds)	Yes, default 30

Negative Assertion, Assert Blob Does Not Exist

And blob should not exist
  | containerAlias | blobName              |
  | output-blobs   | orders/error-file.json|

Complete Scenario Example

Feature: Order processor writes results to Azure Blob

  Background:
    Given namespace
      | namespace       |
      | test-namespace  |
    Given input topic
      | topicName   | topicAlias  |
      | orders-in   | orders      |
    Given output topic
      | topicName   | topicAlias  |
      | orders-out  | results     |
    Given Azure Blob container
      | containerName | containerAlias |
      | inputs        | input-blobs    |
      | outputs       | output-blobs   |
    Given assets directory
      | absolutePath |
      | src/test/resources/data |

  Scenario: Process order and write result blob
    Given blob from file is uploaded
      | containerAlias | blobName       | file       |
      | input-blobs    | order-001.json | order.json |

    When record from file is sent
      | topicName | file              | recordKey |
      | orders-in | order-trigger.json| ORD-001   |

    And wait for 5 seconds

    Then expected record from file
      | topicName | file              |
      | orders-out| order-result.json |

    And expected blob from file
      | containerAlias | blobName          | file              |
      | output-blobs   | order-001-result  | expected-blob.json|

Local Development with Azurite

For local testing and CI/CD, use the Azurite emulator instead of a real Azure account,

Start Azurite

docker run -p 10000:10000 mcr.microsoft.com/azure-storage/azurite azurite-blob --loose

Configure for Local Testing

Add this to your test application.conf,

ktestify.plugins.azure-blob {
  connection-string = "UseDevelopmentStorage=true"
}

Or set the environment variable,

export AZURE_STORAGE_CONNECTION_STRING="UseDevelopmentStorage=true"

The plugin will automatically use the local Azurite instance running on 127.0.0.1:10000.

Implementation Structure

The plugin follows the three-layer architecture defined by ktestify-core,

ktestify-plugin-azureblob/
└── src/main/java/io/github/ktestify/azureblob/
    ├── AzureBlobPlugin.java              ← KtestifyPlugin SPI impl
    ├── config/
    │   └── AzureBlobConfig.java          ← reads ktestify.plugins.azure-blob HOCON
    ├── entities/
    │   └── KtestifyBlobContainer.java    ← container metadata model
    ├── io/
    │   ├── AzureBlobClient.java          ← Azure SDK wrapper (lifecycle)
    │   └── AzureBlobRecordFetcher.java   ← implements RecordFetcher<String>
    ├── services/
    │   ├── BlobUploadService.java        ← upload logic
    │   └── BlobValidationService.java    ← content matching
    └── steps/
        ├── AzureBlobBackgroundSteps.java
        ├── AzureBlobActionSteps.java
        ├── AzureBlobValidationSteps.java
        └── SharedAzureBlobResources.java  ← scenario-scoped state

Key layers :

Transport, AzureBlobRecordFetcher implements RecordFetcher<String> and polls Azure Blob Storage
Orchestration, Service classes wire upload, fetch, and validation
Assertion, Reuses standard RecordMatcher implementations from ktestify-core

Error Handling

Container Does Not Exist

If auto-create-containers = true (default), the plugin automatically creates missing containers. Otherwise,

io.github.ktestify.exceptions.PluginException:
  Container 'test-inputs' does not exist.
  Set auto-create-containers=true or create it manually.

Authentication Failed

io.github.ktestify.exceptions.PluginException:
  Azure Blob authentication failed.
  Verify AZURE_STORAGE_CONNECTION_STRING or credentials in config.

Blob Not Found

If a blob is not found within readTimeout,

AssertionError: Blob not found within 30s: orders/order.json

Check that,

The container name and blob path are correct
The previous step that was supposed to create the blob ran successfully
The timeout value is sufficient for your system's latency

Advanced Topics

Dynamic Blob Names

Use ktestify's dynamic variable syntax in blob names,

Then expected blob from file
  | containerAlias | blobName                           | file         |
  | output-blobs   | {{env:ORDER_ID}}/result-{{date}}.json | expected.json|

See Dynamic Variables for the full syntax.

Accessing Created Blobs in Code

If you need to access blob data from Java test code,

@Inject SharedAzureBlobResources resources;

@Test
public void myTest() {
    // Upload
    resources.uploadBlob("input-blobs", "test.json", new File("payload.json"));

    // Download
    String content = resources.downloadBlobAsString("output-blobs", "result.json");
    assertThat(content).contains("expected-value");
}

Troubleshooting

Issue	Solution
Plugin not discovered	Ensure JAR is in classpath or `/workspace/plugins`, check logs for `ServiceLoader` errors
"Container does not exist"	Set `auto-create-containers = true` or create the container in Azure Portal
Timeout waiting for blob	Increase `readTimeout` parameter or check that the producing system wrote the blob
"Invalid connection string"	Verify the format, use `DefaultEndpointsProtocol=https;...` for real Azure or `UseDevelopmentStorage=true` for Azurite

Contributing

The Azure Blob plugin is actively maintained. Report issues and contribute improvements at,

ktestify-plugin-azureblob on GitHub

When to Use​

Installation​

Maven Dependency​

With ktestify-cucumber (Docker)​

Configuration​

Configuration Keys​

Configuration Priority​

Gherkin Steps​

Background, Register a Blob Container​

Action, Upload a Blob from File​

Validation, Assert Blob Content​

Negative Assertion, Assert Blob Does Not Exist​

Complete Scenario Example​

Local Development with Azurite​

Start Azurite​

Configure for Local Testing​

Implementation Structure​

Error Handling​

Container Does Not Exist​

Authentication Failed​

Blob Not Found​

Advanced Topics​

Dynamic Blob Names​

Accessing Created Blobs in Code​

Troubleshooting​

Contributing​

See Also​