Redpanda

Testcontainers can be used to automatically instantiate and manage Redpanda containers. More precisely Testcontainers uses the official Docker images for Redpanda

Note

This module uses features provided in docker.redpanda.com/redpandadata/redpanda.

Example

Create a Redpanda to use it in your tests:

Creating a Redpanda

RedpandaContainer container = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.2")

Now your tests or any other process running on your machine can get access to running Redpanda broker by using the following bootstrap server location:

Bootstrap Servers

container.getBootstrapServers()

Redpanda also provides a schema registry implementation. Like the Redpanda broker, you can access by using the following schema registry location:

Schema Registry

container.getSchemaRegistryAddress()

It is also possible to enable security capabilities of Redpanda by using:

Enable security

RedpandaContainer redpanda = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .enableAuthorization()
    .enableSasl()
    .withSuperuser("superuser-1")

Superusers can be created by using:

Register Superuser

String adminUrl = String.format("%s/v1/security/users", redpanda.getAdminAddress());
RestAssured
    .given()
    .contentType("application/json")
    .body("{\"username\": \"superuser-1\", \"password\": \"test\", \"algorithm\": \"SCRAM-SHA-256\"}")
    .post(adminUrl)
    .then()
    .statusCode(200);

Below is an example of how to create the AdminClient:

Create Admin Client

AdminClient adminClient = AdminClient.create(
    ImmutableMap.of(
        AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG,
        bootstrapServer,
        AdminClientConfig.SECURITY_PROTOCOL_CONFIG,
        "SASL_PLAINTEXT",
        SaslConfigs.SASL_MECHANISM,
        "SCRAM-SHA-256",
        SaslConfigs.SASL_JAAS_CONFIG,
        "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"superuser-1\" password=\"test\";"
    )
);

There are scenarios where additional listeners are needed because the consumer/producer can be another container in the same network or a different process where the port to connect differs from the default exposed port 9092. E.g Toxiproxy.

Register additional listener

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092")
    .withNetwork(network);

    ⋯

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092", () -> socat.getHost() + ":" + socat.getMappedPort(2000))
    .withNetwork(network)

Container defined in the same network:

Create kcat container

GenericContainer<?> kcat = new GenericContainer<>("confluentinc/cp-kcat:7.4.1")
    .withCreateContainerCmdModifier(cmd -> {
        cmd.withEntrypoint("sh");
    })
    .withCopyToContainer(Transferable.of("Message produced by kcat"), "/data/msgs.txt")
    .withNetwork(network)
    .withCommand("-c", "tail -f /dev/null")

Client using the new registered listener:

Produce/Consume via new listener

kcat.execInContainer("kcat", "-b", "kafka:19092", "-t", "msgs", "-P", "-l", "/data/msgs.txt");
String stdout = kcat
    .execInContainer("kcat", "-b", "kafka:19092", "-C", "-t", "msgs", "-c", "1")
    .getStdout();

    ⋯

String bootstrapServers = String.format("%s:%s", socat.getHost(), socat.getMappedPort(2000));
testKafkaFunctionality(bootstrapServers);

The following examples shows how to register a proxy as a new listener in RedpandaContainer:

Use SocatContainer to create the proxy

Create Proxy

SocatContainer socat = new SocatContainer().withNetwork(network).withTarget(2000, "kafka", 19092);

Register the listener and advertised listener

Register Listener

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092", () -> socat.getHost() + ":" + socat.getMappedPort(2000))
    .withNetwork(network)

Client using the new registered listener:

Produce/Consume via new listener

String bootstrapServers = String.format("%s:%s", socat.getHost(), socat.getMappedPort(2000));
testKafkaFunctionality(bootstrapServers);

Adding this module to your project dependencies

Add the following dependency to your pom.xml/build.gradle file:

Gradle

testImplementation "org.testcontainers:redpanda:1.20.4"

Maven

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>redpanda</artifactId>
    <version>1.20.4</version>
    <scope>test</scope>
</dependency>