Skip to content

Redpanda

Testcontainers can be used to automatically instantiate and manage Redpanda containers. More precisely Testcontainers uses the official Docker images for Redpanda

Note

This module uses features provided in docker.redpanda.com/redpandadata/redpanda.

Example

Create a Redpanda to use it in your tests:

RedpandaContainer container = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.2")

Now your tests or any other process running on your machine can get access to running Redpanda broker by using the following bootstrap server location:

container.getBootstrapServers()

Redpanda also provides a schema registry implementation. Like the Redpanda broker, you can access by using the following schema registry location:

container.getSchemaRegistryAddress()

It is also possible to enable security capabilities of Redpanda by using:

RedpandaContainer redpanda = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .enableAuthorization()
    .enableSasl()
    .withSuperuser("superuser-1")

Superusers can be created by using:

String adminUrl = String.format("%s/v1/security/users", redpanda.getAdminAddress());
RestAssured
    .given()
    .contentType("application/json")
    .body("{\"username\": \"superuser-1\", \"password\": \"test\", \"algorithm\": \"SCRAM-SHA-256\"}")
    .post(adminUrl)
    .then()
    .statusCode(200);

Below is an example of how to create the AdminClient:

AdminClient adminClient = AdminClient.create(
    ImmutableMap.of(
        AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG,
        bootstrapServer,
        AdminClientConfig.SECURITY_PROTOCOL_CONFIG,
        "SASL_PLAINTEXT",
        SaslConfigs.SASL_MECHANISM,
        "SCRAM-SHA-256",
        SaslConfigs.SASL_JAAS_CONFIG,
        "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"superuser-1\" password=\"test\";"
    )
);

There are scenarios where additional listeners are needed because the consumer/producer can be another container in the same network or a different process where the port to connect differs from the default exposed port 9092. E.g Toxiproxy.

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092")
    .withNetwork(network);

    

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092", () -> socat.getHost() + ":" + socat.getMappedPort(2000))
    .withNetwork(network)

Container defined in the same network:

GenericContainer<?> kcat = new GenericContainer<>("confluentinc/cp-kcat:7.4.1")
    .withCreateContainerCmdModifier(cmd -> {
        cmd.withEntrypoint("sh");
    })
    .withCopyToContainer(Transferable.of("Message produced by kcat"), "/data/msgs.txt")
    .withNetwork(network)
    .withCommand("-c", "tail -f /dev/null")

Client using the new registered listener:

kcat.execInContainer("kcat", "-b", "kafka:19092", "-t", "msgs", "-P", "-l", "/data/msgs.txt");
String stdout = kcat
    .execInContainer("kcat", "-b", "kafka:19092", "-C", "-t", "msgs", "-c", "1")
    .getStdout();

    

String bootstrapServers = String.format("%s:%s", socat.getHost(), socat.getMappedPort(2000));
testKafkaFunctionality(bootstrapServers);

The following examples shows how to register a proxy as a new listener in RedpandaContainer:

Use SocatContainer to create the proxy

SocatContainer socat = new SocatContainer().withNetwork(network).withTarget(2000, "kafka", 19092);

Register the listener and advertised listener

RedpandaContainer kafka = new RedpandaContainer("docker.redpanda.com/redpandadata/redpanda:v23.1.7")
    .withListener("kafka:19092", () -> socat.getHost() + ":" + socat.getMappedPort(2000))
    .withNetwork(network)

Client using the new registered listener:

String bootstrapServers = String.format("%s:%s", socat.getHost(), socat.getMappedPort(2000));
testKafkaFunctionality(bootstrapServers);

Adding this module to your project dependencies

Add the following dependency to your pom.xml/build.gradle file:

testImplementation "org.testcontainers:redpanda:1.20.2"
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>redpanda</artifactId>
    <version>1.20.2</version>
    <scope>test</scope>
</dependency>