10 essential Apache Kafka program code examples in Java to help you get started with Kafka

Itexamtools.com
2 min readOct 21, 2023

1. Creating a Kafka Producer:

import org.apache.kafka.clients.producer.*;

public class KafkaProducerExample {
public static void main(String[] args) {
Properties properties = new Properties();
properties.put("bootstrap.servers", "localhost:9092");
properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Producer<String, String> producer = new KafkaProducer<>(properties);

ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key", "Hello, Kafka!");
producer.send(record);
producer.close();
}
}

2. Creating a Kafka Consumer:

import org.apache.kafka.clients.consumer.*;

public class KafkaConsumerExample {
public static void main(String[] args) {
Properties properties = new Properties();
properties.put("bootstrap.servers", "localhost:9092");
properties.put("group.id", "my-group");
properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties);

consumer.subscribe(Collections.singletonList("my-topic"));

while (true) {
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
for (ConsumerRecord<String, String> record : records) {
System.out.println("Received message: " + record.value());
}
}
}
}

3. Kafka Topic Creation:

bin/kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

4. Kafka Topic Deletion:

bin/kafka-topics.sh --delete --topic my-topic --bootstrap-server localhost:9092

5. Producer with Callback:

producer.send(record, (metadata, exception) -> {
if (exception == null) {
System.out.println("Message sent to partition " + metadata.partition());
} else {
System.err.println("Error sending message: " + exception.getMessage());
}
});

6. Custom Partitioning:

properties.put("partitioner.class", "com.example.CustomPartitioner");

7. Kafka Streams Example:

import org.apache.kafka.streams.*;
import org.apache.kafka.streams.kstream.KStream;

public class KafkaStreamsExample {
public static void main(String[] args) {
Properties config = new Properties();
config.put(StreamsConfig.APPLICATION_ID_CONFIG, "my-streams-app");
config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
StreamsBuilder builder = new StreamsBuilder();

KStream<String, String> source = builder.stream("input-topic");
source.to("output-topic");

KafkaStreams streams = new KafkaStreams(builder.build(), config);
streams.start();

Runtime.getRuntime().addShutdownHook(new Thread(streams::close));
}
}

8. Producer with Avro Serialization:

properties.put("key.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");
properties.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");

9. Consumer with Avro Deserialization:

properties.put("key.deserializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");
properties.put("value.deserializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");

10. Kafka Admin Client Example (Creating Topics Programmatically):

import org.apache.kafka.clients.admin.*;

public class KafkaAdminClientExample {
public static void main(String[] args) throws Exception {
Properties properties = new Properties();
properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
AdminClient adminClient = AdminClient.create(properties);

NewTopic newTopic = new NewTopic("my-new-topic", 1, (short) 1);
CreateTopicsResult result = adminClient.createTopics(Collections.singletonList(newTopic));
result.all().get();

adminClient.close();
}
}

These code examples cover the basics of Kafka producers, consumers, topic management, custom partitioning, Kafka Streams, Avro serialization, and Kafka Admin Client usage. You can further expand on these examples to suit your specific use case.

Hope this helps.

Learn more
— — — — — — — — — —
https://itexamtools.com
https://itexamsusa.blogspot.com/
https://askcarlito.blogspot.com/

--

--

Itexamtools.com

At ITExamtools.com we help IT students and Professionals by providing important info. about latest IT Trends & for selecting various Academic Training courses.