Migrating from Kafka to Azure Event Hubs

Event-driven architectures rely heavily on robust messaging backbones. Apache Kafka has long been a popular choice, but with the rise of cloud-native ecosystems, Azure Event Hubs has become a powerful alternative.

In this blog, we’ll cover:

• How Kafka and Event Hubs differ in architecture.
• Two migration approaches:
  1. Using Azure SDK directly.
  2. Using Kafka SDK with minimal config changes.
• How Event Hubs supports AMQP and WebSocket transports.
• A comparison of producer and consumer models.

Kafka vs Event Hubs Architecture

• Kafka stores messages in distributed logs (topics → partitions).
• Event Hubs also uses partitioned consumers, but runs as a PaaS service managed by Azure.

Both support publish-subscribe and stream processing, but Event Hubs removes the overhead of cluster management.

Migration Approaches

1. Using Azure SDK

Azure Event Hubs provides an official Java SDK that makes producing and consuming messages straightforward.

Producer Example (Java):

EventHubProducerClient producer = new EventHubClientBuilder()
    .connectionString("<EVENT_HUB_CONNECTION_STRING>", "<EVENT_HUB_NAME>")
    .buildProducerClient();
 
EventDataBatch batch = producer.createBatch();
batch.tryAdd(new EventData("Hello Event Hubs!"));
producer.send(batch);
producer.close();

Consumer Example (Java):

EventHubConsumerAsyncClient consumer = new EventHubClientBuilder()
    .connectionString("<EVENT_HUB_CONNECTION_STRING>", "<EVENT_HUB_NAME>")
    .consumerGroup(EventHubClientBuilder.DEFAULT_CONSUMER_GROUP_NAME)
    .buildAsyncConsumerClient();
 
consumer.receive(false).subscribe(event -> {
    System.out.println("Received event: " + event.getData().getBodyAsString());
});

This approach is cleaner but requires code refactoring.

2. Using Kafka SDK (Minimal Code Changes)

Event Hubs provides a Kafka-compatible endpoint. This means existing Kafka producers/consumers can work with Event Hubs by just changing configurations.

# Kafka Config → Event Hub Config
bootstrap.servers=<EVENTHUB_NAMESPACE>.servicebus.windows.net:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required   username="$ConnectionString"   password="<EVENT_HUB_CONNECTION_STRING>";

This approach is faster since it avoids large code changes.

Transport Protocols: AMQP and WebSockets

Event Hubs supports:

• AMQP (Advanced Message Queuing Protocol) → Default, efficient for most scenarios.
• AMQP over WebSockets → Works better behind firewalls/proxies (e.g., corporate environments).

Producers and Consumers

Key Considerations

• If you want quick migration, use the Kafka SDK approach.
• If you want long-term native integration, use the Azure SDK.
• For firewall-constrained environments, prefer AMQP over WebSockets.

Conclusion

Migrating from Kafka to Event Hubs can be as simple as changing configs or as robust as rewriting clients with Azure SDK. The choice depends on your timeline, maintainability goals, and cloud adoption strategy.