Spread the love

In this post we will see how to enable Security between Kafka and zookeeper using SASL. SASL is nothing but Simple Authentication and Security Layer.

Following are the configuration files:

server.properties

security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
auto.create.topics.enable=false
broker.id=0

listeners=SASL_PLAINTEXT://localhost:9092
advertised.listeners=SASL_PLAINTEXT://localhost:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

advertised.host.name=localhost
num.partitions=1
num.recovery.threads.per.data.dir=1
log.flush.interval.messages=30000000
log.flush.interval.ms=1800000
log.retention.minutes=30
log.segment.bytes=1073741824

log.retention.check.interval.ms=300000
delete.topic.enable=true
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
super.users=User:admin
zookeeper.properties

dataDir=/tmp/zookeeper
clientPort=2181
maxClientCnxns=0
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
producer.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
bootstrap.servers=localhost:9092
compression.type=none
consumer.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.id=test-consumer-group
Now are the most important files for making your server starting without any issue:

zookeeper_jaas.conf

Server {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=”admin”
password=”admin-secret”
user_admin=”admin-secret”;
};

kafka_server.jaas

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=”admin”
password=”admin-secret”
user_admin=”admin-secret”;
};

Client {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=”admin”
password=”admin-secret”;
};

kafka_client_jaas.conf

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=”admin”
password=”admin-secret”;
};

After doing all these configuration, Follow below steps

Terminal 1: From kafka root directory

$ export KAFKA_OPTS=”-Djava.security.auth.login.config=/home/usename/Documents/kafka_2.11-0.10.1.0/config/zookeeper_jaas.conf”
$ bin/zookeeper-server-start.sh config/zookeeper.properties

Terminal 2: From kafka root directory

$ export KAFKA_OPTS=”-Djava.security.auth.login.config=/home/usename/Documents/kafka_2.11-0.10.1.0/config/kafka_server_jaas.conf”
$ bin/kafka-server-start.sh config/server.properties

Terminal 3

On a client terminal, export client jaas conf file and start consumer:

$ export KAFKA_OPTS=”-Djava.security.auth.login.config=/home/username/Documents/kafka_2.11-0.10.1.0/kafka_client_jaas.conf”
$ ./bin/kafka-console-consumer.sh –new-consumer –zookeeper localhost:2181 –topic test-topic –from-beginning –consumer.config=config/consumer.properties –bootstrap-server=localhost:9092

Terminal 4

If you also want to produce, do this on another terminal window:

$ export KAFKA_OPTS=”-Djava.security.auth.login.config=/home/username/Documents/kafka_2.11-0.10.1.0/kafka_client_jaas.conf”
$ ./bin/kafka-console-producer.sh –broker-list localhost:9092 –topic test-topic –producer.config=config/producer.properties

Thank you.

Related Posts

blog

Apache Kafka Commands Cheat sheet

Spread the loveKafka Topics List existing topics bin/kafka-topics.sh –zookeeper localhost:2181 –list Purge a topic bin/kafka-topics.sh –zookeeper localhost:2181 –alter –topic mytopic –config retention.ms=1000 … wait a minute … bin/kafka-topics.sh –zookeeper localhost:2181 –alter –topic mytopic –delete-config retention.ms
Read more…

blog

What is Apache Maven | Apache Maven complete tutorial from scratch pdf

Spread the love In this post you will learn the complete tutorial of Apache Maven build tool What is Maven ? Apache Maven is a software project management and comprehension tool. Based on the concept
Read more…

blog

Practical Guide for Web Development in 2018

Spread the loveWelcome to my practical guide  for web development in 2018 in terms of  technology and career. Before we start I just want to  mention a few things, you don’t need to learn  everything that
Read more…