Authentication & Authorization
Prerequisite
Stop all server in previous lab
type ctrl+c in each terminal (stop kafka before stop zookeeper)
check kafka broker and zookeeper process with jps command
jps
clear old data in previous lab
rm -rf /tmp/zookeep* rm -rf /tmp/kaf*
Create Java Authentication and Authorization Service (JAAS) for Kafa Authentication
Configure the broker with its user credentials and authorize the client's user credentials. These credentials along with the login module specification, are stored in a JAAS login configuration file , config JAAS at jaas.conf
This example defines the following for the KafkaServer entity:
The custom login module that is used for user authentication,
admin/admin is the username and password for inter-broker communication (i.e. the credentials the broker uses to connect to other brokers in the cluster),
admin/admin, alice/alice, bob/bob, and charlie/charlie as client user credentials. Note that the valid username and password is provided in this format: user_username="password". If the line user_admin="admin" is removed from this file, the broker is not able to authenticate and authorize an admin user. Only the admin user can to connect to other brokers in this case.
Pass in this file as a JVM configuration option when running the broker, using -Djava.security.auth.login.config=[path_to_jaas_file]. [path_to_jaas_file] can be something like: config/jaas-kafka-server.conf. This can be done by setting the KAFKA_OPTS environment variable, for example:
run below command to add jaas to "KAFKA_OPTS" environment variable
Set Kafka Broker Authentication
Define the accepted protocol and the ACL authorizer used by the broker by adding the following configuration to the broker properties file server.properties
The other configuration that can be added is for Kafka super users: users with full access to all APIs. This configuration reduces the overhead of defining per-API ACLs for the user who is meant to have full API access. From our list of users, let's make admin a super user with the following configuration:
When the broker runs with this security configuration, only authenticated and authorized clients are able to connect to and use it.
review server.properties in folder 5-basic-acl/kafka/bin/config/server.properties
Try Authentication to Kafka
start zookeeper, open new terminal and run below command to start zookeeper
start kafka, open new terminal and run below command to start kafka
try to get list of topic, open new terminal and run below command
example result, So far, the broker is configured for authenticated access. Running a Kafka console producer or consumer not configured for authenticated and authorized access fails with messages like the following
error result in kafka console
Create Client-Site Setting
Specify the broker protocol as well as the credentials to use on the client side. The following configuration is placed inside the corresponding configuration file (admin.properties) provided to the particular client, review following properties in kafka/config folder
Try list topic again with admin user
Verify that it can be called without error.
create topic "test"
List the topics.
Try to put data to topic "test" with user:alice
example result, ctrl+c to exit
error result in kafka console
Test Again with alice user
example result
try to send message such as "message1"
example result, Authorization Exception will be throw to your console.
exit from producer console by type ctrl+c.
Adding ACL rules
Check current access control list
example result, no acl
add authorized to alice user
requirement are:
run command to add authorized
example result
Test Again with alice user
example result, try to send message to topic "test"
exit from kafka console producer and check current acl
example result
Add acl for consume topic
test consume message with user: bob
example result, error is
add read authorized
requirement are:
run command to add authorized
example result
add authorized for committing offsets to group bob-group
example result
test consume message again
example result
exit from kafka console and check current acl
Add authorized for Describe Consumer Group
add authorized for describe consumer group to user: charlie
requirement are
run command to add authorized to read group bob-group
example result
run command to add authorized to describe topic test
example result
test describe consumer group
example result
List current ACL
run command to check current ACL in this kafka cluster
example result
Last updated