Hi
I am trying to setup a simple setup with one Kafka broker, and zookeeper on
the same VM. One producer and one consumer on each VM. I have setup a KDC
I am trying to following this guide:
http://docs.confluent.io/2.0.0/kafka/sasl.html#kerberos
When I start Kafka, it errors out with the following error. Do I need to
setup anything on zookeeper side as well to fix these errors ?
Thanks.
*[2017-02-16 19:05:00,583] WARN Could not login: the client is being asked
for a password, but the Zookeeper client code does not currently support
obtaining a password from the user. Make sure that the client is configured
to use a ticket cache (using the JAAS configuration setting
'useTicketCache=true)' and restart the client. If you still get this
message after that, the TGT in the ticket cache has expired and must be
manually refreshed. To do so, first determine if you are using a password
or a keytab. If the former, run kinit in a Unix shell in the environment of
the user who is running this Zookeeper client using the command 'kinit
<princ>' (where <princ> is the name of the client's Kerberos principal). If
the latter, do 'kinit -k -t <keytab> <princ>' (where <princ> is the name of
the Kerberos principal, and <keytab> is the location of the keytab file).
After manually refreshing your cache, restart this client. If you continue
to see this message after manually refreshing your cache, ensure that your
KDC host's clock is in sync with this host's clock.
(org.apache.zookeeper.client.ZooKeeperSaslClient)*
*[2017-02-16 19:05:00,584] WARN SASL configuration failed:
javax.security.auth.login.LoginException: No password provided Will
continue connection to Zookeeper server without SASL authentication, if
Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,585] INFO Opening socket connection to server
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,585] INFO zookeeper state changed (AuthFailed)
(org.I0Itec.zkclient.ZkClient)*
*[2017-02-16 19:05:00,586] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)*
*[2017-02-16 19:05:00,591] INFO Socket connection established to
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>, initiating session
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,597] INFO Session establishment complete on server
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>, sessionid 0x15a4a0678610002, negotiated timeout = 6000
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,599] INFO Session: 0x15a4a0678610002 closed
(org.apache.zookeeper.ZooKeeper)*
*[2017-02-16 19:05:00,599] FATAL Fatal error during KafkaServer startup.
Prepare to shutdown (kafka.server.KafkaServer)*
*org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication
failure*
* at
org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946)*
* at
org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923)*
* at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230)*
* at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)*
* at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)*
* at
kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:76)*
* at kafka.utils.ZkUtils$.apply(ZkUtils.scala:58)*
* at kafka.server.KafkaServer.initZk(KafkaServer.scala:327)*
* at kafka.server.KafkaServer.startup(KafkaServer.scala:200)*
* at
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)*
* at kafka.Kafka$.main(Kafka.scala:67)*
* at kafka.Kafka.main(Kafka.scala)*
Raghav
Raghav 's gravatar image asked Feb 17 2017 at 08:56 in Incubator-Kafka-Users by Raghav

1 Answers

Hi
It seems that your keytab doesn't have the principal you configured your
"client" section to use. Post your jaas here if you want further help but
basically you should be able to do
kinit -V -k -t <your.keytab> <[email protected]>
Hi
I am trying to setup a simple setup with one Kafka broker, and zookeeper on
the same VM. One producer and one consumer on each VM. I have setup a KDC
I am trying to following this guide:
http://docs.confluent.io/2.0.0/kafka/sasl.html#kerberos
When I start Kafka, it errors out with the following error. Do I need to
setup anything on zookeeper side as well to fix these errors ?
Thanks.
*[2017-02-16 19:05:00,583] WARN Could not login: the client is being asked
for a password, but the Zookeeper client code does not currently support
obtaining a password from the user. Make sure that the client is configured
to use a ticket cache (using the JAAS configuration setting
'useTicketCache=true)' and restart the client. If you still get this
message after that, the TGT in the ticket cache has expired and must be
manually refreshed. To do so, first determine if you are using a password
or a keytab. If the former, run kinit in a Unix shell in the environment of
the user who is running this Zookeeper client using the command 'kinit
<princ>' (where <princ> is the name of the client's Kerberos principal). If
the latter, do 'kinit -k -t <keytab> <princ>' (where <princ> is the name of
the Kerberos principal, and <keytab> is the location of the keytab file).
After manually refreshing your cache, restart this client. If you continue
to see this message after manually refreshing your cache, ensure that your
KDC host's clock is in sync with this host's clock.
(org.apache.zookeeper.client.ZooKeeperSaslClient)*
*[2017-02-16 19:05:00,584] WARN SASL configuration failed:
javax.security.auth.login.LoginException: No password provided Will
continue connection to Zookeeper server without SASL authentication, if
Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,585] INFO Opening socket connection to server
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,585] INFO zookeeper state changed (AuthFailed)
(org.I0Itec.zkclient.ZkClient)*
*[2017-02-16 19:05:00,586] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)*
*[2017-02-16 19:05:00,591] INFO Socket connection established to
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>, initiating session
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,597] INFO Session establishment complete on server
kafka2.example.com/172.26.230.67:2181
<http://kafka2.example.com/172.26.230.67:2181>, sessionid 0x15a4a0678610002, negotiated timeout = 6000
(org.apache.zookeeper.ClientCnxn)*
*[2017-02-16 19:05:00,599] INFO Session: 0x15a4a0678610002 closed
(org.apache.zookeeper.ZooKeeper)*
*[2017-02-16 19:05:00,599] FATAL Fatal error during KafkaServer startup.
Prepare to shutdown (kafka.server.KafkaServer)*
*org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication
failure*
* at
org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946)*
* at
org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923)*
* at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230)*
* at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)*
* at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)*
* at
kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:76)*
* at kafka.utils.ZkUtils$.apply(ZkUtils.scala:58)*
* at kafka.server.KafkaServer.initZk(KafkaServer.scala:327)*
* at kafka.server.KafkaServer.startup(KafkaServer.scala:200)*
* at
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)*
* at kafka.Kafka$.main(Kafka.scala:67)*
* at kafka.Kafka.main(Kafka.scala)*
Raghav
Stephane Maarek 's gravatar image answered Feb 17 2017 at 13:45 by Stephane Maarek

Related Discussions

  • Kafka Authentication With Zookeeper Question in Incubator-kafka-users

  • Hi, Is there any support in Kafka for authentication with Zookeeper regarding the publication of brokers endpoints etc. to ZK? i.e. If we wanted to secure writes/reads of the /brokers node using e.g. SASL authentication (which Zookeeper supports out-of-the-box), is there support in Kafka for this? or is it possible to plugin support for it? thanks David Black Workday...

  • Regarding Kafka Server Integration With Kerberos in Incubator-kafka-users

  • Hi, I am using Kafka Server kafka_2.10-0.10.1.0. I am trying to integrate it with Kerberos server. I followed steps mentioned at http://kafka.apache.org/documentation.html#security_sasl However I am not able to start the server and getting the following error. [2016-11-16 09:47:46,578] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=org.I0Itec...

  • Dual Commit With Zookeeper And Kafka in Incubator-kafka-users

  • Hello! Can somebody explain me how to use multiple consumers with different commit storage... For example, java-based consumers use kafka commit storage... python-based consumers use zookeeper commit storage My question is: Is it true that when one consumer commit to kafka, server also commit automatically to zookeeper (so all consumers see the same commit) Is it true that when one consumer...

  • #Kafka #Zookeeper #Kerberos in Incubator-kafka-users

  • Hello Kafka, I am Mohammed Amine GARMES from AMADEUS SAS. First of all I want to thank you for your solution is very helpful. Indeed, this is my first message to Kafka team, so I have an issue with Kafka security : I have 2 kafka servers configured to start with kafka security, I try to start the akfka servers with the JASS below ==> server 1 KafkaServer { com.sun.security...

  • Kakfa Question About Starting Kafka With Same Broker Id in Incubator-kafka-users

  • Hi, My name is Deepak and I work for salesforce. We are using kafka 8.11 and have a question about starting kafka with same broker id. Steps: Start a kakfa broker with broker id =1 -> it starts fine with external ZK Start another kafka with same broker id =1 .. it does not start the kafka which is expected but I am seeing the following log and it keeps retrying forever. Is there way...

  • Understanding Zookeper And Kafka Server Failures in Incubator-kafka-users

  • Hi, We have following setup. 1. Three instances of zookeeper on three machines. 2. Three instances of kafka server on same three machines. 3. All the topics have replication factor 3. So when we create a topic on any node, i see that it gets replicated on all three instances. I also see that topic data is getting replicated to all three nodes. The data to main topic is written by three...

  • Doubts Kafka in Incubator-kafka-users

  • Hi Guys, I have some doubts about the Kafka, the first is Why sometimes the applications prefer to connect to zookeeper instead brokers? Connecting to zookeeper could create an overhead, because we are inserting other element between producer and consumer. Another question is about the information sent by producer, in my tests the producer send the messages to brokers and a few minutes my HardDisk...

  • Produce/Consumer Example With In-Memory Kafka And Zookeeper Test Fixture Servers - I Would Welcome Help With The Maven Dependencies. in Incubator-kafka-users

  • Hi, Kafka users - I'm just starting out with the framework and I've created a Maven-buildable example that should work out of the box. The example is available here > https://github.com/buildlackey/cep and it illustrates how to: * unit test message passing between Kafka producers and consumers using basic String serialization. * use of Netflix's curator API to instantiate an in-process...

  • What's The Relationship Between Zookeeper And Kafka ? in Incubator-kafka-users

  • I notice that some Spark programs would contact something like 'zoo1:2181' when trying to suck data out of Kafka. Does the kafka data actually get routed out of zookeeper before delivering the payload onto Spark ? -eric ho...

  • Custom Zookeeper Install With Kafka in Incubator-kafka-users

  • Hi, I've read on the Kafka documentation page that the zookeeper version used is 3.3.4 However, at my work, I've noticed certain problems with v3.3.4 (and the problems are documented here: http://zookeeper.apache.org/releases.html The latest stable version of zookeeper is 3.4.6 and has a lot of bug fixes from 3.3.4 Has anyone tried using this version of zookeeper with kafka 0.8.2.1 ? ...

  • Zookeeper Use Cases With Kafka in Incubator-kafka-users

  • Hello Folks, I wish to contribute to Kafka internals. And, one of the things which can help me do that is understanding how kafka uses zookeeper. I have some of these basic doubts:- 1. Is zookeeper primarily used for locking ? If yes, in what cases and what kind of nodes does it use - sequential/ephemeral? 2. Does kafka use zookeeper watches for any of functions ? 3. What kind of state...

  • Setting Up Kafka With Zookeeper in Incubator-kafka-users

  • Hi, In DEV, Kafka and Zookeeper services can be co- located on the same physical hosts In Prod moving forward do we need to set up Zookeeper on its own cluster not sharing with Hadoop cluster? Can these services be shared within the Hadoop cluster? Additionally if there many topics for Kafka what would be an optimum setup specifically with separation of hard disks How best to set up...

  • Kafka Mirroring And Zookeeper in Incubator-kafka-users

  • Hi , I was reading the mirroring guide and I wonder if it is required that the mirror runs it's own zookeeper? We have a zookeeper cluster running which is used by different applications, so can we use that zookeeper cluster for the kafka source and kafka mirror? What is the procedure if the kafka source server fails to switch the applications to use the mirrored instance? Are there any backup...

  • Failover For Zookeeper And Kafka in Incubator-kafka-users

  • Hi everyone, I am trying to setup a Kafka cluster and have a couple of questions about failover. Has anyone deployed more than one zookeeper for a single Kafka cluster and have high availability so if one zookeeper node goes down, the cluster automatically fails over to a backup zookeeper node? If so, how is this done? My second question is how can I set up for automatic failover if I have...

  • Keeping Zookeeper And Kafka Server Up in Incubator-kafka-users

  • Hello Everyone, I'm wondering how to keep Zookeeper and Kafka Server up even when my SSH (using putty) becomes inactive. I've tried running it in the background (using &), but it seems like it stops sometimes after a couple hours or so and I'll have to restart zookeeper and/or the kafka server. The only remediation i've found is to export TMOUT=[big number], but there must be another solution...

  • Kafka Simpleconsumer Question in Incubator-kafka-users

  • Hi , Just want to ask some basic question about kafka simple consumer. 1. if I'm using simple consumer and doesn't really depend on zookeeper to manage partition offset. (application manage offset themselves). Will that remove the zookeeper dependency for consumer ? 2. if zookeeper dies, will simple consumer still able to get partition leadership information from broker itself ? Or it indirectly...

  • Kafka Consumer Behaviour in Incubator-kafka-users

  • I was writing Kafka consumer and I have a query related to consumer processes. I have a consumer with groupId="testGroupId" and using the same groupId I consume from multiple topics say, "topic1" and "topic2". Also, assume "topic1" is already created on broker whereas "topic2" is not yet created. Now If I start the consumer I see consumer threads for "topic1" (which is already created...

  • Is There A Way To Pull Out Kafka Metadata From Zookeeper? in Incubator-kafka-users

  • Hi guys, I'm trying to maintain a bunch of simple kafka consumer to consume messages from brokers. I know there is a way to send TopicMetadataRequest to broker and get the response from the broker. But you have to specify the broker list to query the information. But broker might not be available because of some failure. My question is is there any api I can call and query broker metadata ...

  • Can't Produce To Kafka With A Remote Standalone Zookeeper in Incubator-kafka-users

  • Guys, it can work if i use kafka server (0.8) and zookeeper (3.4.5) in same machine (centos 6.3 final), but failed by seperating them into two machines. 1, when trying to produce, find some error messages, 2,, try to consume, failed because " WARN [console-consumer-65386_server3-1390013879485-3f815f14], no brokers found when trying to rebalance. (kafka.consumer.ZookeeperConsumerConnector)" But the...

  • Kafka Connection With Zookeeper in Incubator-kafka-users

  • can we find from some api in kafka that how many number of connections we have kafka broker to zookeeper, as my kafka is getting down again and again ....