Hi,
I am trying to setup a two node Cassandra cluster
Able to start the first node, but not seeing the following exception while
starting the second node
ERROR 17:31:34,315 Exception encountered during startup
java.lang.IllegalStateException: Unable to contact any seeds!
at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
java.lang.IllegalStateException: Unable to contact any seeds!
at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Exception encountered during startup: Unable to contact any seeds!
ERROR 17:31:34,322 Exception in thread
Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at
org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
at
org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
============================My yaml configuration files have these modified
first node yaml
initial_token: -9223372036854775808 # generated this using tokengen tool
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.19.207 # which is the IP of first node itself
rpc_address: 0.0.0.0
second node yaml
initial_token: 0
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.10.223 # which is the IP of second node
rpc_address: 0.0.0.0
================Can anyone please help me what went wrong with my configuration?
Regards
Dinesh
Dinesh 's gravatar image asked Aug 27 2013 at 21:45 in Cassandra-User by Dinesh

9 Answers

You would need to configure rpc_address also with hostname/ips on both the
nodes.
Naresh
Naresh Yadav 's gravatar image answered Aug 27 2013 at 22:10 by Naresh Yadav
In my case rpc_address in both the nodes is set to 0.0.0.0 which means it
listens on all interfaces. it has a larger scope (to listen on all
localhost, ipv4, hostnames, ipv6 addresses) than providing just the
hostname/ipv4 addresses
anyway I initially checked that, but it's the same exception I got in this
case also
Regards
Dinesh
Dinesh 's gravatar image answered Aug 27 2013 at 22:36 by Dinesh
Have you verified that your firewall is configured for the cassandra
traffic. At the very least you need to make certain the following ports are
open between nodes: 7000, 7001, 7199, 9160, 61620 and 61621.
John Pyeatt
Singlewire Software,
John Pyeatt 's gravatar image answered Aug 28 2013 at 14:04 by John Pyeatt
Hi John,
I had my firewall disabled in both the nodes
To make sure. I checked it
# rcSuSEfirewall2 status
Checking the status of SuSEfirewall2
unused
if it's, on it says running
Please suggest the further steps, where to look and troubleshoot, if you
have any idea
Regards
Dinesh
Dinesh 's gravatar image answered Aug 28 2013 at 22:41 by Dinesh
To be sure ports are open try to connect from one node to another:
telnet <node ip> 7000
try all ports.
Andrey
Andrey Ilinykh 's gravatar image answered Aug 28 2013 at 22:53 by Andrey Ilinykh
My first node is running and second node is not running in this case
I tried telnet from second node to first node. Following is the stdout
# telnet 10.96.10.207 7000
Trying 10.96.10.207...
Connected to 10.96.10.207.
Escape character is '^]'.
quit
Connection closed by foreign host.
I see the same output for 7199 & 9160 ports. is this correct. I am not
seeing the telnet session as something like telnet>
But when I tried with 7001. I am seeing the below output
# telnet 10.96.10.207 7001
Trying 10.96.10.207...
telnet: connect to address 10.96.10.207: Connection refused
I see the same output for 61620 & 61621 ports
Please suggest
Regards
Dinesh
Dinesh 's gravatar image answered Aug 28 2013 at 23:47 by Dinesh
Have you tried setting rpc_address to the same as listen_address ?
Andy
My first node is running and second node is not running in this case
I tried telnet from second node to first node. Following is the stdout
# telnet 10.96.10.207 7000
Trying 10.96.10.207...
Connected to 10.96.10.207.
Escape character is '^]'.
quit
Connection closed by foreign host.
I see the same output for 7199 & 9160 ports. is this correct. I am not seeing the telnet session as something like telnet>
But when I tried with 7001. I am seeing the below output
# telnet 10.96.10.207 7001
Trying 10.96.10.207...
telnet: connect to address 10.96.10.207<http://10.96.10.207/>: Connection refused
I see the same output for 61620 & 61621 ports
Please suggest
To be sure ports are open try to connect from one node to another:
telnet <node ip> 7000
try all ports.
Andrey
Hi John,
I had my firewall disabled in both the nodes
To make sure. I checked it
# rcSuSEfirewall2 status
Checking the status of SuSEfirewall2 unused
if it's, on it says running
Please suggest the further steps, where to look and troubleshoot, if you have any idea
Have you verified that your firewall is configured for the cassandra traffic. At the very least you need to make certain the following ports are open between nodes: 7000, 7001, 7199, 9160, 61620 and 61621.
In my case rpc_address in both the nodes is set to 0.0.0.0 which means it listens on all interfaces. it has a larger scope (to listen on all localhost, ipv4, hostnames, ipv6 addresses) than providing just the hostname/ipv4 addresses
anyway I initially checked that, but it's the same exception I got in this case also
You would need to configure rpc_address also with hostname/ips on both the nodes.
Naresh
Hi,
I am trying to setup a two node Cassandra cluster
Able to start the first node, but not seeing the following exception while starting the second node
ERROR 17:31:34,315 Exception encountered during startup
java.lang.IllegalStateException: Unable to contact any seeds!
at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
java.lang.IllegalStateException: Unable to contact any seeds!
at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Exception encountered during startup: Unable to contact any seeds!
ERROR 17:31:34,322 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
=============================
My yaml configuration files have these modified
first node yaml
initial_token: -9223372036854775808 # generated this using tokengen tool
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.19.207 # which is the IP of first node itself
rpc_address: 0.0.0.0
second node yaml
initial_token: 0
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.10.223 # which is the IP of second node
rpc_address: 0.0.0.0
==================
Can anyone please help me what went wrong with my configuration?
Regards
Dinesh
Regards
Dinesh
John Pyeatt
Singlewire Software, University of Dundee is a registered Scottish Charity, No: SC015096
Andrew Cobley 's gravatar image answered Aug 29 2013 at 00:10 by Andrew Cobley
Facing the same issue with rpc_address set to the same as listen address on
both nodes
Checked with having
in first node - rpc_address: 10.96.10.207
in second node - rpc_address: 10.96.10.223
Regards
Dinesh
Dinesh 's gravatar image answered Aug 29 2013 at 01:18 by Dinesh
I enabled the debug level for logging. following is the debug entry for the
same exception from node2. Someone please help me
DEBUG 14:53:58,372 Connection version 6 from /10.96.10.105
DEBUG 14:53:58,374 Upgrading incoming connection to be compressed
DEBUG 14:53:58,378 Max version for /10.96.10.105 is 6
DEBUG 14:53:58,379 Setting version 6 for /10.96.10.105
DEBUG 14:53:58,380 set version for /10.96.10.105 to 6
DEBUG 14:53:58,380 Reseting version for /10.96.10.105
DEBUG 14:53:58,683 error writing to rhlcas1/10.96.10.105
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:40)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:336)
at java.nio.channels.Channels.writeFullyImpl(Channels.java:59)
at java.nio.channels.Channels.writeFully(Channels.java:81)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:155)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at
org.xerial.snappy.SnappyOutputStream.flush(SnappyOutputStream.java:95)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:205)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:149)
Regards
Dinesh
Dinesh 's gravatar image answered Aug 29 2013 at 21:08 by Dinesh