QnaList > Groups > Cassandra-User > Aug 2013
faq

Setting Up A Multi-node Cluster

Hi,
I am trying to setup a two node Cassandra cluster
Able to start the first node, but not seeing the following exception while
starting the second node
ERROR 17:31:34,315 Exception encountered during startup
java.lang.IllegalStateException: Unable to contact any seeds!
        at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
        at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
        at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
        at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
        at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
        at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
        at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
java.lang.IllegalStateException: Unable to contact any seeds!
        at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
        at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
        at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
        at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
        at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
        at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
        at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Exception encountered during startup: Unable to contact any seeds!
ERROR 17:31:34,322 Exception in thread
Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
        at
org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
        at
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
        at
org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
============================My yaml configuration files have these modified
first node yaml
initial_token: -9223372036854775808 # generated this using tokengen tool
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.19.207 # which is the IP of first node itself
rpc_address: 0.0.0.0
second node yaml
initial_token: 0
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.10.223 # which is the IP of second node
rpc_address: 0.0.0.0
================Can anyone please help me what went wrong with my configuration?
Regards
Dinesh

asked Aug 27 2013 at 21:45

Dinesh 's gravatar image



9 Replies for : Setting Up A Multi-node Cluster
You would need to configure rpc_address also with hostname/ips on both the
nodes.
Naresh

answered Aug 27 2013 at 22:10

Naresh Yadav 's gravatar image


In my case rpc_address in both the nodes is set to 0.0.0.0 which means it
listens on all interfaces. it has a larger scope (to listen on all
localhost, ipv4, hostnames, ipv6 addresses) than providing just the
hostname/ipv4 addresses
anyway I initially checked that, but it's the same exception I got in this
case also
Regards
Dinesh

answered Aug 27 2013 at 22:36

Dinesh 's gravatar image


Have you verified that your firewall is configured for the cassandra
traffic. At the very least you need to make certain the following ports are
open between nodes: 7000, 7001, 7199, 9160, 61620 and 61621.
John Pyeatt
Singlewire Software, LLC
www.singlewire.com
608.661.1184
john.pyeatt@singlewire.com

answered Aug 28 2013 at 14:04

John Pyeatt 's gravatar image


Hi John,
I had my firewall disabled in both the nodes
To make sure. I checked it
# rcSuSEfirewall2 status
Checking the status of SuSEfirewall2
                                                                 unused
if it's, on it says running
Please suggest the further steps, where to look and troubleshoot, if you
have any idea
Regards
Dinesh

answered Aug 28 2013 at 22:41

Dinesh 's gravatar image


To be sure ports are open try to connect from one node to  another:
telnet  7000
try all ports.
Andrey

answered Aug 28 2013 at 22:53

Andrey Ilinykh 's gravatar image


My first node is running and second node is not running in this case
I tried telnet from second node to first node. Following is the stdout
# telnet 10.96.10.207 7000
Trying 10.96.10.207...
Connected to 10.96.10.207.
Escape character is '^]'.
quit
Connection closed by foreign host.
I see the same output for 7199 & 9160 ports. is this correct. I am not
seeing the telnet session as something like telnet>
But when I tried with 7001. I am seeing the below output
# telnet 10.96.10.207 7001
Trying 10.96.10.207...
telnet: connect to address 10.96.10.207: Connection refused
I see the same output for 61620 & 61621 ports
Please suggest
Regards
Dinesh

answered Aug 28 2013 at 23:47

Dinesh 's gravatar image


Have you tried setting rpc_address to the same as listen_address ?
Andy
My first node is running and second node is not running in this case
I tried telnet from second node to first node. Following is the stdout
# telnet 10.96.10.207 7000
Trying 10.96.10.207...
Connected to 10.96.10.207.
Escape character is '^]'.
quit
Connection closed by foreign host.
I see the same output for 7199 & 9160 ports. is this correct. I am not seeing the telnet session as something like telnet>
But when I tried with 7001. I am seeing the below output
# telnet 10.96.10.207 7001
Trying 10.96.10.207...
telnet: connect to address 10.96.10.207: Connection refused
I see the same output for 61620 & 61621 ports
Please suggest
To be sure ports are open try to connect from one node to  another:
telnet  7000
try all ports.
Andrey
Hi John,
I had my firewall disabled in both the nodes
To make sure. I checked it
# rcSuSEfirewall2 status
Checking the status of SuSEfirewall2                                                                                                        unused
if it's, on it says running
Please suggest the further steps, where to look and troubleshoot, if you have any idea
Have you verified that your firewall is configured for the cassandra traffic. At the very least you need to make certain the following ports are open between nodes: 7000, 7001, 7199, 9160, 61620 and 61621.
In my case rpc_address in both the nodes is set to 0.0.0.0 which means it listens on all interfaces. it has a larger scope (to listen on all localhost, ipv4, hostnames, ipv6 addresses) than providing just the hostname/ipv4 addresses
anyway I initially checked that, but it's the same exception I got in this case also
You would need to configure rpc_address also with hostname/ips on both the nodes.
Naresh
Hi,
I am trying to setup a two node Cassandra cluster
Able to start the first node, but not seeing the following exception while starting the second node
ERROR 17:31:34,315 Exception encountered during startup
java.lang.IllegalStateException: Unable to contact any seeds!
        at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
        at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
java.lang.IllegalStateException: Unable to contact any seeds!
        at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:947)
        at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Exception encountered during startup: Unable to contact any seeds!
ERROR 17:31:34,322 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
        at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
        at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
        at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
=============================
My yaml configuration files have these modified
first node yaml
initial_token: -9223372036854775808 # generated this using tokengen tool
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.19.207 # which is the IP of first node itself
rpc_address: 0.0.0.0
second node yaml
initial_token: 0
seeds: "10.96.19.207" # which is the IP of first node
listen_address: 10.96.10.223 # which is the IP of second node
rpc_address: 0.0.0.0
==================
Can anyone please help me what went wrong with my configuration?
Regards
Dinesh
Regards
Dinesh
John Pyeatt
Singlewire Software, LLC
www.singlewire.com
608.661.1184
john.pyeatt@singlewire.com
Regards
Dinesh
Regards
Dinesh
The University of Dundee is a registered Scottish Charity, No: SC015096

answered Aug 29 2013 at 00:10

Andrew Cobley 's gravatar image


Facing the same issue with rpc_address set to the same as listen address on
both nodes
Checked with having
in first node - rpc_address: 10.96.10.207
in second node - rpc_address: 10.96.10.223
Regards
Dinesh

answered Aug 29 2013 at 01:18

Dinesh 's gravatar image


I enabled the debug level for logging. following is the debug entry for the
same exception from node2. Someone please help me
DEBUG 14:53:58,372 Connection version 6 from /10.96.10.105
DEBUG 14:53:58,374 Upgrading incoming connection to be compressed
DEBUG 14:53:58,378 Max version for /10.96.10.105 is 6
DEBUG 14:53:58,379 Setting version 6 for /10.96.10.105
DEBUG 14:53:58,380 set version for /10.96.10.105 to 6
DEBUG 14:53:58,380 Reseting version for /10.96.10.105
DEBUG 14:53:58,683 error writing to rhlcas1/10.96.10.105
java.io.IOException: Broken pipe
        at sun.nio.ch.FileDispatcher.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
        at sun.nio.ch.IOUtil.write(IOUtil.java:40)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:336)
        at java.nio.channels.Channels.writeFullyImpl(Channels.java:59)
        at java.nio.channels.Channels.writeFully(Channels.java:81)
        at java.nio.channels.Channels.access$000(Channels.java:47)
        at java.nio.channels.Channels$1.write(Channels.java:155)
        at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
        at
org.xerial.snappy.SnappyOutputStream.flush(SnappyOutputStream.java:95)
        at java.io.DataOutputStream.flush(DataOutputStream.java:106)
        at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:205)
        at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:149)
Regards
Dinesh

answered Aug 29 2013 at 21:08

Dinesh 's gravatar image


Related discussions

Tagged

Group Cassandra-user

asked Aug 27 2013 at 21:45

active Aug 29 2013 at 21:08

posts:10

users:5

©2013 QnaList.com . QnaList is part of ZisaTechnologies LLC.