By default, Cassandra JMX listens to localhost only. When Cassandra started up, port 7199 was brought up on localhost. This issue has been bothering me for nearly 2 weeks until I read everything hint in the cassandra-env.sh and refer to this documentation https://docs.datastax.com/en/
Hopefully, this blog will help someone with a similar issue. The ConnectException can be vague and refer to other issues. As far as I can tell, the same error message has 4 variants of different issues.
Note: anything connecting to JMX monitoring will fail with ConnectException
cassandra@cassandra01:/etc/
nodetool: Failed to connect to '102.10.10.91:7199' - ConnectException: 'Connection refused (Connection refused)'.
cassandra@cassandra01:/etc/
nodetool: Failed to connect to '102.10.10.91:7199' - ConnectException: 'Connection refused (Connection refused)'.
Further research showing Cassandra will only be brought up to localhost.
cassandra@cassandra01:/etc/
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 1778 cassandra 95u IPv4 28317 0t0 TCP localhost:7199 (LISTEN)
Here is how to resolve the issue.
in Cassandra-env.sh
Enable and set the port for JMS as 7199
Uncomment and insert the ip address
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<
TO
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=
Change the yes as local to no, so it enables JMX remote connectivity.
if [ "x$LOCAL_JMX" = "x" ]; then
LOCAL_JMX=yes
fi
TO
if [ "x$LOCAL_JMX" = "x" ]; then
LOCAL_JMX=no
fi
Change the remote authentication from false to true. All these have to be done in the set. Changing on without the others will not work.
if [ "$LOCAL_JMX" = "yes" ]; then
JVM_OPTS="$JVM_OPTS -Dcassandra.jmx.local.port=$
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.
else
TO
if [ "$LOCAL_JMX" = "yes" ]; then
JVM_OPTS="$JVM_OPTS -Dcassandra.jmx.local.port=$
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.
else
Create and insert the following. This will allow the jmx to go to the file to grab the username and password.
sudo vi /etc/cassandra/jmxremote.
cassandra casspass
chown cassandra:cassandra /etc/cassandra/jmxremote.
sudo chown cassandra:cassandra /etc/cassandra/jmxremote.
sudo chmod 400 /etc/cassandra/jmxremote.
Restart Cassandra
cassandra@cassandra01:/etc/
Test the dedicated host connectivity with username and password
cassandra@cassandra01:/etc/
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/
-- Address Load Tokens Owns (effective) Host ID Rack
UN 102/10/10.92 24.42 MiB 16 51.2% d9d37072-817a-4095-87da-
UN 102.10.10.91 23.31 MiB 16 48.8% 37562f38-b9da-4e77-b44d-
It is no longer listening to only localhost.
cassandra@cassandra01:/etc/
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 3293 cassandra 95u IPv4 58397 0t0 TCP *:7199 (LISTEN)
Test without username and password will work as well.
cassandra@cassandra01:/etc/
