You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Amir Shahinpour <am...@holisticlabs.net> on 2017/08/28 22:45:32 UTC
Conection refuse
Hi,
I am getting an error connecting to cqlsh. I am getting the following
error.
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
Connection refused")})
I change the Cassandra.yaml file setting for rpc_address to my ip address
and listen_address to localhost.
listen_address: localhost
rpc_address: my_IP
I also tried to change the cassandra-env.sh to add my IP address but still
same error.
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
Any suggestion?
Re: Conection refuse
Posted by Amir Shahinpour <am...@holisticlabs.net>.
Thanks Akhil, I will follow your steps. Thanks for the links.
Best,
Amirali
On Tue, Aug 29, 2017 at 8:19 PM, Akhil Mehra <ak...@gmail.com> wrote:
> To install make sure:
>
> Your cluster name on each node are the same.
> You configure your seed nodes. Make one node the seed node and added seed
> configuration to all your yaml files.
> Ensure that intra-node communication is on the same port. By default this
> is on port 7000
> Do not startup all nodes at the same time. Only add nodes when others have
> already bootstrapped. This is essential
>
> I have writing a blog post outline installing Cassandra using docker (
> http://abiasforaction.net/apache-cassandra-cluster-docker/). Good way to
> creating a test cluster quickly.
>
> Docker cluster is good for dev and test purposes only. Would not use
> docker for.
>
> Alternately https://github.com/pcmanus/ccm is a popular tool for creating
> and destroying local clusters.
>
> I hope that helps.
>
> Cheers,
> Akhil
>
>
>
> On 30/08/2017, at 1:59 PM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Thanks Akhil,
>
> I will do it. For setting up my second node, do you have any good source
> that I can follow to make sure I am doing everything correct? I have been
> googling around and quite frankly all the source that I found in the Google
> were kind of different from each other and I guess that is why I was not
> able to connect these two nodes together. So I am still not sure what steps
> should I take to add a new node to a cluster. Thanks again.
>
> Thanks in Advance,
> Amir
>
> On Tue, Aug 29, 2017 at 6:53 PM, Akhil Mehra <ak...@gmail.com> wrote:
>
>>
>> If the *data is not important* then stop all nodes. On each node empty
>> your commitlog, data, hints and saved_cache directories.
>>
>> Start one node. Wait for it to boot up successfully i.e. logs have no
>> errors and you can connect to it using cqlsh.
>>
>> Start your second node and make sure it bootstraps and becomes part of
>> the cluster. Since you will have no data this should be quick and simple.
>>
>> Regards,
>> Akhil
>>
>> On 30/08/2017, at 1:24 PM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>
>> Akhil,
>>
>> Commit log directory from yaml file is: /var/lib/cassandra/commitlog
>>
>> So basically I removed it. Can I copy a new one from another node? or
>> somehow generate one?
>>
>> Yes, the rm -rf was on the original and the only node. I stopped the C*
>> and ran the rm -rf /var/lib/cassandra/*.
>> At this point, the data that I lost is not very important, because it was
>> a dev environment that I am setting up. But I have to be able to make this
>> node running and talking to the new node. Neither CQLSH nor nodetool works
>> at this time.
>>
>> Best,
>> Amirali
>>
>>
>> On Tue, Aug 29, 2017 at 2:49 PM, Akhil Mehra <ak...@gmail.com> w
>> rote:
>> What directory was the data and commit logs stored on the original
>> working node. You can look up your cassandra.yaml to figure this out. Its
>> good to confirm.
>>
>> Was the rm -rf run on the original working node?
>>
>> Cheers,
>> Akhil
>>
>> On 30/08/2017, at 9:37 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>
>> Yes both of the nodes are down.
>>
>> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <ak...@gmail.com> wrote:
>> Cassandra is doing a health check when it is starting up and failing due
>> to being unable to ready files in the system key space. Here is the comment
>> in the segment of the code that threw the exception.
>>
>> https://github.com/apache/cassandra/blob/trunk/src/java/org/
>> apache/cassandra/db/SystemKeyspace.java#L804-L810
>> /**
>> * One of three things will happen if you try to read the system
>> keyspace:
>> * 1. files are present and you can read them: great
>> * 2. no files are there: great (new node is assumed)
>> * 3. files are present but you can't read them: bad
>> * @throws ConfigurationException
>> */
>>
>> Removing files for bootstrapping (adding a new node) a node sounds
>> incorrect. Depending on your configuration the /var/lib/cassandar by
>> default houses table data, commit logs, hints and cache. An rm -rf on it
>> sounds ominous.
>>
>> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>>
>> Regards,
>> Akhil
>>
>>
>>
>>
>> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>
>> Hi Lucas,
>>
>> Thanks for your response. So I checked the system.log and I found the
>> following error at the end which I think is causing the problem.
>>
>> Fatal exception during initialization
>> org.apache.cassandra.exceptions.ConfigurationException: Found system
>> keyspace files, but they couldn't be loaded!
>>
>> It could be due to removing some of the data. I ran the following command
>> to remove some data. sudo rm -rf /var/lib/cassandra/*
>>
>> I am new to Cassandra and I think I made a mistake. So I had only one
>> node which was working fine with my tables that I had. I wanted to add a
>> second node and start using the real power of Cassandra. So I follow one of
>> post that I found, there were some changes in cassandra.yaml file and
>> afterwards I had to remove the files and that's why I run the remove
>> command. So right now neither of CQLSH and nodetool works. Please let me
>> know if you need any other information.
>>
>> Here is a screenshot of the system.log. Thanks a lot for your help.
>>
>> Best,
>> Amir
>>
>> <image.png>
>>
>> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lucas@maurobenevide
>> s.com.br> wrote:
>> Hello Amir,
>>
>> You should see the log. If it was installed by the apt-get tool, it
>> should be in /var/log/cassandra/system.log.
>> It can occur when the schema of the node you are trying to connect is out
>> of date with the cluster.
>> How many nodes are there in you cluster?
>> What is the output of "nodetool describecluster"?
>>
>> Best regards,
>> Lucas Benevides
>>
>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
>> Hi,
>>
>> I am getting an error connecting to cqlsh. I am getting the following
>> error.
>>
>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>> Connection refused")})
>>
>> I change the Cassandra.yaml file setting for rpc_address to my ip address
>> and listen_address to localhost.
>>
>>
>> listen_address: localhost
>> rpc_address: my_IP
>>
>> I also tried to change the cassandra-env.sh to add my IP address but
>> still same error.
>>
>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>> Any suggestion?
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
Re: Conection refuse
Posted by Akhil Mehra <ak...@gmail.com>.
To install make sure:
Your cluster name on each node are the same.
You configure your seed nodes. Make one node the seed node and added seed configuration to all your yaml files.
Ensure that intra-node communication is on the same port. By default this is on port 7000
Do not startup all nodes at the same time. Only add nodes when others have already bootstrapped. This is essential
I have writing a blog post outline installing Cassandra using docker (http://abiasforaction.net/apache-cassandra-cluster-docker/ <http://abiasforaction.net/apache-cassandra-cluster-docker/>). Good way to creating a test cluster quickly.
Docker cluster is good for dev and test purposes only. Would not use docker for.
Alternately https://github.com/pcmanus/ccm <https://github.com/pcmanus/ccm> is a popular tool for creating and destroying local clusters.
I hope that helps.
Cheers,
Akhil
> On 30/08/2017, at 1:59 PM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Thanks Akhil,
>
> I will do it. For setting up my second node, do you have any good source that I can follow to make sure I am doing everything correct? I have been googling around and quite frankly all the source that I found in the Google were kind of different from each other and I guess that is why I was not able to connect these two nodes together. So I am still not sure what steps should I take to add a new node to a cluster. Thanks again.
>
> Thanks in Advance,
> Amir
>
> On Tue, Aug 29, 2017 at 6:53 PM, Akhil Mehra <akhilmehra@gmail.com <ma...@gmail.com>> wrote:
>
> If the data is not important then stop all nodes. On each node empty your commitlog, data, hints and saved_cache directories.
>
> Start one node. Wait for it to boot up successfully i.e. logs have no errors and you can connect to it using cqlsh.
>
> Start your second node and make sure it bootstraps and becomes part of the cluster. Since you will have no data this should be quick and simple.
>
> Regards,
> Akhil
>
>> On 30/08/2017, at 1:24 PM, Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>> wrote:
>>
>> Akhil,
>>
>> Commit log directory from yaml file is: /var/lib/cassandra/commitlog
>>
>> So basically I removed it. Can I copy a new one from another node? or somehow generate one?
>>
>> Yes, the rm -rf was on the original and the only node. I stopped the C* and ran the rm -rf /var/lib/cassandra/*.
>> At this point, the data that I lost is not very important, because it was a dev environment that I am setting up. But I have to be able to make this node running and talking to the new node. Neither CQLSH nor nodetool works at this time.
>>
>> Best,
>> Amirali
>>
>>
>> On Tue, Aug 29, 2017 at 2:49 PM, Akhil Mehra <akhilmehra@gmail.com <ma...@gmail.com>> wrote:
>> What directory was the data and commit logs stored on the original working node. You can look up your cassandra.yaml to figure this out. Its good to confirm.
>>
>> Was the rm -rf run on the original working node?
>>
>> Cheers,
>> Akhil
>>
>>> On 30/08/2017, at 9:37 AM, Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>> wrote:
>>>
>>> Yes both of the nodes are down.
>>>
>>> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <akhilmehra@gmail.com <ma...@gmail.com>> wrote:
>>> Cassandra is doing a health check when it is starting up and failing due to being unable to ready files in the system key space. Here is the comment in the segment of the code that threw the exception.
>>>
>>> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810 <https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810>
>>> /**
>>> * One of three things will happen if you try to read the system keyspace:
>>> * 1. files are present and you can read them: great
>>> * 2. no files are there: great (new node is assumed)
>>> * 3. files are present but you can't read them: bad
>>> * @throws ConfigurationException
>>> */
>>>
>>> Removing files for bootstrapping (adding a new node) a node sounds incorrect. Depending on your configuration the /var/lib/cassandar by default houses table data, commit logs, hints and cache. An rm -rf on it sounds ominous.
>>>
>>> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>>>
>>> Regards,
>>> Akhil
>>>
>>>
>>>
>>>
>>>> On 30/08/2017, at 9:01 AM, Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>> wrote:
>>>>
>>>> Hi Lucas,
>>>>
>>>> Thanks for your response. So I checked the system.log and I found the following error at the end which I think is causing the problem.
>>>>
>>>> Fatal exception during initialization
>>>> org.apache.cassandra.exceptions.ConfigurationException: Found system keyspace files, but they couldn't be loaded!
>>>>
>>>> It could be due to removing some of the data. I ran the following command to remove some data. sudo rm -rf /var/lib/cassandra/*
>>>>
>>>> I am new to Cassandra and I think I made a mistake. So I had only one node which was working fine with my tables that I had. I wanted to add a second node and start using the real power of Cassandra. So I follow one of post that I found, there were some changes in cassandra.yaml file and afterwards I had to remove the files and that's why I run the remove command. So right now neither of CQLSH and nodetool works. Please let me know if you need any other information.
>>>>
>>>> Here is a screenshot of the system.log. Thanks a lot for your help.
>>>>
>>>> Best,
>>>> Amir
>>>>
>>>> <image.png>
>>>>
>>>> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lucas@maurobenevides.com.br <ma...@maurobenevides.com.br>> wrote:
>>>> Hello Amir,
>>>>
>>>> You should see the log. If it was installed by the apt-get tool, it should be in /var/log/cassandra/system.log.
>>>> It can occur when the schema of the node you are trying to connect is out of date with the cluster.
>>>> How many nodes are there in you cluster?
>>>> What is the output of "nodetool describecluster"?
>>>>
>>>> Best regards,
>>>> Lucas Benevides
>>>>
>>>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>>:
>>>> Hi,
>>>>
>>>> I am getting an error connecting to cqlsh. I am getting the following error.
>>>>
>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>>>>
>>>> I change the Cassandra.yaml file setting for rpc_address to my ip address and listen_address to localhost.
>>>>
>>>>
>>>> listen_address: localhost
>>>> rpc_address: my_IP
>>>>
>>>> I also tried to change the cassandra-env.sh to add my IP address but still same error.
>>>>
>>>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>>>> Any suggestion?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>
>
Re: Conection refuse
Posted by Amir Shahinpour <am...@holisticlabs.net>.
Thanks Akhil,
I will do it. For setting up my second node, do you have any good source
that I can follow to make sure I am doing everything correct? I have been
googling around and quite frankly all the source that I found in the Google
were kind of different from each other and I guess that is why I was not
able to connect these two nodes together. So I am still not sure what steps
should I take to add a new node to a cluster. Thanks again.
Thanks in Advance,
Amir
On Tue, Aug 29, 2017 at 6:53 PM, Akhil Mehra <ak...@gmail.com> wrote:
>
> If the *data is not important* then stop all nodes. On each node empty
> your commitlog, data, hints and saved_cache directories.
>
> Start one node. Wait for it to boot up successfully i.e. logs have no
> errors and you can connect to it using cqlsh.
>
> Start your second node and make sure it bootstraps and becomes part of the
> cluster. Since you will have no data this should be quick and simple.
>
> Regards,
> Akhil
>
> On 30/08/2017, at 1:24 PM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Akhil,
>
> Commit log directory from yaml file is: /var/lib/cassandra/commitlog
>
> So basically I removed it. Can I copy a new one from another node? or
> somehow generate one?
>
> Yes, the rm -rf was on the original and the only node. I stopped the C*
> and ran the rm -rf /var/lib/cassandra/*.
> At this point, the data that I lost is not very important, because it was
> a dev environment that I am setting up. But I have to be able to make this
> node running and talking to the new node. Neither CQLSH nor nodetool works
> at this time.
>
> Best,
> Amirali
>
>
> On Tue, Aug 29, 2017 at 2:49 PM, Akhil Mehra <ak...@gmail.com> wrote:
> What directory was the data and commit logs stored on the original working
> node. You can look up your cassandra.yaml to figure this out. Its good to
> confirm.
>
> Was the rm -rf run on the original working node?
>
> Cheers,
> Akhil
>
> On 30/08/2017, at 9:37 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Yes both of the nodes are down.
>
> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <ak...@gmail.com> wrote:
> Cassandra is doing a health check when it is starting up and failing due
> to being unable to ready files in the system key space. Here is the comment
> in the segment of the code that threw the exception.
>
> https://github.com/apache/cassandra/blob/trunk/src/java/
> org/apache/cassandra/db/SystemKeyspace.java#L804-L810
> /**
> * One of three things will happen if you try to read the system
> keyspace:
> * 1. files are present and you can read them: great
> * 2. no files are there: great (new node is assumed)
> * 3. files are present but you can't read them: bad
> * @throws ConfigurationException
> */
>
> Removing files for bootstrapping (adding a new node) a node sounds
> incorrect. Depending on your configuration the /var/lib/cassandar by
> default houses table data, commit logs, hints and cache. An rm -rf on it
> sounds ominous.
>
> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>
> Regards,
> Akhil
>
>
>
>
> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Hi Lucas,
>
> Thanks for your response. So I checked the system.log and I found the
> following error at the end which I think is causing the problem.
>
> Fatal exception during initialization
> org.apache.cassandra.exceptions.ConfigurationException: Found system
> keyspace files, but they couldn't be loaded!
>
> It could be due to removing some of the data. I ran the following command
> to remove some data. sudo rm -rf /var/lib/cassandra/*
>
> I am new to Cassandra and I think I made a mistake. So I had only one node
> which was working fine with my tables that I had. I wanted to add a second
> node and start using the real power of Cassandra. So I follow one of post
> that I found, there were some changes in cassandra.yaml file and afterwards
> I had to remove the files and that's why I run the remove command. So right
> now neither of CQLSH and nodetool works. Please let me know if you need
> any other information.
>
> Here is a screenshot of the system.log. Thanks a lot for your help.
>
> Best,
> Amir
>
> <image.png>
>
> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lucas@
> maurobenevides.com.br> wrote:
> Hello Amir,
>
> You should see the log. If it was installed by the apt-get tool, it should
> be in /var/log/cassandra/system.log.
> It can occur when the schema of the node you are trying to connect is out
> of date with the cluster.
> How many nodes are there in you cluster?
> What is the output of "nodetool describecluster"?
>
> Best regards,
> Lucas Benevides
>
> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
> Hi,
>
> I am getting an error connecting to cqlsh. I am getting the following
> error.
>
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
> Connection refused")})
>
> I change the Cassandra.yaml file setting for rpc_address to my ip address
> and listen_address to localhost.
>
>
> listen_address: localhost
> rpc_address: my_IP
>
> I also tried to change the cassandra-env.sh to add my IP address but
> still same error.
>
> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
> Any suggestion?
>
>
>
>
>
>
>
>
>
>
Re: Conection refuse
Posted by Akhil Mehra <ak...@gmail.com>.
If the data is not important then stop all nodes. On each node empty your commitlog, data, hints and saved_cache directories.
Start one node. Wait for it to boot up successfully i.e. logs have no errors and you can connect to it using cqlsh.
Start your second node and make sure it bootstraps and becomes part of the cluster. Since you will have no data this should be quick and simple.
Regards,
Akhil
> On 30/08/2017, at 1:24 PM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Akhil,
>
> Commit log directory from yaml file is: /var/lib/cassandra/commitlog
>
> So basically I removed it. Can I copy a new one from another node? or somehow generate one?
>
> Yes, the rm -rf was on the original and the only node. I stopped the C* and ran the rm -rf /var/lib/cassandra/*.
> At this point, the data that I lost is not very important, because it was a dev environment that I am setting up. But I have to be able to make this node running and talking to the new node. Neither CQLSH nor nodetool works at this time.
>
> Best,
> Amirali
>
>
> On Tue, Aug 29, 2017 at 2:49 PM, Akhil Mehra <ak...@gmail.com> wrote:
> What directory was the data and commit logs stored on the original working node. You can look up your cassandra.yaml to figure this out. Its good to confirm.
>
> Was the rm -rf run on the original working node?
>
> Cheers,
> Akhil
>
>> On 30/08/2017, at 9:37 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>
>> Yes both of the nodes are down.
>>
>> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <ak...@gmail.com> wrote:
>> Cassandra is doing a health check when it is starting up and failing due to being unable to ready files in the system key space. Here is the comment in the segment of the code that threw the exception.
>>
>> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810
>> /**
>> * One of three things will happen if you try to read the system keyspace:
>> * 1. files are present and you can read them: great
>> * 2. no files are there: great (new node is assumed)
>> * 3. files are present but you can't read them: bad
>> * @throws ConfigurationException
>> */
>>
>> Removing files for bootstrapping (adding a new node) a node sounds incorrect. Depending on your configuration the /var/lib/cassandar by default houses table data, commit logs, hints and cache. An rm -rf on it sounds ominous.
>>
>> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>>
>> Regards,
>> Akhil
>>
>>
>>
>>
>>> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>>
>>> Hi Lucas,
>>>
>>> Thanks for your response. So I checked the system.log and I found the following error at the end which I think is causing the problem.
>>>
>>> Fatal exception during initialization
>>> org.apache.cassandra.exceptions.ConfigurationException: Found system keyspace files, but they couldn't be loaded!
>>>
>>> It could be due to removing some of the data. I ran the following command to remove some data. sudo rm -rf /var/lib/cassandra/*
>>>
>>> I am new to Cassandra and I think I made a mistake. So I had only one node which was working fine with my tables that I had. I wanted to add a second node and start using the real power of Cassandra. So I follow one of post that I found, there were some changes in cassandra.yaml file and afterwards I had to remove the files and that's why I run the remove command. So right now neither of CQLSH and nodetool works. Please let me know if you need any other information.
>>>
>>> Here is a screenshot of the system.log. Thanks a lot for your help.
>>>
>>> Best,
>>> Amir
>>>
>>> <image.png>
>>>
>>> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lu...@maurobenevides.com.br> wrote:
>>> Hello Amir,
>>>
>>> You should see the log. If it was installed by the apt-get tool, it should be in /var/log/cassandra/system.log.
>>> It can occur when the schema of the node you are trying to connect is out of date with the cluster.
>>> How many nodes are there in you cluster?
>>> What is the output of "nodetool describecluster"?
>>>
>>> Best regards,
>>> Lucas Benevides
>>>
>>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
>>> Hi,
>>>
>>> I am getting an error connecting to cqlsh. I am getting the following error.
>>>
>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>>>
>>> I change the Cassandra.yaml file setting for rpc_address to my ip address and listen_address to localhost.
>>>
>>>
>>> listen_address: localhost
>>> rpc_address: my_IP
>>>
>>> I also tried to change the cassandra-env.sh to add my IP address but still same error.
>>>
>>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>>> Any suggestion?
>>>
>>>
>>>
>>>
>>>
>>
>
>
Re: Conection refuse
Posted by Amir Shahinpour <am...@holisticlabs.net>.
Akhil,
Commit log directory from yaml file is: /var/lib/cassandra/commitlog
So basically I removed it. Can I copy a new one from another node? or
somehow generate one?
Yes, the rm -rf was on the original and the only node. I stopped the C* and
ran the rm -rf /var/lib/cassandra/*.
At this point, the data that I lost is not very important, because it was a
dev environment that I am setting up. But I have to be able to make this
node running and talking to the new node. Neither CQLSH nor nodetool works
at this time.
Best,
Amirali
On Tue, Aug 29, 2017 at 2:49 PM, Akhil Mehra <ak...@gmail.com> wrote:
> What directory was the data and commit logs stored on the original working
> node. You can look up your cassandra.yaml to figure this out. Its good to
> confirm.
>
> Was the rm -rf run on the original working node?
>
> Cheers,
> Akhil
>
> On 30/08/2017, at 9:37 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Yes both of the nodes are down.
>
> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <ak...@gmail.com> wrote:
>
>> Cassandra is doing a health check when it is starting up and failing due
>> to being unable to ready files in the system key space. Here is the comment
>> in the segment of the code that threw the exception.
>>
>> https://github.com/apache/cassandra/blob/trunk/src/java/org/
>> apache/cassandra/db/SystemKeyspace.java#L804-L810
>> /**
>> * One of three things will happen if you try to read the system keyspace:
>> * 1. files are present and you can read them: great
>> * 2. no files are there: great (new node is assumed)
>> * 3. files are present but you can't read them: bad
>> * @throws ConfigurationException
>> */
>>
>> Removing files for bootstrapping (adding a new node) a node sounds
>> incorrect. Depending on your configuration the /var/lib/cassandar by
>> default houses table data, commit logs, hints and cache. An rm -rf on it
>> sounds ominous.
>>
>> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>>
>> Regards,
>> Akhil
>>
>>
>>
>>
>> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>>
>> Hi Lucas,
>>
>> Thanks for your response. So I checked the system.log and I found the
>> following error at the end which I think is causing the problem.
>>
>> Fatal exception during initialization
>> org.apache.cassandra.exceptions.ConfigurationException: Found system
>> keyspace files, but they couldn't be loaded!
>>
>> It could be due to removing some of the data. I ran the following command
>> to remove some data. sudo rm -rf /var/lib/cassandra/*
>>
>> I am new to Cassandra and I think I made a mistake. So I had only one
>> node which was working fine with my tables that I had. I wanted to add a
>> second node and start using the real power of Cassandra. So I follow one of
>> post that I found, there were some changes in cassandra.yaml file and
>> afterwards I had to remove the files and that's why I run the remove
>> command. So right now neither of CQLSH and nodetool works. Please let me
>> know if you need any other information.
>>
>> Here is a screenshot of the system.log. Thanks a lot for your help.
>>
>> Best,
>> Amir
>>
>> <image.png>
>>
>> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <
>> lucas@maurobenevides.com.br> wrote:
>>
>>> Hello Amir,
>>>
>>> You should see the log. If it was installed by the apt-get tool, it
>>> should be in /var/log/cassandra/system.log.
>>> It can occur when the schema of the node you are trying to connect is
>>> out of date with the cluster.
>>> How many nodes are there in you cluster?
>>> What is the output of "nodetool describecluster"?
>>>
>>> Best regards,
>>> Lucas Benevides
>>>
>>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
>>>
>>>> Hi,
>>>>
>>>> I am getting an error connecting to cqlsh. I am getting the following
>>>> error.
>>>>
>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>>>> Connection refused")})
>>>>
>>>> I change the Cassandra.yaml file setting for rpc_address to my ip
>>>> address and listen_address to localhost.
>>>>
>>>>
>>>> listen_address: localhost
>>>> rpc_address: my_IP
>>>>
>>>> I also tried to change the cassandra-env.sh to add my IP address but
>>>> still same error.
>>>>
>>>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>>>>
>>>> Any suggestion?
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Conection refuse
Posted by Akhil Mehra <ak...@gmail.com>.
What directory was the data and commit logs stored on the original working node. You can look up your cassandra.yaml to figure this out. Its good to confirm.
Was the rm -rf run on the original working node?
Cheers,
Akhil
> On 30/08/2017, at 9:37 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Yes both of the nodes are down.
>
> On Aug 29, 2017 2:30 PM, "Akhil Mehra" <akhilmehra@gmail.com <ma...@gmail.com>> wrote:
> Cassandra is doing a health check when it is starting up and failing due to being unable to ready files in the system key space. Here is the comment in the segment of the code that threw the exception.
>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810 <https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810>
> /**
> * One of three things will happen if you try to read the system keyspace:
> * 1. files are present and you can read them: great
> * 2. no files are there: great (new node is assumed)
> * 3. files are present but you can't read them: bad
> * @throws ConfigurationException
> */
>
> Removing files for bootstrapping (adding a new node) a node sounds incorrect. Depending on your configuration the /var/lib/cassandar by default houses table data, commit logs, hints and cache. An rm -rf on it sounds ominous.
>
> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>
> Regards,
> Akhil
>
>
>
>
>> On 30/08/2017, at 9:01 AM, Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>> wrote:
>>
>> Hi Lucas,
>>
>> Thanks for your response. So I checked the system.log and I found the following error at the end which I think is causing the problem.
>>
>> Fatal exception during initialization
>> org.apache.cassandra.exceptions.ConfigurationException: Found system keyspace files, but they couldn't be loaded!
>>
>> It could be due to removing some of the data. I ran the following command to remove some data. sudo rm -rf /var/lib/cassandra/*
>>
>> I am new to Cassandra and I think I made a mistake. So I had only one node which was working fine with my tables that I had. I wanted to add a second node and start using the real power of Cassandra. So I follow one of post that I found, there were some changes in cassandra.yaml file and afterwards I had to remove the files and that's why I run the remove command. So right now neither of CQLSH and nodetool works. Please let me know if you need any other information.
>>
>> Here is a screenshot of the system.log. Thanks a lot for your help.
>>
>> Best,
>> Amir
>>
>> <image.png>
>>
>> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lucas@maurobenevides.com.br <ma...@maurobenevides.com.br>> wrote:
>> Hello Amir,
>>
>> You should see the log. If it was installed by the apt-get tool, it should be in /var/log/cassandra/system.log.
>> It can occur when the schema of the node you are trying to connect is out of date with the cluster.
>> How many nodes are there in you cluster?
>> What is the output of "nodetool describecluster"?
>>
>> Best regards,
>> Lucas Benevides
>>
>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>>:
>> Hi,
>>
>> I am getting an error connecting to cqlsh. I am getting the following error.
>>
>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>>
>> I change the Cassandra.yaml file setting for rpc_address to my ip address and listen_address to localhost.
>>
>>
>> listen_address: localhost
>> rpc_address: my_IP
>>
>> I also tried to change the cassandra-env.sh to add my IP address but still same error.
>>
>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>> Any suggestion?
>>
>>
>>
>>
>>
>
Re: Conection refuse
Posted by Amir Shahinpour <am...@holisticlabs.net>.
Yes both of the nodes are down.
On Aug 29, 2017 2:30 PM, "Akhil Mehra" <ak...@gmail.com> wrote:
> Cassandra is doing a health check when it is starting up and failing due
> to being unable to ready files in the system key space. Here is the comment
> in the segment of the code that threw the exception.
>
> https://github.com/apache/cassandra/blob/trunk/src/java/
> org/apache/cassandra/db/SystemKeyspace.java#L804-L810
> /**
> * One of three things will happen if you try to read the system keyspace:
> * 1. files are present and you can read them: great
> * 2. no files are there: great (new node is assumed)
> * 3. files are present but you can't read them: bad
> * @throws ConfigurationException
> */
>
> Removing files for bootstrapping (adding a new node) a node sounds
> incorrect. Depending on your configuration the /var/lib/cassandar by
> default houses table data, commit logs, hints and cache. An rm -rf on it
> sounds ominous.
>
> Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
>
> Regards,
> Akhil
>
>
>
>
> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Hi Lucas,
>
> Thanks for your response. So I checked the system.log and I found the
> following error at the end which I think is causing the problem.
>
> Fatal exception during initialization
> org.apache.cassandra.exceptions.ConfigurationException: Found system
> keyspace files, but they couldn't be loaded!
>
> It could be due to removing some of the data. I ran the following command
> to remove some data. sudo rm -rf /var/lib/cassandra/*
>
> I am new to Cassandra and I think I made a mistake. So I had only one node
> which was working fine with my tables that I had. I wanted to add a second
> node and start using the real power of Cassandra. So I follow one of post
> that I found, there were some changes in cassandra.yaml file and
> afterwards I had to remove the files and that's why I run the remove
> command. So right now neither of CQLSH and nodetool works. Please let me
> know if you need any other information.
>
> Here is a screenshot of the system.log. Thanks a lot for your help.
>
> Best,
> Amir
>
> <image.png>
>
> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <
> lucas@maurobenevides.com.br> wrote:
>
>> Hello Amir,
>>
>> You should see the log. If it was installed by the apt-get tool, it
>> should be in /var/log/cassandra/system.log.
>> It can occur when the schema of the node you are trying to connect is out
>> of date with the cluster.
>> How many nodes are there in you cluster?
>> What is the output of "nodetool describecluster"?
>>
>> Best regards,
>> Lucas Benevides
>>
>> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
>>
>>> Hi,
>>>
>>> I am getting an error connecting to cqlsh. I am getting the following
>>> error.
>>>
>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>>> Connection refused")})
>>>
>>> I change the Cassandra.yaml file setting for rpc_address to my ip
>>> address and listen_address to localhost.
>>>
>>>
>>> listen_address: localhost
>>> rpc_address: my_IP
>>>
>>> I also tried to change the cassandra-env.sh to add my IP address but
>>> still same error.
>>>
>>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>>>
>>> Any suggestion?
>>>
>>>
>>>
>>>
>>
>
>
Re: Conection refuse
Posted by Akhil Mehra <ak...@gmail.com>.
Cassandra is doing a health check when it is starting up and failing due to being unable to ready files in the system key space. Here is the comment in the segment of the code that threw the exception.
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L804-L810
/**
* One of three things will happen if you try to read the system keyspace:
* 1. files are present and you can read them: great
* 2. no files are there: great (new node is assumed)
* 3. files are present but you can't read them: bad
* @throws ConfigurationException
*/
Removing files for bootstrapping (adding a new node) a node sounds incorrect. Depending on your configuration the /var/lib/cassandar by default houses table data, commit logs, hints and cache. An rm -rf on it sounds ominous.
Are both your nodes down i.e. you cannot cqlsh in any of your nodes?
Regards,
Akhil
> On 30/08/2017, at 9:01 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Hi Lucas,
>
> Thanks for your response. So I checked the system.log and I found the following error at the end which I think is causing the problem.
>
> Fatal exception during initialization
> org.apache.cassandra.exceptions.ConfigurationException: Found system keyspace files, but they couldn't be loaded!
>
> It could be due to removing some of the data. I ran the following command to remove some data. sudo rm -rf /var/lib/cassandra/*
>
> I am new to Cassandra and I think I made a mistake. So I had only one node which was working fine with my tables that I had. I wanted to add a second node and start using the real power of Cassandra. So I follow one of post that I found, there were some changes in cassandra.yaml file and afterwards I had to remove the files and that's why I run the remove command. So right now neither of CQLSH and nodetool works. Please let me know if you need any other information.
>
> Here is a screenshot of the system.log. Thanks a lot for your help.
>
> Best,
> Amir
>
> <image.png>
>
> On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <lucas@maurobenevides.com.br <ma...@maurobenevides.com.br>> wrote:
> Hello Amir,
>
> You should see the log. If it was installed by the apt-get tool, it should be in /var/log/cassandra/system.log.
> It can occur when the schema of the node you are trying to connect is out of date with the cluster.
> How many nodes are there in you cluster?
> What is the output of "nodetool describecluster"?
>
> Best regards,
> Lucas Benevides
>
> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <amir@holisticlabs.net <ma...@holisticlabs.net>>:
> Hi,
>
> I am getting an error connecting to cqlsh. I am getting the following error.
>
> Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>
> I change the Cassandra.yaml file setting for rpc_address to my ip address and listen_address to localhost.
>
>
> listen_address: localhost
> rpc_address: my_IP
>
> I also tried to change the cassandra-env.sh to add my IP address but still same error.
>
> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
> Any suggestion?
>
>
>
>
>
Re: Conection refuse
Posted by Amir Shahinpour <am...@holisticlabs.net>.
Hi Lucas,
Thanks for your response. So I checked the system.log and I found the
following error at the end which I think is causing the problem.
Fatal exception during initialization
org.apache.cassandra.exceptions.ConfigurationException: Found system
keyspace files, but they couldn't be loaded!
It could be due to removing some of the data. I ran the following command
to remove some data. sudo rm -rf /var/lib/cassandra/*
I am new to Cassandra and I think I made a mistake. So I had only one node
which was working fine with my tables that I had. I wanted to add a second
node and start using the real power of Cassandra. So I follow one of post
that I found, there were some changes in cassandra.yaml file and
afterwards I had to remove the files and that's why I run the remove
command. So right now neither of CQLSH and nodetool works. Please let me
know if you need any other information.
Here is a screenshot of the system.log. Thanks a lot for your help.
Best,
Amir
[image: Inline image 1]
On Tue, Aug 29, 2017 at 7:17 AM, Lucas Benevides <
lucas@maurobenevides.com.br> wrote:
> Hello Amir,
>
> You should see the log. If it was installed by the apt-get tool, it should
> be in /var/log/cassandra/system.log.
> It can occur when the schema of the node you are trying to connect is out
> of date with the cluster.
> How many nodes are there in you cluster?
> What is the output of "nodetool describecluster"?
>
> Best regards,
> Lucas Benevides
>
> 2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
>
>> Hi,
>>
>> I am getting an error connecting to cqlsh. I am getting the following
>> error.
>>
>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>> Connection refused")})
>>
>> I change the Cassandra.yaml file setting for rpc_address to my ip address
>> and listen_address to localhost.
>>
>>
>> listen_address: localhost
>> rpc_address: my_IP
>>
>> I also tried to change the cassandra-env.sh to add my IP address but
>> still same error.
>>
>> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>>
>> Any suggestion?
>>
>>
>>
>>
>
Re: Conection refuse
Posted by Lucas Benevides <lu...@maurobenevides.com.br>.
Hello Amir,
You should see the log. If it was installed by the apt-get tool, it should
be in /var/log/cassandra/system.log.
It can occur when the schema of the node you are trying to connect is out
of date with the cluster.
How many nodes are there in you cluster?
What is the output of "nodetool describecluster"?
Best regards,
Lucas Benevides
2017-08-28 19:45 GMT-03:00 Amir Shahinpour <am...@holisticlabs.net>:
> Hi,
>
> I am getting an error connecting to cqlsh. I am getting the following
> error.
>
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
> Connection refused")})
>
> I change the Cassandra.yaml file setting for rpc_address to my ip address
> and listen_address to localhost.
>
>
> listen_address: localhost
> rpc_address: my_IP
>
> I also tried to change the cassandra-env.sh to add my IP address but
> still same error.
>
> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
>
> Any suggestion?
>
>
>
>
Re: Conection refuse
Posted by Akhil Mehra <ak...@gmail.com>.
Is this a local install? If so you can set your rpc_address to 127.0.0.1. The rpc_address are the IP addresses that will be allowed to make connections to Cassandra.
I suspect the problem is that you have set your RPC address to your IP address with is different from 127.0.0.1. cqlsh by default connects to localhost.
You can set the rpc_address to 0.0.0.0 for allowing anyone to connect. Possible option to eliminate Cassandra rejecting connection. Note this is a security hole for production deployments.
Cheers,
Warm Regards,
Akhil Mehra
http://abiasforaction.net
> On 29/08/2017, at 10:45 AM, Amir Shahinpour <am...@holisticlabs.net> wrote:
>
> Hi,
>
> I am getting an error connecting to cqlsh. I am getting the following error.
>
> Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
>
> I change the Cassandra.yaml file setting for rpc_address to my ip address and listen_address to localhost.
>
>
> listen_address: localhost
> rpc_address: my_IP
>
> I also tried to change the cassandra-env.sh to add my IP address but still same error.
>
> JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=my_IP"
> Any suggestion?
>
>
>