You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Xu, Zaili" <ZX...@pershing.com> on 2012/09/14 16:05:48 UTC
Is it possible to create a schema before a Cassandra node starts up
?
Guys
I am pretty new to Cassandra. I have a script that needs to set up a schema first before starting up the cassandra node. Is this possible ? Can I create the schema directly on cassandra storage and then when the node starts up it will pick up the schema ?
Zaili
From: rohit reddy [mailto:rohit.kommareddy@gmail.com]
Sent: Friday, September 14, 2012 9:50 AM
To: user@cassandra.apache.org
Subject: Re: Cassandra node going down
Hi Robin,
I had checked that. Our disk size is about 800GB, and the total data size is not more than 40GB. Even if all the data is stored in one node, this won't happen.
I'll try to see if the disk failed.
Is this anything to do with VM memory?.. cause this logs suggests that..
Heap is 0.7515559786053904 full. You may need to reduce memtable and/or cache sizes. Cassandra will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
But, i'm only testing writes, there are no reads on the cluster. Will the writes require so much memory. A large instance has 7.5GB, so by default cassandra allocates about 3.75 GB for the VM.
On Fri, Sep 14, 2012 at 6:58 PM, Robin Verlangen <ro...@us2.nl>> wrote:
Hi Robbit,
I think it's running out of disk space, please verify that (on Linux: df -h ).
Best regards,
Robin Verlangen
Software engineer
W http://www.robinverlangen.nl
E robin@us2.nl<ma...@us2.nl>
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
2012/9/14 rohit reddy <ro...@gmail.com>>
Hi,
I'm facing a problem in Cassandra cluster deployed on EC2 where the node is going down under write load.
I have configured a cluster of 4 Large EC2 nodes with RF of 2.
All nodes are instance storage backed. DISK is RAID0 with 800GB
I'm pumping in write requests at about 4000 writes/sec. One of the node went down under this load. The total data size in each node was not more than 7GB
Got the following WARN messages in the LOG file...
1. setting live ratio to minimum of 1.0 instead of 0.9003153296009601
2. Heap is 0.7515559786053904 full. You may need to reduce memtable and/or cache sizes. Cassandra will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do
this automatically
3. WARN [CompactionExecutor:570] 2012-09-14 11:45:12,024 CompactionTask.java (line 84) insufficient space to compact all requested files
All cassandra settings are default settings.
Do i need to tune anything to support this write rate?
Thanks
Rohit
******************************************************
IMPORTANT: Any information contained in this communication is intended for the use of the named individual or entity. All information contained in this communication is not intended or construed as an offer, solicitation, or a recommendation to purchase any security. Advice, suggestions or views presented in this communication are not necessarily those of Pershing LLC nor do they warrant a complete or accurate statement.
If you are not an intended party to this communication, please notify the sender and delete/destroy any and all copies of this communication. Unintended recipients shall not review, reproduce, disseminate nor disclose any information contained in this communication. Pershing LLC reserves the right to monitor and retain all incoming and outgoing communications as permitted by applicable law.
Email communications may contain viruses or other defects. Pershing LLC does not accept liability nor does it warrant that email communications are virus or defect free.
******************************************************
Re: Is it possible to create a schema before a Cassandra node starts up ?
Posted by aaron morton <aa...@thelastpickle.com>.
> I have a script that needs to set up a schema first before starting up the cassandra node. Is this possible ?
No.
A node must be running to make a DDL change.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 15/09/2012, at 2:05 AM, "Xu, Zaili" <ZX...@pershing.com> wrote:
> Guys
>
> I am pretty new to Cassandra. I have a script that needs to set up a schema first before starting up the cassandra node. Is this possible ? Can I create the schema directly on cassandra storage and then when the node starts up it will pick up the schema ?
>
> Zaili
>
> From: rohit reddy [mailto:rohit.kommareddy@gmail.com]
> Sent: Friday, September 14, 2012 9:50 AM
> To: user@cassandra.apache.org
> Subject: Re: Cassandra node going down
>
> Hi Robin,
>
> I had checked that. Our disk size is about 800GB, and the total data size is not more than 40GB. Even if all the data is stored in one node, this won't happen.
>
> I'll try to see if the disk failed.
>
> Is this anything to do with VM memory?.. cause this logs suggests that..
> Heap is 0.7515559786053904 full. You may need to reduce memtable and/or cache sizes. Cassandra will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
>
> But, i'm only testing writes, there are no reads on the cluster. Will the writes require so much memory. A large instance has 7.5GB, so by default cassandra allocates about 3.75 GB for the VM.
>
>
>
> On Fri, Sep 14, 2012 at 6:58 PM, Robin Verlangen <ro...@us2.nl> wrote:
> Hi Robbit,
>
> I think it's running out of disk space, please verify that (on Linux: df -h ).
>
> Best regards,
>
> Robin Verlangen
> Software engineer
>
> W http://www.robinverlangen.nl
> E robin@us2.nl
>
> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
>
>
>
> 2012/9/14 rohit reddy <ro...@gmail.com>
> Hi,
>
> I'm facing a problem in Cassandra cluster deployed on EC2 where the node is going down under write load.
>
> I have configured a cluster of 4 Large EC2 nodes with RF of 2.
> All nodes are instance storage backed. DISK is RAID0 with 800GB
>
> I'm pumping in write requests at about 4000 writes/sec. One of the node went down under this load. The total data size in each node was not more than 7GB
> Got the following WARN messages in the LOG file...
>
> 1. setting live ratio to minimum of 1.0 instead of 0.9003153296009601
> 2. Heap is 0.7515559786053904 full. You may need to reduce memtable and/or cache sizes. Cassandra will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do
> this automatically
> 3. WARN [CompactionExecutor:570] 2012-09-14 11:45:12,024 CompactionTask.java (line 84) insufficient space to compact all requested files
>
> All cassandra settings are default settings.
> Do i need to tune anything to support this write rate?
>
> Thanks
> Rohit
>
>
>
>
> ******************************************************
> IMPORTANT: Any information contained in this communication is intended for the use of the named individual or entity. All information contained in this communication is not intended or construed as an offer, solicitation, or a recommendation to purchase any security. Advice, suggestions or views presented in this communication are not necessarily those of Pershing LLC nor do they warrant a complete or accurate statement.
>
> If you are not an intended party to this communication, please notify the sender and delete/destroy any and all copies of this communication. Unintended recipients shall not review, reproduce, disseminate nor disclose any information contained in this communication. Pershing LLC reserves the right to monitor and retain all incoming and outgoing communications as permitted by applicable law.
>
> Email communications may contain viruses or other defects. Pershing LLC does not accept liability nor does it warrant that email communications are virus or defect free.
> ******************************************************
Re: Is it possible to create a schema before a Cassandra node starts
up ?
Posted by Rob Coli <rc...@palominodb.com>.
On Fri, Sep 14, 2012 at 7:05 AM, Xu, Zaili <ZX...@pershing.com> wrote:
> I am pretty new to Cassandra. I have a script that needs to set up a schema
> first before starting up the cassandra node. Is this possible ? Can I create
> the schema directly on cassandra storage and then when the node starts up it
> will pick up the schema ?
Aaron gave you the scientific answer, which is that you can't load
schema without starting a node.
However if you :
1) start a node for the first time
2) load schema
3) call nodetool drain so all system keyspace CFs are guaranteed to be
flushed to sstables
4) then, from your script, start that node (or a node with identical
configuration) using the flushed system sstables ("directly on the
storage")
You can "set up a schema before starting up the cassandra node" or
having a cassandra node or cluster running all the time. This might be
useful in for example testing contexts...
=Rob
--
=Robert Coli
AIM>ALK - rcoli@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb