You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ahmed Ferdous <Ah...@ze.com> on 2015/08/14 20:06:43 UTC
Configuring Cassandra to limit number of columns to read
Hi Guys,
We have designed a table to have rows with large number of columns (more than 250k). One of my colleagues, mistakenly ran a select on the and that caused the nodes to go out of memory. I was just wondering if there are ways to configure Cassandra 1. To limit number of columns that can be read 2. To gracefully reject a read request if it appears to be consuming a lot of memory. Otherwise, we are leaving too much open to human mistakes.
Cheers,
Ahmed
________________________________
Ahmed Ferdous
Systems Architect
Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com<ma...@ze.com>
________________________________
[2015 ZEMA User Forum]<http://www.ze.com/events/zemauserforum2015/>
ZE PowerGroup Inc.
130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com<http://www.ze.com>
North America: 1-866-944-1469 Europe: 0-800-520-0193 Singapore: 800-130-1609
________________________________
ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and contain confidential and proprietary information. If you are not the intended recipient, be aware that any reading, distribution, disclosure, copying, printing or taking action in reliance upon this e-mail is prohibited. If you have received this in error, please notify us immediately by replying to this e-mail and promptly delete this e-mail and its attachments from your computer system.
Re: Configuring Cassandra to limit number of columns to read
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
Also, are you using Thrift? The terminology you're using, specifically
column keys, suggests you are.
On Mon, Aug 17, 2015 at 5:21 PM Jonathan Haddad <jo...@jonhaddad.com> wrote:
> What version of Cassandra are you running?
>
> On Fri, Aug 14, 2015 at 11:35 AM Ahmed Ferdous <Ah...@ze.com>
> wrote:
>
>> My mistake. I wanted to mean column keys. For each row, we have one
>> column and for this column we have 250k+ column keys. When we are trying to
>> load the list of column keys , Cassandra is having OOM error.
>>
>>
>>
>> Ahmed
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Ahmed Ferdous
>>
>> Systems Architect
>>
>> ZE PowerGroup Inc.
>>
>> Corporate: 604-244-1469
>>
>> *From:* Russell Bradberry [mailto:rbradberry@gmail.com]
>> *Sent:* August-14-15 11:20 AM
>>
>>
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: Configuring Cassandra to limit number of columns to read
>>
>>
>>
>> The idea that you have 250k columns is somewhat of an anti-pattern. In
>> this case you would typically have a few columns and many rows, then just
>> run a select with a limit clause in your partition.
>>
>>
>>
>> *From: *Jonathan Haddad
>> *Reply-To: *<us...@cassandra.apache.org>
>> *Date: *Friday, August 14, 2015 at 2:16 PM
>> *To: *"user@cassandra.apache.org"
>> *Subject: *Re: Configuring Cassandra to limit number of columns to read
>>
>>
>>
>> 250k columns? As in, you have a CREATE TABLE statement that would have
>> over 250K separate, typed fields?
>>
>>
>>
>> On Fri, Aug 14, 2015 at 11:07 AM Ahmed Ferdous <Ah...@ze.com>
>> wrote:
>>
>> Hi Guys,
>>
>>
>>
>> We have designed a table to have rows with large number of columns (more
>> than 250k). One of my colleagues, mistakenly ran a select on the and that
>> caused the nodes to go out of memory. I was just wondering if there are
>> ways to configure Cassandra 1. To limit number of columns that can be read
>> 2. To gracefully reject a read request if it appears to be consuming a lot
>> of memory. Otherwise, we are leaving too much open to human mistakes.
>>
>>
>>
>> Cheers,
>>
>>
>>
>> Ahmed
>>
>>
>> ------------------------------
>>
>> *Ahmed Ferdous*
>> Systems Architect
>> Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com
>> ------------------------------
>>
>> [image: 2015 ZEMA User Forum]
>> <http://www.ze.com/events/zemauserforum2015/>
>>
>> *Z**E PowerGroup Inc.*
>> 130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com
>> North America: 1-866-944-1469 Europe:
>> 0-800-520-0193 Singapore: 800-130-1609
>> ------------------------------
>>
>> ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any
>> attachments are for the exclusive and confidential use of the intended
>> recipient and contain confidential and proprietary information. If you are
>> not the intended recipient, be aware that any reading, distribution,
>> disclosure, copying, printing or taking action in reliance upon this e-mail
>> is prohibited. If you have received this in error, please notify us
>> immediately by replying to this e-mail and promptly delete this e-mail and
>> its attachments from your computer system.
>>
>>
>>
>>
Re: Configuring Cassandra to limit number of columns to read
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
What version of Cassandra are you running?
On Fri, Aug 14, 2015 at 11:35 AM Ahmed Ferdous <Ah...@ze.com> wrote:
> My mistake. I wanted to mean column keys. For each row, we have one column
> and for this column we have 250k+ column keys. When we are trying to load
> the list of column keys , Cassandra is having OOM error.
>
>
>
> Ahmed
>
>
>
>
>
>
>
>
>
> Ahmed Ferdous
>
> Systems Architect
>
> ZE PowerGroup Inc.
>
> Corporate: 604-244-1469
>
> *From:* Russell Bradberry [mailto:rbradberry@gmail.com]
> *Sent:* August-14-15 11:20 AM
>
>
> *To:* user@cassandra.apache.org
> *Subject:* Re: Configuring Cassandra to limit number of columns to read
>
>
>
> The idea that you have 250k columns is somewhat of an anti-pattern. In
> this case you would typically have a few columns and many rows, then just
> run a select with a limit clause in your partition.
>
>
>
> *From: *Jonathan Haddad
> *Reply-To: *<us...@cassandra.apache.org>
> *Date: *Friday, August 14, 2015 at 2:16 PM
> *To: *"user@cassandra.apache.org"
> *Subject: *Re: Configuring Cassandra to limit number of columns to read
>
>
>
> 250k columns? As in, you have a CREATE TABLE statement that would have
> over 250K separate, typed fields?
>
>
>
> On Fri, Aug 14, 2015 at 11:07 AM Ahmed Ferdous <Ah...@ze.com>
> wrote:
>
> Hi Guys,
>
>
>
> We have designed a table to have rows with large number of columns (more
> than 250k). One of my colleagues, mistakenly ran a select on the and that
> caused the nodes to go out of memory. I was just wondering if there are
> ways to configure Cassandra 1. To limit number of columns that can be read
> 2. To gracefully reject a read request if it appears to be consuming a lot
> of memory. Otherwise, we are leaving too much open to human mistakes.
>
>
>
> Cheers,
>
>
>
> Ahmed
>
>
> ------------------------------
>
> *Ahmed Ferdous*
> Systems Architect
> Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com
> ------------------------------
>
> [image: 2015 ZEMA User Forum]
> <http://www.ze.com/events/zemauserforum2015/>
>
> *Z**E PowerGroup Inc.*
> 130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com
> North America: 1-866-944-1469 Europe: 0-800-520-0193 Singapore:
> 800-130-1609
> ------------------------------
>
> ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any attachments
> are for the exclusive and confidential use of the intended recipient and
> contain confidential and proprietary information. If you are not the
> intended recipient, be aware that any reading, distribution, disclosure,
> copying, printing or taking action in reliance upon this e-mail is
> prohibited. If you have received this in error, please notify us
> immediately by replying to this e-mail and promptly delete this e-mail and
> its attachments from your computer system.
>
>
>
>
RE: Configuring Cassandra to limit number of columns to read
Posted by Ahmed Ferdous <Ah...@ze.com>.
My mistake. I wanted to mean column keys. For each row, we have one column and for this column we have 250k+ column keys. When we are trying to load the list of column keys , Cassandra is having OOM error.
Ahmed
Ahmed Ferdous
Systems Architect
ZE PowerGroup Inc.
Corporate: 604-244-1469
From: Russell Bradberry [mailto:rbradberry@gmail.com]
Sent: August-14-15 11:20 AM
To: user@cassandra.apache.org
Subject: Re: Configuring Cassandra to limit number of columns to read
The idea that you have 250k columns is somewhat of an anti-pattern. In this case you would typically have a few columns and many rows, then just run a select with a limit clause in your partition.
From: Jonathan Haddad
Reply-To: <us...@cassandra.apache.org>>
Date: Friday, August 14, 2015 at 2:16 PM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>"
Subject: Re: Configuring Cassandra to limit number of columns to read
250k columns? As in, you have a CREATE TABLE statement that would have over 250K separate, typed fields?
On Fri, Aug 14, 2015 at 11:07 AM Ahmed Ferdous <Ah...@ze.com>> wrote:
Hi Guys,
We have designed a table to have rows with large number of columns (more than 250k). One of my colleagues, mistakenly ran a select on the and that caused the nodes to go out of memory. I was just wondering if there are ways to configure Cassandra 1. To limit number of columns that can be read 2. To gracefully reject a read request if it appears to be consuming a lot of memory. Otherwise, we are leaving too much open to human mistakes.
Cheers,
Ahmed
________________________________
Ahmed Ferdous
Systems Architect
Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com<ma...@ze.com>
________________________________
[2015 ZEMA User Forum]<http://www.ze.com/events/zemauserforum2015/>
ZE PowerGroup Inc.
130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com<http://www.ze.com>
North America: 1-866-944-1469 Europe: 0-800-520-0193 Singapore: 800-130-1609
________________________________
ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and contain confidential and proprietary information. If you are not the intended recipient, be aware that any reading, distribution, disclosure, copying, printing or taking action in reliance upon this e-mail is prohibited. If you have received this in error, please notify us immediately by replying to this e-mail and promptly delete this e-mail and its attachments from your computer system.
Re: Configuring Cassandra to limit number of columns to read
Posted by Russell Bradberry <rb...@gmail.com>.
The idea that you have 250k columns is somewhat of an anti-pattern. In this case you would typically have a few columns and many rows, then just run a select with a limit clause in your partition.
From: Jonathan Haddad
Reply-To: <us...@cassandra.apache.org>
Date: Friday, August 14, 2015 at 2:16 PM
To: "user@cassandra.apache.org"
Subject: Re: Configuring Cassandra to limit number of columns to read
250k columns? As in, you have a CREATE TABLE statement that would have over 250K separate, typed fields?
On Fri, Aug 14, 2015 at 11:07 AM Ahmed Ferdous <Ah...@ze.com> wrote:
Hi Guys,
We have designed a table to have rows with large number of columns (more than 250k). One of my colleagues, mistakenly ran a select on the and that caused the nodes to go out of memory. I was just wondering if there are ways to configure Cassandra 1. To limit number of columns that can be read 2. To gracefully reject a read request if it appears to be consuming a lot of memory. Otherwise, we are leaving too much open to human mistakes.
Cheers,
Ahmed
Ahmed Ferdous
Systems Architect
Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com
ZE PowerGroup Inc.
130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com
North America: 1-866-944-1469 Europe: 0-800-520-0193 Singapore: 800-130-1609
ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and contain confidential and proprietary information. If you are not the intended recipient, be aware that any reading, distribution, disclosure, copying, printing or taking action in reliance upon this e-mail is prohibited. If you have received this in error, please notify us immediately by replying to this e-mail and promptly delete this e-mail and its attachments from your computer system.
Re: Configuring Cassandra to limit number of columns to read
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
250k columns? As in, you have a CREATE TABLE statement that would have
over 250K separate, typed fields?
On Fri, Aug 14, 2015 at 11:07 AM Ahmed Ferdous <Ah...@ze.com> wrote:
> Hi Guys,
>
>
>
> We have designed a table to have rows with large number of columns (more
> than 250k). One of my colleagues, mistakenly ran a select on the and that
> caused the nodes to go out of memory. I was just wondering if there are
> ways to configure Cassandra 1. To limit number of columns that can be read
> 2. To gracefully reject a read request if it appears to be consuming a lot
> of memory. Otherwise, we are leaving too much open to human mistakes.
>
>
>
> Cheers,
>
>
>
> Ahmed
>
>
>
> ------------------------------
>
> *Ahmed Ferdous*
> Systems Architect
> Corporate: 604-244-1469 Email: Ahmed.Ferdous@ze.com
> ------------------------------
> [image: 2015 ZEMA User Forum]
> <http://www.ze.com/events/zemauserforum2015/>
>
> *Z**E PowerGroup Inc.*
> 130 - 5920 No. Two Road, Richmond, BC, Canada V7C 4R9 Web: www.ze.com
> North America: 1-866-944-1469 Europe: 0-800-520-0193 Singapore:
> 800-130-1609
> ------------------------------
>
> ZE PowerGroup Inc. Confidentiality Notice: This e-mail and any attachments
> are for the exclusive and confidential use of the intended recipient and
> contain confidential and proprietary information. If you are not the
> intended recipient, be aware that any reading, distribution, disclosure,
> copying, printing or taking action in reliance upon this e-mail is
> prohibited. If you have received this in error, please notify us
> immediately by replying to this e-mail and promptly delete this e-mail and
> its attachments from your computer system.
>
>