You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Rahul Bhardwaj <ra...@indiamart.com> on 2015/04/30 06:20:34 UTC
calculation of disk size
Hi All,
We are planning to set up a cluster of 5 nodes with RF 3 for write heavy
project, our current database size is around 500 GB. And it is growing at
rate of 15 GB every day. We learnt that cassandra consumes space for
compaction processes, So how can we calculate the amount of disk space we
would require.
Kindly suggest.
Regards:
Rahul Bhardwaj
--
Follow IndiaMART.com <http://www.indiamart.com> for latest updates on this
and more: <https://plus.google.com/+indiamart>
<https://www.facebook.com/IndiaMART> <https://twitter.com/IndiaMART> Mobile
Channel:
<https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=668561641&mt=8>
<https://play.google.com/store/apps/details?id=com.indiamart.m>
<http://m.indiamart.com/>
<https://www.youtube.com/watch?v=DzORNbeSXN8&list=PL2o4J51MqpL0mbue6kzDa6eymLVUXtlR1&index=2>
Watch how IndiaMART Maximiser helped Mr. Khanna expand his business. kyunki Kaam
Yahin Banta Hai <https://www.youtube.com/watch?v=cy1jiNXrzxc>!!!
Re: calculation of disk size
Posted by Rahul Bhardwaj <ra...@indiamart.com>.
Thanks Rahul and Arun
On Thu, Apr 30, 2015 at 3:24 PM, Rahul Neelakantan <ra...@rahul.be> wrote:
> Here is a calculator someone has put together
>
>
> http://btoddb-cass-storage.blogspot.com/2012/01/ive-been-asked-for-spreadsheet-to-help.html?m=1
>
> Rahul
>
> On Apr 30, 2015, at 12:53 AM, arun sirimalla <ar...@gmail.com> wrote:
>
> Hi Rahul,
>
> If you are expecting 15 GB of data per day, here is the calculation.
>
> 1 Day = 15 GB, 1 Month = 450 GB, 1 Year = 5.4 TB, so your raw data size
> for one year is 5.4 TB with replication factor of 3 it would be around 16.2
> TB of data for one year.
>
> Taking compaction into consideration and your use case being write heavy,
> if you go with size tiered compaction. you would need twice the space of
> your raw data.
>
> So you would need around 32-34 TB of disk space.
>
> Reference:
> http://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architecturePlanningDiskCapacity_t.html
>
> Thanks
>
> On Wed, Apr 29, 2015 at 9:20 PM, Rahul Bhardwaj <
> rahul.bhardwaj@indiamart.com> wrote:
>
>> Hi All,
>>
>>
>> We are planning to set up a cluster of 5 nodes with RF 3 for write heavy
>> project, our current database size is around 500 GB. And it is growing at
>> rate of 15 GB every day. We learnt that cassandra consumes space for
>> compaction processes, So how can we calculate the amount of disk space we
>> would require.
>>
>> Kindly suggest.
>>
>>
>>
>> Regards:
>> Rahul Bhardwaj
>>
>>
>> Follow IndiaMART.com <http://www.indiamart.com> for latest updates on
>> this and more: <https://plus.google.com/+indiamart>
>> <https://www.facebook.com/IndiaMART> <https://twitter.com/IndiaMART>
>> Mobile Channel:
>> <https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=668561641&mt=8>
>> <https://play.google.com/store/apps/details?id=com.indiamart.m>
>> <http://m.indiamart.com/>
>>
>> <https://www.youtube.com/watch?v=DzORNbeSXN8&list=PL2o4J51MqpL0mbue6kzDa6eymLVUXtlR1&index=2>
>> Watch how IndiaMART Maximiser helped Mr. Khanna expand his business.
>> kyunki Kaam Yahin Banta Hai <https://www.youtube.com/watch?v=cy1jiNXrzxc>
>> !!!
>
>
>
>
> --
> Arun
> Senior Hadoop/Cassandra Engineer
> Cloudwick
>
> Champion of Big Data (Cloudera)
>
> http://www.cloudera.com/content/dev-center/en/home/champions-of-big-data.html
>
> 2014 Data Impact Award Winner (Cloudera)
>
> http://www.cloudera.com/content/cloudera/en/campaign/data-impact-awards.html
>
>
--
Follow IndiaMART.com <http://www.indiamart.com> for latest updates on this
and more: <https://plus.google.com/+indiamart>
<https://www.facebook.com/IndiaMART> <https://twitter.com/IndiaMART> Mobile
Channel:
<https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=668561641&mt=8>
<https://play.google.com/store/apps/details?id=com.indiamart.m>
<http://m.indiamart.com/>
<https://www.youtube.com/watch?v=DzORNbeSXN8&list=PL2o4J51MqpL0mbue6kzDa6eymLVUXtlR1&index=2>
Watch how IndiaMART Maximiser helped Mr. Khanna expand his business. kyunki Kaam
Yahin Banta Hai <https://www.youtube.com/watch?v=cy1jiNXrzxc>!!!
Re: calculation of disk size
Posted by Rahul Neelakantan <ra...@rahul.be>.
Here is a calculator someone has put together
http://btoddb-cass-storage.blogspot.com/2012/01/ive-been-asked-for-spreadsheet-to-help.html?m=1
Rahul
> On Apr 30, 2015, at 12:53 AM, arun sirimalla <ar...@gmail.com> wrote:
>
> Hi Rahul,
>
> If you are expecting 15 GB of data per day, here is the calculation.
>
> 1 Day = 15 GB, 1 Month = 450 GB, 1 Year = 5.4 TB, so your raw data size for one year is 5.4 TB with replication factor of 3 it would be around 16.2 TB of data for one year.
>
> Taking compaction into consideration and your use case being write heavy, if you go with size tiered compaction. you would need twice the space of your raw data.
>
> So you would need around 32-34 TB of disk space.
>
> Reference: http://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architecturePlanningDiskCapacity_t.html
>
> Thanks
>
>> On Wed, Apr 29, 2015 at 9:20 PM, Rahul Bhardwaj <ra...@indiamart.com> wrote:
>> Hi All,
>>
>>
>> We are planning to set up a cluster of 5 nodes with RF 3 for write heavy project, our current database size is around 500 GB. And it is growing at rate of 15 GB every day. We learnt that cassandra consumes space for compaction processes, So how can we calculate the amount of disk space we would require.
>>
>> Kindly suggest.
>>
>>
>>
>> Regards:
>> Rahul Bhardwaj
>>
>>
>> Follow IndiaMART.com for latest updates on this and more: Mobile Channel:
>>
>> Watch how IndiaMART Maximiser helped Mr. Khanna expand his business. kyunki Kaam Yahin Banta Hai!!!
>
>
>
> --
> Arun
> Senior Hadoop/Cassandra Engineer
> Cloudwick
>
> Champion of Big Data (Cloudera)
> http://www.cloudera.com/content/dev-center/en/home/champions-of-big-data.html
>
> 2014 Data Impact Award Winner (Cloudera)
> http://www.cloudera.com/content/cloudera/en/campaign/data-impact-awards.html
>
Re: calculation of disk size
Posted by arun sirimalla <ar...@gmail.com>.
Hi Rahul,
If you are expecting 15 GB of data per day, here is the calculation.
1 Day = 15 GB, 1 Month = 450 GB, 1 Year = 5.4 TB, so your raw data size for
one year is 5.4 TB with replication factor of 3 it would be around 16.2 TB
of data for one year.
Taking compaction into consideration and your use case being write heavy,
if you go with size tiered compaction. you would need twice the space of
your raw data.
So you would need around 32-34 TB of disk space.
Reference:
http://docs.datastax.com/en/cassandra/2.0/cassandra/architecture/architecturePlanningDiskCapacity_t.html
Thanks
On Wed, Apr 29, 2015 at 9:20 PM, Rahul Bhardwaj <
rahul.bhardwaj@indiamart.com> wrote:
> Hi All,
>
>
> We are planning to set up a cluster of 5 nodes with RF 3 for write heavy
> project, our current database size is around 500 GB. And it is growing at
> rate of 15 GB every day. We learnt that cassandra consumes space for
> compaction processes, So how can we calculate the amount of disk space we
> would require.
>
> Kindly suggest.
>
>
>
> Regards:
> Rahul Bhardwaj
>
>
> Follow IndiaMART.com <http://www.indiamart.com> for latest updates on
> this and more: <https://plus.google.com/+indiamart>
> <https://www.facebook.com/IndiaMART> <https://twitter.com/IndiaMART>
> Mobile Channel:
> <https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=668561641&mt=8>
> <https://play.google.com/store/apps/details?id=com.indiamart.m>
> <http://m.indiamart.com/>
>
> <https://www.youtube.com/watch?v=DzORNbeSXN8&list=PL2o4J51MqpL0mbue6kzDa6eymLVUXtlR1&index=2>
> Watch how IndiaMART Maximiser helped Mr. Khanna expand his business.
> kyunki Kaam Yahin Banta Hai <https://www.youtube.com/watch?v=cy1jiNXrzxc>
> !!!
--
Arun
Senior Hadoop/Cassandra Engineer
Cloudwick
Champion of Big Data (Cloudera)
http://www.cloudera.com/content/dev-center/en/home/champions-of-big-data.html
2014 Data Impact Award Winner (Cloudera)
http://www.cloudera.com/content/cloudera/en/campaign/data-impact-awards.html