You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by sha p <sh...@gmail.com> on 2018/09/05 13:21:01 UTC
Regarding migrating data from Oracle to Cassandra.migrate data from
Oracle to Cassandra.
> Hi all ,
> Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
> Please help me giving your valuable guidance.
> 1) Can it be done using open source Cassandra.
> 2) Where should I start data model from?
> 3) I should use java, what kind of jar/libs/tools I need use ?
> 4) How I decide the size of cluster , please provide some sample
> guidelines.
> 5) this should be in production , so what kind of things i should take
> care for better support or debugging tomorrow?
> 6) Please provide some good books /links which can help me in this task.
>
>
> Thanks in advance.
> Highly appreciated your every amal help.
>
> Regards,
> Shyam
>
How to build a auditing process in data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Hi All,
Thanks a lot for all your previous suggestions.
As you suggested I am using Spark SQL for data migration i.e oracle ->
cassandra.
But I have two challenges
1) How should I handle failures , how that logic should be build?
2) How to do/handle any incremental migration?
3) How to do auditing ?
Please give me some ideas around these.
Regards,
Shyam
On Fri, Sep 14, 2018 at 9:42 PM sha p <sh...@gmail.com> wrote:
> Hi All,
> Is there any free Cassandra IDEs available? If so please share?
>
> what is advantages of enterprise edition IDEs?
>
> Regards,
> Shyam
>
> On Wed, Sep 5, 2018 at 6:51 PM sha p <sh...@gmail.com> wrote:
>
>>
>> Hi all ,
>>> Me new to Cassandra , i was asked to migrate data from Oracle to
>>> Cassandra.
>>> Please help me giving your valuable guidance.
>>> 1) Can it be done using open source Cassandra.
>>> 2) Where should I start data model from?
>>> 3) I should use java, what kind of jar/libs/tools I need use ?
>>> 4) How I decide the size of cluster , please provide some sample
>>> guidelines.
>>> 5) this should be in production , so what kind of things i should take
>>> care for better support or debugging tomorrow?
>>> 6) Please provide some good books /links which can help me in this task.
>>>
>>>
>>> Thanks in advance.
>>> Highly appreciated your every amal help.
>>>
>>> Regards,
>>> Shyam
>>>
>>
Re: Error during truncate: Cannot achieve consistency level ALL , how
to fix it
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hello Shyam,
I think Jonathan understood the meaning of 'RF'. He is suggesting you to
look at all your keyspaces strategy/RF for the system_auth table and make
sure to use the right replication factor (and probably
a NetworkTopologyStrategy).
This might help: cqlsh -e "DESCRIBE KEYSPACE system_auth;"
Or to check them all: cqlsh -e "DESCRIBE KEYSPACES;"
I don't know what's wrong exactly, but your application is truncating with
a consistency level of 'ALL', meaning all the replicas must be up for your
application to work.
Le mer. 19 sept. 2018 à 14:30, sha p <sh...@gmail.com> a écrit :
> RF is replication factor. Sorry for confusing
>
> On 19 Sep 2018 5:45 p.m., "Jonathan Baynes" <Jo...@tradeweb.com>
> wrote:
>
> What RF is your system_auth keyspace?
>
>
>
> If its one, match it to the user keyspace, and restart the node.
>
>
>
> *From:* sha p [mailto:shatesttest@gmail.com]
> *Sent:* 19 September 2018 11:49
> *To:* user@cassandra.apache.org
> *Subject:* Error during truncate: Cannot achieve consistency level ALL ,
> how to fix it
>
>
>
> Hi All,
>
> I am new to Cassandra. Following below link
>
>
>
>
> https://grokonez.com/spring-framework/spring-data/start-spring-data-cassandra-springboot#III_Sourcecode
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__grokonez.com_spring-2Dframework_spring-2Ddata_start-2Dspring-2Ddata-2Dcassandra-2Dspringboot-23III-5FSourcecode&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=j2h9K99DL76u4mWIOBsOk5lr41V4bADBYHYktY6e154&s=LzDHllUpsVHGyHiYwEkCXDyapbFOatkyPSmPsiFaCFo&e=>
>
>
>
>
>
> I have three node cluster , keyspace set with RF = 2 , but when I run this
> application from above source code bellow error is thrown
>
> Caused by: com.datastax.driver.core.exceptions.TruncateException: Error
> during truncate: Cannot achieve consistency level ALL """ ....
>
>
>
>
>
> What wrong i am doing here ..How to fix it ? Plz help me.
>
>
>
> Regards,
>
> Shyam
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contactus@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>
>
>
Re: Error during truncate: Cannot achieve consistency level ALL , how
to fix it
Posted by sha p <sh...@gmail.com>.
RF is replication factor. Sorry for confusing
On 19 Sep 2018 5:45 p.m., "Jonathan Baynes" <Jo...@tradeweb.com>
wrote:
What RF is your system_auth keyspace?
If its one, match it to the user keyspace, and restart the node.
*From:* sha p [mailto:shatesttest@gmail.com]
*Sent:* 19 September 2018 11:49
*To:* user@cassandra.apache.org
*Subject:* Error during truncate: Cannot achieve consistency level ALL ,
how to fix it
Hi All,
I am new to Cassandra. Following below link
https://grokonez.com/spring-framework/spring-data/start-spring-data-cassandra-springboot#III_Sourcecode
<https://urldefense.proofpoint.com/v2/url?u=https-3A__grokonez.com_spring-2Dframework_spring-2Ddata_start-2Dspring-2Ddata-2Dcassandra-2Dspringboot-23III-5FSourcecode&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=j2h9K99DL76u4mWIOBsOk5lr41V4bADBYHYktY6e154&s=LzDHllUpsVHGyHiYwEkCXDyapbFOatkyPSmPsiFaCFo&e=>
I have three node cluster , keyspace set with RF = 2 , but when I run this
application from above source code bellow error is thrown
Caused by: com.datastax.driver.core.exceptions.TruncateException: Error
during truncate: Cannot achieve consistency level ALL """ ....
What wrong i am doing here ..How to fix it ? Plz help me.
Regards,
Shyam
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy it. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden. Tradeweb reserves the right to monitor all e-mail
communications through its networks. If you do not wish to receive
marketing emails about our products / services, please let us know by
contacting us, either by email at contactus@tradeweb.com or by writing to
us at the registered office of Tradeweb in the UK, which is: Tradeweb
Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
RE: Error during truncate: Cannot achieve consistency level ALL ,
how to fix it
Posted by Jonathan Baynes <Jo...@tradeweb.com>.
What RF is your system_auth keyspace?
If its one, match it to the user keyspace, and restart the node.
From: sha p [mailto:shatesttest@gmail.com]
Sent: 19 September 2018 11:49
To: user@cassandra.apache.org
Subject: Error during truncate: Cannot achieve consistency level ALL , how to fix it
Hi All,
I am new to Cassandra. Following below link
https://grokonez.com/spring-framework/spring-data/start-spring-data-cassandra-springboot#III_Sourcecode<https://urldefense.proofpoint.com/v2/url?u=https-3A__grokonez.com_spring-2Dframework_spring-2Ddata_start-2Dspring-2Ddata-2Dcassandra-2Dspringboot-23III-5FSourcecode&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=j2h9K99DL76u4mWIOBsOk5lr41V4bADBYHYktY6e154&s=LzDHllUpsVHGyHiYwEkCXDyapbFOatkyPSmPsiFaCFo&e=>
I have three node cluster , keyspace set with RF = 2 , but when I run this application from above source code bellow error is thrown
Caused by: com.datastax.driver.core.exceptions.TruncateException: Error during truncate: Cannot achieve consistency level ALL """ ....
What wrong i am doing here ..How to fix it ? Plz help me.
Regards,
Shyam
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
Error during truncate: Cannot achieve consistency level ALL , how to
fix it
Posted by sha p <sh...@gmail.com>.
>
> Hi All,
>>> I am new to Cassandra. Following below link
>>>
>>>
>>> https://grokonez.com/spring-framework/spring-data/start-spring-data-cassandra-springboot#III_Sourcecode
>>>
>>>
>>> I have three node cluster , keyspace set with RF = 2 , but when I run
>>> this application from above source code bellow error is thrown
>>> Caused by: com.datastax.driver.core.exceptions.TruncateException: Error
>>> during truncate: Cannot achieve consistency level ALL """ ....
>>>
>>>
>>> What wrong i am doing here ..How to fix it ? Plz help me.
>>>
>>> Regards,
>>> Shyam
>>>
>>
Re: Regarding migrating data from Oracle to Cassandra.migrate data
from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Hi All,
Is there any free Cassandra IDEs available? If so please share?
what is advantages of enterprise edition IDEs?
Regards,
Shyam
On Wed, Sep 5, 2018 at 6:51 PM sha p <sh...@gmail.com> wrote:
>
> Hi all ,
>> Me new to Cassandra , i was asked to migrate data from Oracle to
>> Cassandra.
>> Please help me giving your valuable guidance.
>> 1) Can it be done using open source Cassandra.
>> 2) Where should I start data model from?
>> 3) I should use java, what kind of jar/libs/tools I need use ?
>> 4) How I decide the size of cluster , please provide some sample
>> guidelines.
>> 5) this should be in production , so what kind of things i should take
>> care for better support or debugging tomorrow?
>> 6) Please provide some good books /links which can help me in this task.
>>
>>
>> Thanks in advance.
>> Highly appreciated your every amal help.
>>
>> Regards,
>> Shyam
>>
>
Re: [EXTERNAL] Regarding migrating data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by Rahul Singh <ra...@gmail.com>.
The biggest issue you’ll have is that “migration” from a relational to Cassandra is not a 1 to 1. The schemas will have to change.
DSE has other technology that is a little more useful - such as Spark / Spark SQL / Solr that is built in which helps meet the needs which Oracle was previously providing.
Rahul Singh
Chief Executive Officer
m 202.905.2818
Anant Corporation
1010 Wisconsin Ave NW, Suite 250
Washington, D.C. 20007
We build and manage digital business technology platforms.
On Sep 5, 2018, 10:47 AM -0500, Jeff Jirsa <jj...@gmail.com>, wrote:
> All of Sean's points are good, a few more:
> - Apache Cassandra (free, open source, official) is usually sufficient. DSE may be faster, but really it's about whether or not you're willing to pay for support. If you're trying to stop paying Oracle, I suspect you'd probably not want to start paying someone else - try the free version first, and you can look for proprietary options after that.
> - http://shop.oreilly.com/product/0636920043041.do is relatively recent and mostly pretty good
> - Ask a lot of questions, use this list, but try things out first so people have a way to point you in the right direction.
>
>
>
> > On Wed, Sep 5, 2018 at 7:58 AM Durity, Sean R <SE...@homedepot.com> wrote:
> > > 3 starting points:
> > > - DO NOT migrate your tables as they are in Oracle to Cassandra. In most cases, you need a different model for Cassandra
> > > - DO take the (free) DataStax Academy courses to learn much more about Cassandra as you dive in. It is a systematic and bite-size approach to learning all things Cassandra (and eventually, DataStax Enterprise, should you go that way). However, open source Cassandra is fine as a data platform. DSE gives you more options for data models, better administration and monitoring tools, support, etc. It all depends on what you need/want to build/can afford
> > > - Cluster sizing depends on your goals for the data platform. Do you need lots of storage, lots of throughput, high availability, low latency, workload separation, etc.? A couple guidelines – use at least 3 nodes per data center (DC) and at least 2 DCs for availability. Use SSDs for storage and keep node size 3 TB or less for reasonable administration. If six nodes are too many – you probably don’t need Cassandra. If you can define what you need your data platform to deliver, then you can start a sizing discussion. The good thing is, you can always scale (as long as the data model is good).
> > >
> > >
> > > Sean Durity
> > >
> > > From: sha p <sh...@gmail.com>
> > > Sent: Wednesday, September 05, 2018 9:21 AM
> > > To: user@cassandra.apache.org
> > > Subject: [EXTERNAL] Regarding migrating data from Oracle to Cassandra.migrate data from Oracle to Cassandra.
> > >
> > >
> > > > quote_type
> > > > Hi all ,
> > > > Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
> > > > Please help me giving your valuable guidance.
> > > > 1) Can it be done using open source Cassandra.
> > > > 2) Where should I start data model from?
> > > > 3) I should use java, what kind of jar/libs/tools I need use ?
> > > > 4) How I decide the size of cluster , please provide some sample guidelines.
> > > > 5) this should be in production , so what kind of things i should take care for better support or debugging tomorrow?
> > > > 6) Please provide some good books /links which can help me in this task.
> > > >
> > > >
> > > > Thanks in advance.
> > > > Highly appreciated your every amal help.
> > > >
> > > > Regards,
> > > > Shyam
> > >
> > >
> > > The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: [EXTERNAL] Regarding migrating data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Thank you all very much.
Migration is due to Oracle not scaling as expected.
Sure I will be posting my queries and doubts time to time for your guidance.
Thank you.
Shyam
On Wed, 5 Sep 2018, 21:28 Rahul Singh, <ra...@gmail.com> wrote:
> Look here for some “migration” or data modeling articles.
>
> *https://anant.github.io/awesome-cassandra/*
> <https://anant.github.io/awesome-cassandra/>
>
> Rahul Singh
> Chief Executive Officer
> m 202.905.2818
>
> Anant Corporation
> 1010 Wisconsin Ave NW, Suite 250
> Washington, D.C. 20007
>
> We build and manage digital business technology platforms.
> On Sep 5, 2018, 10:47 AM -0500, Jeff Jirsa <jj...@gmail.com>, wrote:
>
> All of Sean's points are good, a few more:
> - Apache Cassandra (free, open source, official) is usually sufficient.
> DSE may be faster, but really it's about whether or not you're willing to
> pay for support. If you're trying to stop paying Oracle, I suspect you'd
> probably not want to start paying someone else - try the free version
> first, and you can look for proprietary options after that.
> - http://shop.oreilly.com/product/0636920043041.do is relatively recent
> and mostly pretty good
> - Ask a lot of questions, use this list, but try things out first so
> people have a way to point you in the right direction.
>
>
>
> On Wed, Sep 5, 2018 at 7:58 AM Durity, Sean R <SE...@homedepot.com>
> wrote:
>
>> 3 starting points:
>>
>> - DO NOT migrate your tables as they are in Oracle to
>> Cassandra. In most cases, you need a different model for Cassandra
>>
>> - DO take the (free) DataStax Academy courses to learn much
>> more about Cassandra as you dive in. It is a systematic and bite-size
>> approach to learning all things Cassandra (and eventually, DataStax
>> Enterprise, should you go that way). However, open source Cassandra is fine
>> as a data platform. DSE gives you more options for data models, better
>> administration and monitoring tools, support, etc. It all depends on what
>> you need/want to build/can afford
>>
>> - Cluster sizing depends on your goals for the data platform.
>> Do you need lots of storage, lots of throughput, high availability, low
>> latency, workload separation, etc.? A couple guidelines – use at least 3
>> nodes per data center (DC) and at least 2 DCs for availability. Use SSDs
>> for storage and keep node size 3 TB or less for reasonable administration.
>> If six nodes are too many – you probably don’t need Cassandra. If you can
>> define what you need your data platform to deliver, then you can start a
>> sizing discussion. The good thing is, you can always scale (as long as the
>> data model is good).
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> *From:* sha p <sh...@gmail.com>
>> *Sent:* Wednesday, September 05, 2018 9:21 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Regarding migrating data from Oracle to
>> Cassandra.migrate data from Oracle to Cassandra.
>>
>>
>>
>>
>>
>> Hi all ,
>>
>> Me new to Cassandra , i was asked to migrate data from Oracle to
>> Cassandra.
>>
>> Please help me giving your valuable guidance.
>>
>> 1) Can it be done using open source Cassandra.
>>
>> 2) Where should I start data model from?
>>
>> 3) I should use java, what kind of jar/libs/tools I need use ?
>>
>> 4) How I decide the size of cluster , please provide some sample
>> guidelines.
>>
>> 5) this should be in production , so what kind of things i should take
>> care for better support or debugging tomorrow?
>>
>> 6) Please provide some good books /links which can help me in this task.
>>
>>
>>
>>
>>
>> Thanks in advance.
>>
>> Highly appreciated your every amal help.
>>
>>
>>
>> Regards,
>>
>> Shyam
>>
>>
>> ------------------------------
>>
>> The information in this Internet Email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this Email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful. When addressed
>> to our clients any opinions or advice contained in this Email are subject
>> to the terms and conditions expressed in any applicable governing The Home
>> Depot terms of business or client engagement letter. The Home Depot
>> disclaims all responsibility and liability for the accuracy and content of
>> this attachment and for any damages or losses arising from any
>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>> items of a destructive nature, which may be contained in this attachment
>> and shall not be liable for direct, indirect, consequential or special
>> damages in connection with this e-mail message or its attachment.
>>
>
Re: [EXTERNAL] Regarding migrating data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by Rahul Singh <ra...@gmail.com>.
Look here for some “migration” or data modeling articles.
https://anant.github.io/awesome-cassandra/
Rahul Singh
Chief Executive Officer
m 202.905.2818
Anant Corporation
1010 Wisconsin Ave NW, Suite 250
Washington, D.C. 20007
We build and manage digital business technology platforms.
On Sep 5, 2018, 10:47 AM -0500, Jeff Jirsa <jj...@gmail.com>, wrote:
> All of Sean's points are good, a few more:
> - Apache Cassandra (free, open source, official) is usually sufficient. DSE may be faster, but really it's about whether or not you're willing to pay for support. If you're trying to stop paying Oracle, I suspect you'd probably not want to start paying someone else - try the free version first, and you can look for proprietary options after that.
> - http://shop.oreilly.com/product/0636920043041.do is relatively recent and mostly pretty good
> - Ask a lot of questions, use this list, but try things out first so people have a way to point you in the right direction.
>
>
>
> > On Wed, Sep 5, 2018 at 7:58 AM Durity, Sean R <SE...@homedepot.com> wrote:
> > > 3 starting points:
> > > - DO NOT migrate your tables as they are in Oracle to Cassandra. In most cases, you need a different model for Cassandra
> > > - DO take the (free) DataStax Academy courses to learn much more about Cassandra as you dive in. It is a systematic and bite-size approach to learning all things Cassandra (and eventually, DataStax Enterprise, should you go that way). However, open source Cassandra is fine as a data platform. DSE gives you more options for data models, better administration and monitoring tools, support, etc. It all depends on what you need/want to build/can afford
> > > - Cluster sizing depends on your goals for the data platform. Do you need lots of storage, lots of throughput, high availability, low latency, workload separation, etc.? A couple guidelines – use at least 3 nodes per data center (DC) and at least 2 DCs for availability. Use SSDs for storage and keep node size 3 TB or less for reasonable administration. If six nodes are too many – you probably don’t need Cassandra. If you can define what you need your data platform to deliver, then you can start a sizing discussion. The good thing is, you can always scale (as long as the data model is good).
> > >
> > >
> > > Sean Durity
> > >
> > > From: sha p <sh...@gmail.com>
> > > Sent: Wednesday, September 05, 2018 9:21 AM
> > > To: user@cassandra.apache.org
> > > Subject: [EXTERNAL] Regarding migrating data from Oracle to Cassandra.migrate data from Oracle to Cassandra.
> > >
> > >
> > > > quote_type
> > > > Hi all ,
> > > > Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
> > > > Please help me giving your valuable guidance.
> > > > 1) Can it be done using open source Cassandra.
> > > > 2) Where should I start data model from?
> > > > 3) I should use java, what kind of jar/libs/tools I need use ?
> > > > 4) How I decide the size of cluster , please provide some sample guidelines.
> > > > 5) this should be in production , so what kind of things i should take care for better support or debugging tomorrow?
> > > > 6) Please provide some good books /links which can help me in this task.
> > > >
> > > >
> > > > Thanks in advance.
> > > > Highly appreciated your every amal help.
> > > >
> > > > Regards,
> > > > Shyam
> > >
> > >
> > > The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: [EXTERNAL] Regarding migrating data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by Jeff Jirsa <jj...@gmail.com>.
All of Sean's points are good, a few more:
- Apache Cassandra (free, open source, official) is usually sufficient. DSE
may be faster, but really it's about whether or not you're willing to pay
for support. If you're trying to stop paying Oracle, I suspect you'd
probably not want to start paying someone else - try the free version
first, and you can look for proprietary options after that.
- http://shop.oreilly.com/product/0636920043041.do is relatively recent and
mostly pretty good
- Ask a lot of questions, use this list, but try things out first so people
have a way to point you in the right direction.
On Wed, Sep 5, 2018 at 7:58 AM Durity, Sean R <SE...@homedepot.com>
wrote:
> 3 starting points:
>
> - DO NOT migrate your tables as they are in Oracle to Cassandra.
> In most cases, you need a different model for Cassandra
>
> - DO take the (free) DataStax Academy courses to learn much more
> about Cassandra as you dive in. It is a systematic and bite-size approach
> to learning all things Cassandra (and eventually, DataStax Enterprise,
> should you go that way). However, open source Cassandra is fine as a data
> platform. DSE gives you more options for data models, better administration
> and monitoring tools, support, etc. It all depends on what you need/want to
> build/can afford
>
> - Cluster sizing depends on your goals for the data platform. Do
> you need lots of storage, lots of throughput, high availability, low
> latency, workload separation, etc.? A couple guidelines – use at least 3
> nodes per data center (DC) and at least 2 DCs for availability. Use SSDs
> for storage and keep node size 3 TB or less for reasonable administration.
> If six nodes are too many – you probably don’t need Cassandra. If you can
> define what you need your data platform to deliver, then you can start a
> sizing discussion. The good thing is, you can always scale (as long as the
> data model is good).
>
>
>
>
>
> Sean Durity
>
>
>
> *From:* sha p <sh...@gmail.com>
> *Sent:* Wednesday, September 05, 2018 9:21 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Regarding migrating data from Oracle to
> Cassandra.migrate data from Oracle to Cassandra.
>
>
>
>
>
> Hi all ,
>
> Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
>
> Please help me giving your valuable guidance.
>
> 1) Can it be done using open source Cassandra.
>
> 2) Where should I start data model from?
>
> 3) I should use java, what kind of jar/libs/tools I need use ?
>
> 4) How I decide the size of cluster , please provide some sample
> guidelines.
>
> 5) this should be in production , so what kind of things i should take
> care for better support or debugging tomorrow?
>
> 6) Please provide some good books /links which can help me in this task.
>
>
>
>
>
> Thanks in advance.
>
> Highly appreciated your every amal help.
>
>
>
> Regards,
>
> Shyam
>
>
> ------------------------------
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>
RE: [EXTERNAL] Regarding migrating data from Oracle to
Cassandra.migrate data from Oracle to Cassandra.
Posted by "Durity, Sean R" <SE...@homedepot.com>.
3 starting points:
- DO NOT migrate your tables as they are in Oracle to Cassandra. In most cases, you need a different model for Cassandra
- DO take the (free) DataStax Academy courses to learn much more about Cassandra as you dive in. It is a systematic and bite-size approach to learning all things Cassandra (and eventually, DataStax Enterprise, should you go that way). However, open source Cassandra is fine as a data platform. DSE gives you more options for data models, better administration and monitoring tools, support, etc. It all depends on what you need/want to build/can afford
- Cluster sizing depends on your goals for the data platform. Do you need lots of storage, lots of throughput, high availability, low latency, workload separation, etc.? A couple guidelines – use at least 3 nodes per data center (DC) and at least 2 DCs for availability. Use SSDs for storage and keep node size 3 TB or less for reasonable administration. If six nodes are too many – you probably don’t need Cassandra. If you can define what you need your data platform to deliver, then you can start a sizing discussion. The good thing is, you can always scale (as long as the data model is good).
Sean Durity
From: sha p <sh...@gmail.com>
Sent: Wednesday, September 05, 2018 9:21 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Regarding migrating data from Oracle to Cassandra.migrate data from Oracle to Cassandra.
Hi all ,
Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
Please help me giving your valuable guidance.
1) Can it be done using open source Cassandra.
2) Where should I start data model from?
3) I should use java, what kind of jar/libs/tools I need use ?
4) How I decide the size of cluster , please provide some sample guidelines.
5) this should be in production , so what kind of things i should take care for better support or debugging tomorrow?
6) Please provide some good books /links which can help me in this task.
Thanks in advance.
Highly appreciated your every amal help.
Regards,
Shyam
________________________________
The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: Regarding migrating data from Oracle to Cassandra.migrate data
from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Thanks a lot Rahul sir,
Where can I find more information ? any blog or links on that ?
On Mon, Sep 10, 2018 at 6:52 PM Rahul Singh <ra...@gmail.com>
wrote:
> Look into Kafka Connect. It does tracking internally in a topic. Works
> better going from relational to Cassandra.
>
> Still won’t fix your potential data model issue related to skew and wide
> partitions.
>
> Rahul Singh
> Chief Executive Officer
> m 202.905.2818
>
> Anant Corporation
> 1010 Wisconsin Ave NW, Suite 250
> Washington, D.C. 20007
>
> We build and manage digital business technology platforms.
> On Sep 6, 2018, 9:20 AM -0400, sha p , wrote:
>
> Thank you Jeff.
> While migration , how can test/validate against Cassandra particularly i
> am going for "parallel run". Any sample strategy?
>
>
> Regards,
> Shyam
>
> On Thu, 6 Sep 2018, 09:48 Jeff Jirsa, <jj...@gmail.com> wrote:
>
>> It very much depends on your application. You'll PROBABLY want to double
>> write for some period of time - start writes to both Cassandra and Oracle,
>> and then ensure they're both in sync. Once you're sure they're both in
>> sync, move your reads from Oracle to Cassandra.
>>
>>
>>
>> On Wed, Sep 5, 2018 at 8:58 PM sha p <sh...@gmail.com> wrote:
>>
>>> Hi all,
>>> Sir how should I keep track of the data which is moved to Cassandra ,
>>> what are the best strategies available?
>>>
>>> Regards,
>>> Shyam
>>>
>>> On Wed, 5 Sep 2018, 18:51 sha p, <sh...@gmail.com> wrote:
>>>
>>>>
>>>> Hi all ,
>>>>> Me new to Cassandra , i was asked to migrate data from Oracle to
>>>>> Cassandra.
>>>>> Please help me giving your valuable guidance.
>>>>> 1) Can it be done using open source Cassandra.
>>>>> 2) Where should I start data model from?
>>>>> 3) I should use java, what kind of jar/libs/tools I need use ?
>>>>> 4) How I decide the size of cluster , please provide some sample
>>>>> guidelines.
>>>>> 5) this should be in production , so what kind of things i should take
>>>>> care for better support or debugging tomorrow?
>>>>> 6) Please provide some good books /links which can help me in this
>>>>> task.
>>>>>
>>>>>
>>>>> Thanks in advance.
>>>>> Highly appreciated your every amal help.
>>>>>
>>>>> Regards,
>>>>> Shyam
>>>>>
>>>>
Re: Regarding migrating data from Oracle to Cassandra.migrate
data from Oracle to Cassandra.
Posted by Rahul Singh <ra...@gmail.com>.
Look into Kafka Connect. It does tracking internally in a topic. Works better going from relational to Cassandra.
Still won’t fix your potential data model issue related to skew and wide partitions.
Rahul Singh
Chief Executive Officer
m 202.905.2818
Anant Corporation
1010 Wisconsin Ave NW, Suite 250
Washington, D.C. 20007
We build and manage digital business technology platforms.
On Sep 6, 2018, 9:20 AM -0400, sha p , wrote:
> Thank you Jeff.
> While migration , how can test/validate against Cassandra particularly i am going for "parallel run". Any sample strategy?
>
>
> Regards,
> Shyam
>
> > On Thu, 6 Sep 2018, 09:48 Jeff Jirsa, <jj...@gmail.com> wrote:
> > > It very much depends on your application. You'll PROBABLY want to double write for some period of time - start writes to both Cassandra and Oracle, and then ensure they're both in sync. Once you're sure they're both in sync, move your reads from Oracle to Cassandra.
> > >
> > >
> > >
> > > > On Wed, Sep 5, 2018 at 8:58 PM sha p <sh...@gmail.com> wrote:
> > > > > Hi all,
> > > > > Sir how should I keep track of the data which is moved to Cassandra , what are the best strategies available?
> > > > >
> > > > > Regards,
> > > > > Shyam
> > > > >
> > > > > > On Wed, 5 Sep 2018, 18:51 sha p, <sh...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > Hi all ,
> > > > > > > > > Me new to Cassandra , i was asked to migrate data from Oracle to Cassandra.
> > > > > > > > > Please help me giving your valuable guidance.
> > > > > > > > > 1) Can it be done using open source Cassandra.
> > > > > > > > > 2) Where should I start data model from?
> > > > > > > > > 3) I should use java, what kind of jar/libs/tools I need use ?
> > > > > > > > > 4) How I decide the size of cluster , please provide some sample guidelines.
> > > > > > > > > 5) this should be in production , so what kind of things i should take care for better support or debugging tomorrow?
> > > > > > > > > 6) Please provide some good books /links which can help me in this task.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks in advance.
> > > > > > > > > Highly appreciated your every amal help.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Shyam
Re: Regarding migrating data from Oracle to Cassandra.migrate data
from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Thank you Jeff.
While migration , how can test/validate against Cassandra particularly i am
going for "parallel run". Any sample strategy?
Regards,
Shyam
On Thu, 6 Sep 2018, 09:48 Jeff Jirsa, <jj...@gmail.com> wrote:
> It very much depends on your application. You'll PROBABLY want to double
> write for some period of time - start writes to both Cassandra and Oracle,
> and then ensure they're both in sync. Once you're sure they're both in
> sync, move your reads from Oracle to Cassandra.
>
>
>
> On Wed, Sep 5, 2018 at 8:58 PM sha p <sh...@gmail.com> wrote:
>
>> Hi all,
>> Sir how should I keep track of the data which is moved to Cassandra ,
>> what are the best strategies available?
>>
>> Regards,
>> Shyam
>>
>> On Wed, 5 Sep 2018, 18:51 sha p, <sh...@gmail.com> wrote:
>>
>>>
>>> Hi all ,
>>>> Me new to Cassandra , i was asked to migrate data from Oracle to
>>>> Cassandra.
>>>> Please help me giving your valuable guidance.
>>>> 1) Can it be done using open source Cassandra.
>>>> 2) Where should I start data model from?
>>>> 3) I should use java, what kind of jar/libs/tools I need use ?
>>>> 4) How I decide the size of cluster , please provide some sample
>>>> guidelines.
>>>> 5) this should be in production , so what kind of things i should take
>>>> care for better support or debugging tomorrow?
>>>> 6) Please provide some good books /links which can help me in this task.
>>>>
>>>>
>>>> Thanks in advance.
>>>> Highly appreciated your every amal help.
>>>>
>>>> Regards,
>>>> Shyam
>>>>
>>>
Re: Regarding migrating data from Oracle to Cassandra.migrate data
from Oracle to Cassandra.
Posted by Jeff Jirsa <jj...@gmail.com>.
It very much depends on your application. You'll PROBABLY want to double
write for some period of time - start writes to both Cassandra and Oracle,
and then ensure they're both in sync. Once you're sure they're both in
sync, move your reads from Oracle to Cassandra.
On Wed, Sep 5, 2018 at 8:58 PM sha p <sh...@gmail.com> wrote:
> Hi all,
> Sir how should I keep track of the data which is moved to Cassandra , what
> are the best strategies available?
>
> Regards,
> Shyam
>
> On Wed, 5 Sep 2018, 18:51 sha p, <sh...@gmail.com> wrote:
>
>>
>> Hi all ,
>>> Me new to Cassandra , i was asked to migrate data from Oracle to
>>> Cassandra.
>>> Please help me giving your valuable guidance.
>>> 1) Can it be done using open source Cassandra.
>>> 2) Where should I start data model from?
>>> 3) I should use java, what kind of jar/libs/tools I need use ?
>>> 4) How I decide the size of cluster , please provide some sample
>>> guidelines.
>>> 5) this should be in production , so what kind of things i should take
>>> care for better support or debugging tomorrow?
>>> 6) Please provide some good books /links which can help me in this task.
>>>
>>>
>>> Thanks in advance.
>>> Highly appreciated your every amal help.
>>>
>>> Regards,
>>> Shyam
>>>
>>
Re: Regarding migrating data from Oracle to Cassandra.migrate data
from Oracle to Cassandra.
Posted by sha p <sh...@gmail.com>.
Hi all,
Sir how should I keep track of the data which is moved to Cassandra , what
are the best strategies available?
Regards,
Shyam
On Wed, 5 Sep 2018, 18:51 sha p, <sh...@gmail.com> wrote:
>
> Hi all ,
>> Me new to Cassandra , i was asked to migrate data from Oracle to
>> Cassandra.
>> Please help me giving your valuable guidance.
>> 1) Can it be done using open source Cassandra.
>> 2) Where should I start data model from?
>> 3) I should use java, what kind of jar/libs/tools I need use ?
>> 4) How I decide the size of cluster , please provide some sample
>> guidelines.
>> 5) this should be in production , so what kind of things i should take
>> care for better support or debugging tomorrow?
>> 6) Please provide some good books /links which can help me in this task.
>>
>>
>> Thanks in advance.
>> Highly appreciated your every amal help.
>>
>> Regards,
>> Shyam
>>
>