You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Dattatrya Moin <da...@gmail.com> on 2014/10/08 08:31:35 UTC
Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing
as it gives better scalability and performance. As i am new to hadoop
kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we
have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
Re: ETL using Hadoop
Posted by Alex Kamil <al...@gmail.com>.
the fastest way to do ETL on Hadoop is via Hbase+Phoenix JDBC driver
<http://phoenix.apache.org/>,
as for ODBC mapping you could use Thrift or one of the ODBC-JDBC bridges
<http://stackoverflow.com/questions/5352956/odbc-jdbc-bridge-that-maps-its-own-calls-to-jdbc-driver>
On Thu, Oct 9, 2014 at 8:16 AM, Andrew Machtolff <am...@askcts.com>
wrote:
> The closest thing I can think of to a .NET API would be to set up Hive
> external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You
> could connect from your .NET app using ODBC to the Hive tables, and
> SELECT/INSERT to read/write. If you’re desperate. J
>
>
>
> As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great
> little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve
> been using it for almost a year now, and it’s fantastic. Blazing fast, and
> with a trial download.
>
> (Disclaimer: I’m not affiliated with SyncSort, other than being a happy
> customer)
>
>
>
> Andrew
>
>
>
> *________________________________*
>
> Andrew Machtolff / Senior Consultant
>
> 205.259.2558 o
> 205.447.0956 c
>
> 205.259.2301 f
>
> [image: http://images.askcts.com/images/cts_logo_email.png]
> <http://www.askcts.com/>
> www.askcts.com
> amachtolff@askcts.com
> [image: http://images.askcts.com/images/cts_logo_linkedin.png]
> <http://www.linkedin.com/company/cts-inc>[image:
> http://images.askcts.com/images/cts_logo_twitter.png]
> <https://twitter.com/askCTS>[image:
> http://images.askcts.com/images/cts_logo_facebook.png]
> <https://www.facebook.com/askCTS>
>
>
>
> *From:* Azuryy Yu [mailto:azuryyyu@gmail.com]
> *Sent:* Wednesday, October 08, 2014 1:41 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: ETL using Hadoop
>
>
>
> Hi Moin,
>
> Yes, you can replace your ETL using hadoop. but it would be a big change.
> such as data collection, pre-process, ETL tasks rewrite etc.
>
>
>
> I don't think there is .NET API in Hadoop.
>
>
>
> On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
> wrote:
>
>
>
> Hi ,
>
>
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
>
>
>
> Thanks,
>
> Dattatrya Moin
>
>
>
>
>
Re: ETL using Hadoop
Posted by Alex Kamil <al...@gmail.com>.
the fastest way to do ETL on Hadoop is via Hbase+Phoenix JDBC driver
<http://phoenix.apache.org/>,
as for ODBC mapping you could use Thrift or one of the ODBC-JDBC bridges
<http://stackoverflow.com/questions/5352956/odbc-jdbc-bridge-that-maps-its-own-calls-to-jdbc-driver>
On Thu, Oct 9, 2014 at 8:16 AM, Andrew Machtolff <am...@askcts.com>
wrote:
> The closest thing I can think of to a .NET API would be to set up Hive
> external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You
> could connect from your .NET app using ODBC to the Hive tables, and
> SELECT/INSERT to read/write. If you’re desperate. J
>
>
>
> As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great
> little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve
> been using it for almost a year now, and it’s fantastic. Blazing fast, and
> with a trial download.
>
> (Disclaimer: I’m not affiliated with SyncSort, other than being a happy
> customer)
>
>
>
> Andrew
>
>
>
> *________________________________*
>
> Andrew Machtolff / Senior Consultant
>
> 205.259.2558 o
> 205.447.0956 c
>
> 205.259.2301 f
>
> [image: http://images.askcts.com/images/cts_logo_email.png]
> <http://www.askcts.com/>
> www.askcts.com
> amachtolff@askcts.com
> [image: http://images.askcts.com/images/cts_logo_linkedin.png]
> <http://www.linkedin.com/company/cts-inc>[image:
> http://images.askcts.com/images/cts_logo_twitter.png]
> <https://twitter.com/askCTS>[image:
> http://images.askcts.com/images/cts_logo_facebook.png]
> <https://www.facebook.com/askCTS>
>
>
>
> *From:* Azuryy Yu [mailto:azuryyyu@gmail.com]
> *Sent:* Wednesday, October 08, 2014 1:41 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: ETL using Hadoop
>
>
>
> Hi Moin,
>
> Yes, you can replace your ETL using hadoop. but it would be a big change.
> such as data collection, pre-process, ETL tasks rewrite etc.
>
>
>
> I don't think there is .NET API in Hadoop.
>
>
>
> On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
> wrote:
>
>
>
> Hi ,
>
>
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
>
>
>
> Thanks,
>
> Dattatrya Moin
>
>
>
>
>
Re: ETL using Hadoop
Posted by Alex Kamil <al...@gmail.com>.
the fastest way to do ETL on Hadoop is via Hbase+Phoenix JDBC driver
<http://phoenix.apache.org/>,
as for ODBC mapping you could use Thrift or one of the ODBC-JDBC bridges
<http://stackoverflow.com/questions/5352956/odbc-jdbc-bridge-that-maps-its-own-calls-to-jdbc-driver>
On Thu, Oct 9, 2014 at 8:16 AM, Andrew Machtolff <am...@askcts.com>
wrote:
> The closest thing I can think of to a .NET API would be to set up Hive
> external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You
> could connect from your .NET app using ODBC to the Hive tables, and
> SELECT/INSERT to read/write. If you’re desperate. J
>
>
>
> As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great
> little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve
> been using it for almost a year now, and it’s fantastic. Blazing fast, and
> with a trial download.
>
> (Disclaimer: I’m not affiliated with SyncSort, other than being a happy
> customer)
>
>
>
> Andrew
>
>
>
> *________________________________*
>
> Andrew Machtolff / Senior Consultant
>
> 205.259.2558 o
> 205.447.0956 c
>
> 205.259.2301 f
>
> [image: http://images.askcts.com/images/cts_logo_email.png]
> <http://www.askcts.com/>
> www.askcts.com
> amachtolff@askcts.com
> [image: http://images.askcts.com/images/cts_logo_linkedin.png]
> <http://www.linkedin.com/company/cts-inc>[image:
> http://images.askcts.com/images/cts_logo_twitter.png]
> <https://twitter.com/askCTS>[image:
> http://images.askcts.com/images/cts_logo_facebook.png]
> <https://www.facebook.com/askCTS>
>
>
>
> *From:* Azuryy Yu [mailto:azuryyyu@gmail.com]
> *Sent:* Wednesday, October 08, 2014 1:41 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: ETL using Hadoop
>
>
>
> Hi Moin,
>
> Yes, you can replace your ETL using hadoop. but it would be a big change.
> such as data collection, pre-process, ETL tasks rewrite etc.
>
>
>
> I don't think there is .NET API in Hadoop.
>
>
>
> On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
> wrote:
>
>
>
> Hi ,
>
>
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
>
>
>
> Thanks,
>
> Dattatrya Moin
>
>
>
>
>
Re: ETL using Hadoop
Posted by Alex Kamil <al...@gmail.com>.
the fastest way to do ETL on Hadoop is via Hbase+Phoenix JDBC driver
<http://phoenix.apache.org/>,
as for ODBC mapping you could use Thrift or one of the ODBC-JDBC bridges
<http://stackoverflow.com/questions/5352956/odbc-jdbc-bridge-that-maps-its-own-calls-to-jdbc-driver>
On Thu, Oct 9, 2014 at 8:16 AM, Andrew Machtolff <am...@askcts.com>
wrote:
> The closest thing I can think of to a .NET API would be to set up Hive
> external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You
> could connect from your .NET app using ODBC to the Hive tables, and
> SELECT/INSERT to read/write. If you’re desperate. J
>
>
>
> As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great
> little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve
> been using it for almost a year now, and it’s fantastic. Blazing fast, and
> with a trial download.
>
> (Disclaimer: I’m not affiliated with SyncSort, other than being a happy
> customer)
>
>
>
> Andrew
>
>
>
> *________________________________*
>
> Andrew Machtolff / Senior Consultant
>
> 205.259.2558 o
> 205.447.0956 c
>
> 205.259.2301 f
>
> [image: http://images.askcts.com/images/cts_logo_email.png]
> <http://www.askcts.com/>
> www.askcts.com
> amachtolff@askcts.com
> [image: http://images.askcts.com/images/cts_logo_linkedin.png]
> <http://www.linkedin.com/company/cts-inc>[image:
> http://images.askcts.com/images/cts_logo_twitter.png]
> <https://twitter.com/askCTS>[image:
> http://images.askcts.com/images/cts_logo_facebook.png]
> <https://www.facebook.com/askCTS>
>
>
>
> *From:* Azuryy Yu [mailto:azuryyyu@gmail.com]
> *Sent:* Wednesday, October 08, 2014 1:41 AM
> *To:* user@hadoop.apache.org
> *Subject:* Re: ETL using Hadoop
>
>
>
> Hi Moin,
>
> Yes, you can replace your ETL using hadoop. but it would be a big change.
> such as data collection, pre-process, ETL tasks rewrite etc.
>
>
>
> I don't think there is .NET API in Hadoop.
>
>
>
> On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
> wrote:
>
>
>
> Hi ,
>
>
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
>
>
>
> Thanks,
>
> Dattatrya Moin
>
>
>
>
>
RE: ETL using Hadoop
Posted by Andrew Machtolff <am...@askcts.com>.
The closest thing I can think of to a .NET API would be to set up Hive external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You could connect from your .NET app using ODBC to the Hive tables, and SELECT/INSERT to read/write. If you’re desperate. ☺
As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve been using it for almost a year now, and it’s fantastic. Blazing fast, and with a trial download.
(Disclaimer: I’m not affiliated with SyncSort, other than being a happy customer)
Andrew
________________________________
Andrew Machtolff / Senior Consultant
205.259.2558 o
205.447.0956 c
205.259.2301 f
[http://images.askcts.com/images/cts_logo_email.png]<http://www.askcts.com/>
www.askcts.com<http://www.askcts.com/>
amachtolff@askcts.com<ma...@askcts.com>
[http://images.askcts.com/images/cts_logo_linkedin.png]<http://www.linkedin.com/company/cts-inc>[http://images.askcts.com/images/cts_logo_twitter.png]<https://twitter.com/askCTS>[http://images.askcts.com/images/cts_logo_facebook.png]<https://www.facebook.com/askCTS>
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Wednesday, October 08, 2014 1:41 AM
To: user@hadoop.apache.org
Subject: Re: ETL using Hadoop
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change. such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>> wrote:
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: ETL using Hadoop
Posted by Andrew Machtolff <am...@askcts.com>.
The closest thing I can think of to a .NET API would be to set up Hive external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You could connect from your .NET app using ODBC to the Hive tables, and SELECT/INSERT to read/write. If you’re desperate. ☺
As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve been using it for almost a year now, and it’s fantastic. Blazing fast, and with a trial download.
(Disclaimer: I’m not affiliated with SyncSort, other than being a happy customer)
Andrew
________________________________
Andrew Machtolff / Senior Consultant
205.259.2558 o
205.447.0956 c
205.259.2301 f
[http://images.askcts.com/images/cts_logo_email.png]<http://www.askcts.com/>
www.askcts.com<http://www.askcts.com/>
amachtolff@askcts.com<ma...@askcts.com>
[http://images.askcts.com/images/cts_logo_linkedin.png]<http://www.linkedin.com/company/cts-inc>[http://images.askcts.com/images/cts_logo_twitter.png]<https://twitter.com/askCTS>[http://images.askcts.com/images/cts_logo_facebook.png]<https://www.facebook.com/askCTS>
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Wednesday, October 08, 2014 1:41 AM
To: user@hadoop.apache.org
Subject: Re: ETL using Hadoop
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change. such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>> wrote:
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: ETL using Hadoop
Posted by Andrew Machtolff <am...@askcts.com>.
The closest thing I can think of to a .NET API would be to set up Hive external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You could connect from your .NET app using ODBC to the Hive tables, and SELECT/INSERT to read/write. If you’re desperate. ☺
As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve been using it for almost a year now, and it’s fantastic. Blazing fast, and with a trial download.
(Disclaimer: I’m not affiliated with SyncSort, other than being a happy customer)
Andrew
________________________________
Andrew Machtolff / Senior Consultant
205.259.2558 o
205.447.0956 c
205.259.2301 f
[http://images.askcts.com/images/cts_logo_email.png]<http://www.askcts.com/>
www.askcts.com<http://www.askcts.com/>
amachtolff@askcts.com<ma...@askcts.com>
[http://images.askcts.com/images/cts_logo_linkedin.png]<http://www.linkedin.com/company/cts-inc>[http://images.askcts.com/images/cts_logo_twitter.png]<https://twitter.com/askCTS>[http://images.askcts.com/images/cts_logo_facebook.png]<https://www.facebook.com/askCTS>
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Wednesday, October 08, 2014 1:41 AM
To: user@hadoop.apache.org
Subject: Re: ETL using Hadoop
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change. such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>> wrote:
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: ETL using Hadoop
Posted by Andrew Machtolff <am...@askcts.com>.
The closest thing I can think of to a .NET API would be to set up Hive external tables, and use a vendor’s (Cloudera, et al.) ODBC driver. You could connect from your .NET app using ODBC to the Hive tables, and SELECT/INSERT to read/write. If you’re desperate. ☺
As far as ETL, I’d recommend you give SyncSort DMX-h a try. It’s a great little ETL tool that can translate its ETL tasks to MapReduce jobs. I’ve been using it for almost a year now, and it’s fantastic. Blazing fast, and with a trial download.
(Disclaimer: I’m not affiliated with SyncSort, other than being a happy customer)
Andrew
________________________________
Andrew Machtolff / Senior Consultant
205.259.2558 o
205.447.0956 c
205.259.2301 f
[http://images.askcts.com/images/cts_logo_email.png]<http://www.askcts.com/>
www.askcts.com<http://www.askcts.com/>
amachtolff@askcts.com<ma...@askcts.com>
[http://images.askcts.com/images/cts_logo_linkedin.png]<http://www.linkedin.com/company/cts-inc>[http://images.askcts.com/images/cts_logo_twitter.png]<https://twitter.com/askCTS>[http://images.askcts.com/images/cts_logo_facebook.png]<https://www.facebook.com/askCTS>
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Wednesday, October 08, 2014 1:41 AM
To: user@hadoop.apache.org
Subject: Re: ETL using Hadoop
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change. such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>> wrote:
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
Re: ETL using Hadoop
Posted by Azuryy Yu <az...@gmail.com>.
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change.
such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by daemeon reiydelle <da...@gmail.com>.
Hadoop is in effect a massively fast etl with high latency as the tradeoff.
Other solutions allow different tradeoffs. And some of those occur in Map
phase, some in a reduce phase (e.g. Stream or columnar stores).
On Oct 7, 2014 11:32 PM, "Dattatrya Moin" <da...@gmail.com> wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by daemeon reiydelle <da...@gmail.com>.
Hadoop is in effect a massively fast etl with high latency as the tradeoff.
Other solutions allow different tradeoffs. And some of those occur in Map
phase, some in a reduce phase (e.g. Stream or columnar stores).
On Oct 7, 2014 11:32 PM, "Dattatrya Moin" <da...@gmail.com> wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: ETL using Hadoop
Posted by Azuryy Yu <az...@gmail.com>.
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change.
such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: ETL using Hadoop
Posted by Azuryy Yu <az...@gmail.com>.
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change.
such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
RE: Fwd: ETL using Hadoop
Posted by "Chaudhari, Rahul A" <ra...@verizon.com>.
For Oracle, you can also use GoldenGate for realtime replication and ODI for simplifying the ETS workflow.
Regards,
Rahul Chaudhari
From: Dattatrya Moin [mailto:dattatryamoin@gmail.com]
Sent: Wednesday, October 08, 2014 12:44 PM
To: user
Subject: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
if complex use script such as BRMS to deal the biz
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Blade Liu";<ha...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午4:54
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Why not process data on-the-fly? I guess reading and writing data in HDFS would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
if complex use script such as BRMS to deal the biz
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Blade Liu";<ha...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午4:54
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Why not process data on-the-fly? I guess reading and writing data in HDFS would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
if complex use script such as BRMS to deal the biz
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Blade Liu";<ha...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午4:54
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Why not process data on-the-fly? I guess reading and writing data in HDFS would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
if complex use script such as BRMS to deal the biz
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Blade Liu";<ha...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午4:54
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Why not process data on-the-fly? I guess reading and writing data in HDFS would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
Re: Fwd: ETL using Hadoop
Posted by Blade Liu <ha...@gmail.com>.
Why not process data on-the-fly? I guess reading and writing data in HDFS
would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
> using mr job to "ETL" the temporary data store in hdfs
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:44
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Re: Fwd: ETL using Hadoop
>
> Hi ,
>
> We are extracting data from Oracle database, we are processing that data
> (Transformation) i.e. Substring ,Replace and loading modified data back to
> Oracle database.
>
> On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
>
>> where to "E" where to "L"? what do you do in "E" and "T"?
>>
>> ------------------
>> ---------------------------------------------
>> BestWishes！
>> Blog:http://snv.iteye.com/
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
>> *发送时间:* 2014年10月8日(星期三) 下午3:01
>> *收件人:* "user"<us...@hadoop.apache.org>;
>> *主题:* Fwd: ETL using Hadoop
>>
>>
>> Hi ,
>>
>> We have our own ETL , but we are planning to use Hadoop for data
>> processing as it gives better scalability and performance. As i am new to
>> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
>> And we have any API to connect to Hadoop using .NET.
>>
>>
>> Thanks,
>> Dattatrya Moin
>>
>>
>
Re: Fwd: ETL using Hadoop
Posted by Blade Liu <ha...@gmail.com>.
Why not process data on-the-fly? I guess reading and writing data in HDFS
would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
> using mr job to "ETL" the temporary data store in hdfs
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:44
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Re: Fwd: ETL using Hadoop
>
> Hi ,
>
> We are extracting data from Oracle database, we are processing that data
> (Transformation) i.e. Substring ,Replace and loading modified data back to
> Oracle database.
>
> On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
>
>> where to "E" where to "L"? what do you do in "E" and "T"?
>>
>> ------------------
>> ---------------------------------------------
>> BestWishes！
>> Blog:http://snv.iteye.com/
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
>> *发送时间:* 2014年10月8日(星期三) 下午3:01
>> *收件人:* "user"<us...@hadoop.apache.org>;
>> *主题:* Fwd: ETL using Hadoop
>>
>>
>> Hi ,
>>
>> We have our own ETL , but we are planning to use Hadoop for data
>> processing as it gives better scalability and performance. As i am new to
>> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
>> And we have any API to connect to Hadoop using .NET.
>>
>>
>> Thanks,
>> Dattatrya Moin
>>
>>
>
Re: Fwd: ETL using Hadoop
Posted by Blade Liu <ha...@gmail.com>.
Why not process data on-the-fly? I guess reading and writing data in HDFS
would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
> using mr job to "ETL" the temporary data store in hdfs
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:44
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Re: Fwd: ETL using Hadoop
>
> Hi ,
>
> We are extracting data from Oracle database, we are processing that data
> (Transformation) i.e. Substring ,Replace and loading modified data back to
> Oracle database.
>
> On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
>
>> where to "E" where to "L"? what do you do in "E" and "T"?
>>
>> ------------------
>> ---------------------------------------------
>> BestWishes！
>> Blog:http://snv.iteye.com/
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
>> *发送时间:* 2014年10月8日(星期三) 下午3:01
>> *收件人:* "user"<us...@hadoop.apache.org>;
>> *主题:* Fwd: ETL using Hadoop
>>
>>
>> Hi ,
>>
>> We have our own ETL , but we are planning to use Hadoop for data
>> processing as it gives better scalability and performance. As i am new to
>> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
>> And we have any API to connect to Hadoop using .NET.
>>
>>
>> Thanks,
>> Dattatrya Moin
>>
>>
>
Re: Fwd: ETL using Hadoop
Posted by Blade Liu <ha...@gmail.com>.
Why not process data on-the-fly? I guess reading and writing data in HDFS
would incur high latency.
2014-10-08 15:20 GMT+08:00 snv <sm...@foxmail.com>:
> using mr job to "ETL" the temporary data store in hdfs
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:44
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Re: Fwd: ETL using Hadoop
>
> Hi ,
>
> We are extracting data from Oracle database, we are processing that data
> (Transformation) i.e. Substring ,Replace and loading modified data back to
> Oracle database.
>
> On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
>
>> where to "E" where to "L"? what do you do in "E" and "T"?
>>
>> ------------------
>> ---------------------------------------------
>> BestWishes！
>> Blog:http://snv.iteye.com/
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
>> *发送时间:* 2014年10月8日(星期三) 下午3:01
>> *收件人:* "user"<us...@hadoop.apache.org>;
>> *主题:* Fwd: ETL using Hadoop
>>
>>
>> Hi ,
>>
>> We have our own ETL , but we are planning to use Hadoop for data
>> processing as it gives better scalability and performance. As i am new to
>> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
>> And we have any API to connect to Hadoop using .NET.
>>
>>
>> Thanks,
>> Dattatrya Moin
>>
>>
>
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: Fwd: ETL using Hadoop
Posted by "Chaudhari, Rahul A" <ra...@verizon.com>.
For Oracle, you can also use GoldenGate for realtime replication and ODI for simplifying the ETS workflow.
Regards,
Rahul Chaudhari
From: Dattatrya Moin [mailto:dattatryamoin@gmail.com]
Sent: Wednesday, October 08, 2014 12:44 PM
To: user
Subject: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: Fwd: ETL using Hadoop
Posted by "Chaudhari, Rahul A" <ra...@verizon.com>.
For Oracle, you can also use GoldenGate for realtime replication and ODI for simplifying the ETS workflow.
Regards,
Rahul Chaudhari
From: Dattatrya Moin [mailto:dattatryamoin@gmail.com]
Sent: Wednesday, October 08, 2014 12:44 PM
To: user
Subject: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
RE: Fwd: ETL using Hadoop
Posted by "Chaudhari, Rahul A" <ra...@verizon.com>.
For Oracle, you can also use GoldenGate for realtime replication and ODI for simplifying the ETS workflow.
Regards,
Rahul Chaudhari
From: Dattatrya Moin [mailto:dattatryamoin@gmail.com]
Sent: Wednesday, October 08, 2014 12:44 PM
To: user
Subject: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复： Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
using mr job to "ETL" the temporary data store in hdfs
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:44
收件人: "user"<us...@hadoop.apache.org>;
主题: Re: Fwd: ETL using Hadoop
Hi ,
We are extracting data from Oracle database, we are processing that data (Transformation) i.e. Substring ,Replace and loading modified data back to Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
Re: Fwd: ETL using Hadoop
Posted by Dattatrya Moin <da...@gmail.com>.
Hi ,
We are extracting data from Oracle database, we are processing that data
(Transformation) i.e. Substring ,Replace and loading modified data back to
Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
> where to "E" where to "L"? what do you do in "E" and "T"?
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:01
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Fwd: ETL using Hadoop
>
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by Dattatrya Moin <da...@gmail.com>.
Hi ,
We are extracting data from Oracle database, we are processing that data
(Transformation) i.e. Substring ,Replace and loading modified data back to
Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
> where to "E" where to "L"? what do you do in "E" and "T"?
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:01
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Fwd: ETL using Hadoop
>
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by Dattatrya Moin <da...@gmail.com>.
Hi ,
We are extracting data from Oracle database, we are processing that data
(Transformation) i.e. Substring ,Replace and loading modified data back to
Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
> where to "E" where to "L"? what do you do in "E" and "T"?
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:01
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Fwd: ETL using Hadoop
>
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by Dattatrya Moin <da...@gmail.com>.
Hi ,
We are extracting data from Oracle database, we are processing that data
(Transformation) i.e. Substring ,Replace and loading modified data back to
Oracle database.
On Wed, Oct 8, 2014 at 12:13 PM, snv <sm...@foxmail.com> wrote:
> where to "E" where to "L"? what do you do in "E" and "T"?
>
> ------------------
> ---------------------------------------------
> BestWishes！
> Blog:http://snv.iteye.com/
>
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Dattatrya Moin";<da...@gmail.com>;
> *发送时间:* 2014年10月8日(星期三) 下午3:01
> *收件人:* "user"<us...@hadoop.apache.org>;
> *主题:* Fwd: ETL using Hadoop
>
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
回复：Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复：Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
回复：Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin
Re: Fwd: ETL using Hadoop
Posted by daemeon reiydelle <da...@gmail.com>.
Hadoop is in effect a massively fast etl with high latency as the tradeoff.
Other solutions allow different tradeoffs. And some of those occur in Map
phase, some in a reduce phase (e.g. Stream or columnar stores).
On Oct 7, 2014 11:32 PM, "Dattatrya Moin" <da...@gmail.com> wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: Fwd: ETL using Hadoop
Posted by daemeon reiydelle <da...@gmail.com>.
Hadoop is in effect a massively fast etl with high latency as the tradeoff.
Other solutions allow different tradeoffs. And some of those occur in Map
phase, some in a reduce phase (e.g. Stream or columnar stores).
On Oct 7, 2014 11:32 PM, "Dattatrya Moin" <da...@gmail.com> wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
Re: ETL using Hadoop
Posted by Azuryy Yu <az...@gmail.com>.
Hi Moin,
Yes, you can replace your ETL using hadoop. but it would be a big change.
such as data collection, pre-process, ETL tasks rewrite etc.
I don't think there is .NET API in Hadoop.
On Wed, Oct 8, 2014 at 2:31 PM, Dattatrya Moin <da...@gmail.com>
wrote:
>
> Hi ,
>
> We have our own ETL , but we are planning to use Hadoop for data
> processing as it gives better scalability and performance. As i am new to
> hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop .
> And we have any API to connect to Hadoop using .NET.
>
>
> Thanks,
> Dattatrya Moin
>
>
回复：Fwd: ETL using Hadoop
Posted by snv <sm...@foxmail.com>.
where to "E" where to "L"? what do you do in "E" and "T"?
------------------
---------------------------------------------
BestWishes！
Blog:http://snv.iteye.com/
------------------ 原始邮件 ------------------
发件人: "Dattatrya Moin";<da...@gmail.com>;
发送时间: 2014年10月8日(星期三) 下午3:01
收件人: "user"<us...@hadoop.apache.org>;
主题: Fwd: ETL using Hadoop
Hi ,
We have our own ETL , but we are planning to use Hadoop for data processing as it gives better scalability and performance. As i am new to hadoop kindly guide to start with Hadoop. Can we replace ETL using Hadoop . And we have any API to connect to Hadoop using .NET.
Thanks,
Dattatrya Moin