You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2013/08/16 21:16:46 UTC
Things to keep in mind when writing to a db
Hi,
I am wondering if there is any tutorial to see.
What are the challenges for reading and/or writing to/from database.
Is there a common flavor across all the database.
For example, the dbs start a server on some host : port
Establish connection to that host:port
It can be across proxy?
Which is to say, if I dont want to use Squoop and want to learn a very
basic fragile way to write (and read.. but mostly write) to sql table,
what are the steps.?
Or how should I approach this?
Thanks
Re: Things to keep in mind when writing to a db
Posted by Shahab Yunus <sh...@gmail.com>.
There are quite a few examples on the web, if you search for them. A quick
search yielded the following e.g.:
http://jaganadhg.freeflux.net/blog/archive/tag/example/
http://techannotation.wordpress.com/2012/09/10/hadoop-in-practice/
Regards,
Shahab
On Mon, Aug 19, 2013 at 5:54 AM, Devaraj k <de...@huawei.com> wrote:
> If you want to read/write through mapred Job, You can refer
> org.apache.hadoop.mapred.lib.db.DBInputFormat &
> org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing
> from/into database.****
>
> ** **
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
> ****
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
> ****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* jamal sasha [mailto:jamalshasha@gmail.com]
> *Sent:* 17 August 2013 00:47
> *To:* user@hadoop.apache.org
> *Subject:* Things to keep in mind when writing to a db****
>
> ** **
>
> Hi,****
>
> I am wondering if there is any tutorial to see.****
>
> What are the challenges for reading and/or writing to/from database.****
>
> ** **
>
> Is there a common flavor across all the database.****
>
> For example, the dbs start a server on some host : port****
>
> Establish connection to that host:port****
>
> It can be across proxy?****
>
> ** **
>
> Which is to say, if I dont want to use Squoop and want to learn a very
> basic fragile way to write (and read.. but mostly write) to sql table,
> what are the steps.?****
>
> Or how should I approach this?****
>
> Thanks****
>
Re: Things to keep in mind when writing to a db
Posted by Shahab Yunus <sh...@gmail.com>.
There are quite a few examples on the web, if you search for them. A quick
search yielded the following e.g.:
http://jaganadhg.freeflux.net/blog/archive/tag/example/
http://techannotation.wordpress.com/2012/09/10/hadoop-in-practice/
Regards,
Shahab
On Mon, Aug 19, 2013 at 5:54 AM, Devaraj k <de...@huawei.com> wrote:
> If you want to read/write through mapred Job, You can refer
> org.apache.hadoop.mapred.lib.db.DBInputFormat &
> org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing
> from/into database.****
>
> ** **
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
> ****
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
> ****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* jamal sasha [mailto:jamalshasha@gmail.com]
> *Sent:* 17 August 2013 00:47
> *To:* user@hadoop.apache.org
> *Subject:* Things to keep in mind when writing to a db****
>
> ** **
>
> Hi,****
>
> I am wondering if there is any tutorial to see.****
>
> What are the challenges for reading and/or writing to/from database.****
>
> ** **
>
> Is there a common flavor across all the database.****
>
> For example, the dbs start a server on some host : port****
>
> Establish connection to that host:port****
>
> It can be across proxy?****
>
> ** **
>
> Which is to say, if I dont want to use Squoop and want to learn a very
> basic fragile way to write (and read.. but mostly write) to sql table,
> what are the steps.?****
>
> Or how should I approach this?****
>
> Thanks****
>
Re: Things to keep in mind when writing to a db
Posted by Shahab Yunus <sh...@gmail.com>.
There are quite a few examples on the web, if you search for them. A quick
search yielded the following e.g.:
http://jaganadhg.freeflux.net/blog/archive/tag/example/
http://techannotation.wordpress.com/2012/09/10/hadoop-in-practice/
Regards,
Shahab
On Mon, Aug 19, 2013 at 5:54 AM, Devaraj k <de...@huawei.com> wrote:
> If you want to read/write through mapred Job, You can refer
> org.apache.hadoop.mapred.lib.db.DBInputFormat &
> org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing
> from/into database.****
>
> ** **
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
> ****
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
> ****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* jamal sasha [mailto:jamalshasha@gmail.com]
> *Sent:* 17 August 2013 00:47
> *To:* user@hadoop.apache.org
> *Subject:* Things to keep in mind when writing to a db****
>
> ** **
>
> Hi,****
>
> I am wondering if there is any tutorial to see.****
>
> What are the challenges for reading and/or writing to/from database.****
>
> ** **
>
> Is there a common flavor across all the database.****
>
> For example, the dbs start a server on some host : port****
>
> Establish connection to that host:port****
>
> It can be across proxy?****
>
> ** **
>
> Which is to say, if I dont want to use Squoop and want to learn a very
> basic fragile way to write (and read.. but mostly write) to sql table,
> what are the steps.?****
>
> Or how should I approach this?****
>
> Thanks****
>
Re: Things to keep in mind when writing to a db
Posted by Shahab Yunus <sh...@gmail.com>.
There are quite a few examples on the web, if you search for them. A quick
search yielded the following e.g.:
http://jaganadhg.freeflux.net/blog/archive/tag/example/
http://techannotation.wordpress.com/2012/09/10/hadoop-in-practice/
Regards,
Shahab
On Mon, Aug 19, 2013 at 5:54 AM, Devaraj k <de...@huawei.com> wrote:
> If you want to read/write through mapred Job, You can refer
> org.apache.hadoop.mapred.lib.db.DBInputFormat &
> org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing
> from/into database.****
>
> ** **
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
> ****
>
>
> http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
> ****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* jamal sasha [mailto:jamalshasha@gmail.com]
> *Sent:* 17 August 2013 00:47
> *To:* user@hadoop.apache.org
> *Subject:* Things to keep in mind when writing to a db****
>
> ** **
>
> Hi,****
>
> I am wondering if there is any tutorial to see.****
>
> What are the challenges for reading and/or writing to/from database.****
>
> ** **
>
> Is there a common flavor across all the database.****
>
> For example, the dbs start a server on some host : port****
>
> Establish connection to that host:port****
>
> It can be across proxy?****
>
> ** **
>
> Which is to say, if I dont want to use Squoop and want to learn a very
> basic fragile way to write (and read.. but mostly write) to sql table,
> what are the steps.?****
>
> Or how should I approach this?****
>
> Thanks****
>
RE: Things to keep in mind when writing to a db
Posted by Devaraj k <de...@huawei.com>.
If you want to read/write through mapred Job, You can refer org.apache.hadoop.mapred.lib.db.DBInputFormat & org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing from/into database.
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
Thanks
Devaraj k
From: jamal sasha [mailto:jamalshasha@gmail.com]
Sent: 17 August 2013 00:47
To: user@hadoop.apache.org
Subject: Things to keep in mind when writing to a db
Hi,
I am wondering if there is any tutorial to see.
What are the challenges for reading and/or writing to/from database.
Is there a common flavor across all the database.
For example, the dbs start a server on some host : port
Establish connection to that host:port
It can be across proxy?
Which is to say, if I dont want to use Squoop and want to learn a very basic fragile way to write (and read.. but mostly write) to sql table, what are the steps.?
Or how should I approach this?
Thanks
RE: Things to keep in mind when writing to a db
Posted by Devaraj k <de...@huawei.com>.
If you want to read/write through mapred Job, You can refer org.apache.hadoop.mapred.lib.db.DBInputFormat & org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing from/into database.
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
Thanks
Devaraj k
From: jamal sasha [mailto:jamalshasha@gmail.com]
Sent: 17 August 2013 00:47
To: user@hadoop.apache.org
Subject: Things to keep in mind when writing to a db
Hi,
I am wondering if there is any tutorial to see.
What are the challenges for reading and/or writing to/from database.
Is there a common flavor across all the database.
For example, the dbs start a server on some host : port
Establish connection to that host:port
It can be across proxy?
Which is to say, if I dont want to use Squoop and want to learn a very basic fragile way to write (and read.. but mostly write) to sql table, what are the steps.?
Or how should I approach this?
Thanks
RE: Things to keep in mind when writing to a db
Posted by Devaraj k <de...@huawei.com>.
If you want to read/write through mapred Job, You can refer org.apache.hadoop.mapred.lib.db.DBInputFormat & org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing from/into database.
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
Thanks
Devaraj k
From: jamal sasha [mailto:jamalshasha@gmail.com]
Sent: 17 August 2013 00:47
To: user@hadoop.apache.org
Subject: Things to keep in mind when writing to a db
Hi,
I am wondering if there is any tutorial to see.
What are the challenges for reading and/or writing to/from database.
Is there a common flavor across all the database.
For example, the dbs start a server on some host : port
Establish connection to that host:port
It can be across proxy?
Which is to say, if I dont want to use Squoop and want to learn a very basic fragile way to write (and read.. but mostly write) to sql table, what are the steps.?
Or how should I approach this?
Thanks
RE: Things to keep in mind when writing to a db
Posted by Devaraj k <de...@huawei.com>.
If you want to read/write through mapred Job, You can refer org.apache.hadoop.mapred.lib.db.DBInputFormat & org.apache.hadoop.mapred.lib.db.DBOutputFormat for reading and writing from/into database.
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.html
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.html
Thanks
Devaraj k
From: jamal sasha [mailto:jamalshasha@gmail.com]
Sent: 17 August 2013 00:47
To: user@hadoop.apache.org
Subject: Things to keep in mind when writing to a db
Hi,
I am wondering if there is any tutorial to see.
What are the challenges for reading and/or writing to/from database.
Is there a common flavor across all the database.
For example, the dbs start a server on some host : port
Establish connection to that host:port
It can be across proxy?
Which is to say, if I dont want to use Squoop and want to learn a very basic fragile way to write (and read.. but mostly write) to sql table, what are the steps.?
Or how should I approach this?
Thanks