You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Vineet Mishra <cl...@gmail.com> on 2014/12/12 12:36:22 UTC
Java Client for Sqoop Import
Hi,
I am looking for some client for running sqoop jobs in Java. Although I
have easily imported data through sqoop shell command but looking out for
the other alternative through implementing the same in Java.
On urgent call!
RE: Java Client for Sqoop Import
Posted by David Robson <Da...@software.dell.com>.
Another option if you don’t want to write the output to a file you can add your own appender in log4j. You can then collect the log entries with your custom appender into memory and scan it there.
From: Sethuramaswamy, Suresh [mailto:suresh.sethuramaswamy@credit-suisse.com]
Sent: Tuesday, 16 December 2014 1:39 AM
To: user@sqoop.apache.org; clearmidoubt@gmail.com
Subject: RE: Java Client for Sqoop Import
Hi Vineet,
Writing the output log to a file and grep for the count may be an option to try out.
$(grep -oP 'Retrieved \K\d*' <logfileName>)
Suresh Sethuramaswamy
From: Vineet Mishra [mailto:clearmidoubt@gmail.com]
Sent: Monday, December 15, 2014 6:54 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Re: Java Client for Sqoop Import
Hi Abe,
Thanks for your mail, well I tried this link and able to make Sqoop invocation possible from a java code snippet.
http://www.acnenomor.com/451103p2/netbeanshadoop-errors-sqoophome-is-unset-may-not-be-able-to-find-all-job-dependencieshadoophome-is-not-set
I am much looking for some way of getting the number of records processed through sqoop import
INFO mapreduce.ImportJobBase: Retrieved 5 records.
so as to avoid another level of processing getting the import counts, I can get the records in the console but unfortunately not in the Sqoop.runTool since its not having access to any possible methods for getting processed result counts.
Looking forward for any workaround to get this hack.
Regards!
On Mon, Dec 15, 2014 at 7:08 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
Hey man,
Have you tried Sqoop2's Java client? It would currently only be useful when transfering data FROM/TO HDFS. This is changing fast though.
-Abe
On Sun, Dec 14, 2014 at 1:59 PM, Vineet Mishra <cl...@gmail.com>> wrote:
Has Anyone picked this up!
On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <cl...@gmail.com>> wrote:
Hi,
I am looking for some client for running sqoop jobs in Java. Although I have easily imported data through sqoop shell command but looking out for the other alternative through implementing the same in Java.
On urgent call!
==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================
RE: Java Client for Sqoop Import
Posted by "Sethuramaswamy, Suresh " <su...@credit-suisse.com>.
Hi Vineet,
Writing the output log to a file and grep for the count may be an option to try out.
$(grep -oP 'Retrieved \K\d*' <logfileName>)
Suresh Sethuramaswamy
From: Vineet Mishra [mailto:clearmidoubt@gmail.com]
Sent: Monday, December 15, 2014 6:54 AM
To: user@sqoop.apache.org
Subject: Re: Java Client for Sqoop Import
Hi Abe,
Thanks for your mail, well I tried this link and able to make Sqoop invocation possible from a java code snippet.
http://www.acnenomor.com/451103p2/netbeanshadoop-errors-sqoophome-is-unset-may-not-be-able-to-find-all-job-dependencieshadoophome-is-not-set
I am much looking for some way of getting the number of records processed through sqoop import
INFO mapreduce.ImportJobBase: Retrieved 5 records.
so as to avoid another level of processing getting the import counts, I can get the records in the console but unfortunately not in the Sqoop.runTool since its not having access to any possible methods for getting processed result counts.
Looking forward for any workaround to get this hack.
Regards!
On Mon, Dec 15, 2014 at 7:08 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
Hey man,
Have you tried Sqoop2's Java client? It would currently only be useful when transfering data FROM/TO HDFS. This is changing fast though.
-Abe
On Sun, Dec 14, 2014 at 1:59 PM, Vineet Mishra <cl...@gmail.com>> wrote:
Has Anyone picked this up!
On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <cl...@gmail.com>> wrote:
Hi,
I am looking for some client for running sqoop jobs in Java. Although I have easily imported data through sqoop shell command but looking out for the other alternative through implementing the same in Java.
On urgent call!
===============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
===============================================================================
Re: Java Client for Sqoop Import
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Abe,
Thanks for your mail, well I tried this link and able to make Sqoop
invocation possible from a java code snippet.
http://www.acnenomor.com/451103p2/netbeanshadoop-errors-sqoophome-is-unset-may-not-be-able-to-find-all-job-dependencieshadoophome-is-not-set
I am much looking for some way of getting the number of records processed
through sqoop import
INFO mapreduce.ImportJobBase: Retrieved 5 records.
so as to avoid another level of processing getting the import counts, I can
get the records in the console but unfortunately not in the Sqoop.runTool
since its not having access to any possible methods for getting processed
result counts.
Looking forward for any workaround to get this hack.
Regards!
On Mon, Dec 15, 2014 at 7:08 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
>
> Hey man,
>
> Have you tried Sqoop2's Java client? It would currently only be useful
> when transfering data FROM/TO HDFS. This is changing fast though.
>
> -Abe
>
> On Sun, Dec 14, 2014 at 1:59 PM, Vineet Mishra <cl...@gmail.com>
> wrote:
>>
>> Has Anyone picked this up!
>>
>> On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <cl...@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> I am looking for some client for running sqoop jobs in Java. Although I
>>> have easily imported data through sqoop shell command but looking out for
>>> the other alternative through implementing the same in Java.
>>>
>>> On urgent call!
>>>
>>
Re: Java Client for Sqoop Import
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey man,
Have you tried Sqoop2's Java client? It would currently only be useful when
transfering data FROM/TO HDFS. This is changing fast though.
-Abe
On Sun, Dec 14, 2014 at 1:59 PM, Vineet Mishra <cl...@gmail.com>
wrote:
>
> Has Anyone picked this up!
>
> On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <cl...@gmail.com>
> wrote:
>>
>> Hi,
>>
>> I am looking for some client for running sqoop jobs in Java. Although I
>> have easily imported data through sqoop shell command but looking out for
>> the other alternative through implementing the same in Java.
>>
>> On urgent call!
>>
>
Re: Java Client for Sqoop Import
Posted by Vineet Mishra <cl...@gmail.com>.
Has Anyone picked this up!
On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <cl...@gmail.com>
wrote:
>
> Hi,
>
> I am looking for some client for running sqoop jobs in Java. Although I
> have easily imported data through sqoop shell command but looking out for
> the other alternative through implementing the same in Java.
>
> On urgent call!
>