You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by SULABH <su...@gmail.com> on 2011/03/15 01:52:45 UTC
Cannot connect HBase to Pig
Hi,
I am trying to extract data stored in HBase using Pig.
I got Pig to work on Hadoop and then also ran a sample M/R job and got
correct results.
Now when I try to access HBase using the HbaseStorage the compiler just
hangs after
2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
org.apache.zookeeper.ClientCnxn - Server connection successful
Even after waiting for several minutes it just remains there. There is
no information in the log files too.
Please help!
Re: Cannot connect HBase to Pig
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I don't think the hadoop conf directory is on the pig classpath.
You need to make sure that the following things are on the classpath (the
easiest way to do that is to set PIG_CLASSPATH env var):
the hadoop conf directory
all the jars in the hadoop lib directory
guava jar
hbase jar
zookeeper jar
-D
On Tue, Mar 15, 2011 at 4:13 PM, sulabh choudhury <su...@gmail.com> wrote:
> The logs do not provide any useful info.
>
> I am trying to use the new jar from branch-0.8 now.
>
> Now it pops other errors
> Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor
> core-site.xml was found in the classpath)
> but since I am using Cloudera dist which does not have hadoop-site.xml and
> core-site.xml instead it has mapred-site.xml, hdfs-site.xml,core-site.xml
> and I have already specified the path to my Hadoop conf folder.
>
>
>
> On Tue, Mar 15, 2011 at 3:11 PM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>
>> Anything interesting in Map task logs?
>>
>> As for your previous question -- just check out the branch-0.8, run "ant jar-withouthadoop"
>> and use the the resulting jar.
>>
>>
>> On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury <su...@gmail.com>wrote:
>>
>>> Dmitriy...
>>> I just found out that after 20 mins I get throws errors
>>>
>>> 2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
>>> org.apache.zookeeper.ClientCnxn - Server connection successful
>>> 2011-03-15 11:16:39,186 [Thread-4] INFO
>>> org.apache.hadoop.mapred.JobClient - Cleaning up the staging area
>>> hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
>>> 2011-03-15 11:16:39,188 [main] INFO
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>>> - job null has failed! Stop running all dependent jobs
>>> 2011-03-15 11:16:39,194 [main] INFO
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>>> - 100% complete
>>> 2011-03-15 11:16:39,203 [main] ERROR
>>> org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to recreate
>>> exception from backend error:
>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
>>> trying to locate root region
>>> 2011-03-15 11:16:39,204 [main] ERROR
>>> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
>>>
>>> Are the previous errors related or I am making heads-up and start looking
>>> into different direction now ?
>>>
>>> On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury <su...@gmail.com>wrote:
>>>
>>>> I am a little confused here....
>>>> Can you elaborate a little more on how to achieve the above ( Pig jar
>>>> from the pig-0.8 branch) and also then how to use the same to make it work
>>>> with Hbase 0.90 ?
>>>>
>>>> On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>>
>>>>> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall
>>>>> correctly.
>>>>> You can compile a Pig jar from the pig-0.8 branch, and it should work
>>>>> with 90+
>>>>>
>>>>> D
>>>>>
>>>>>
>>>>> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com>wrote:
>>>>>
>>>>>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
>>>>>> distribution.
>>>>>> Yes it does not crash it just hangs after the
>>>>>> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>>>>>>
>>>>>> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
>>>>>> all the Hunks failed...
>>>>>>
>>>>>> Also I see that
>>>>>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn -
>>>>>> Attempting connection to server localhost/127.0.0.1:2181
>>>>>> So that means that it is trying to connect to the correct (2181)
>>>>>> zookeeper port...
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>>>>
>>>>>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes
>>>>>>> out pretty fast.
>>>>>>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>>>>>>
>>>>>>> D
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>>>>>>
>>>>>>>> What I'm saying is that if your version of HBase is 0.89.0 or
>>>>>>>> greater,
>>>>>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>>>>>>> and rebuild.
>>>>>>>>
>>>>>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>>>>>>> can try applying the patch to that version, but I'd expect you'd run
>>>>>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>>>>>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>>>>>>
>>>>>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <
>>>>>>>> sulabhc@gmail.com> wrote:
>>>>>>>> > Thanks Bill for the response.
>>>>>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no
>>>>>>>> other
>>>>>>>> > version, even 0.70 ?
>>>>>>>> >
>>>>>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <
>>>>>>>> billgraham@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>>>>>>> >> you'll need PIG-1680:
>>>>>>>> >>
>>>>>>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>>>>>>> >>
>>>>>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> >> > Hi,
>>>>>>>> >> >
>>>>>>>> >> > I am trying to extract data stored in HBase using Pig.
>>>>>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job
>>>>>>>> and got
>>>>>>>> >> > correct results.
>>>>>>>> >> > Now when I try to access HBase using the HbaseStorage the
>>>>>>>> compiler just
>>>>>>>> >> > hangs after
>>>>>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>>>>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>>>>>>> >> > Even after waiting for several minutes it just remains there.
>>>>>>>> There is
>>>>>>>> >> > no
>>>>>>>> >> > information in the log files too.
>>>>>>>> >> >
>>>>>>>> >> > Please help!
>>>>>>>> >> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> --
>>>>>> Thanks and Regards,
>>>>>> Sulabh Choudhury
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> SC
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> SC
>>>
>>
>>
>
>
> --
>
>
>
Re: Cannot connect HBase to Pig
Posted by sulabh choudhury <su...@gmail.com>.
The logs do not provide any useful info.
I am trying to use the new jar from branch-0.8 now.
Now it pops other errors
Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor
core-site.xml was found in the classpath)
but since I am using Cloudera dist which does not have hadoop-site.xml and
core-site.xml instead it has mapred-site.xml, hdfs-site.xml,core-site.xml
and I have already specified the path to my Hadoop conf folder.
On Tue, Mar 15, 2011 at 3:11 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Anything interesting in Map task logs?
>
> As for your previous question -- just check out the branch-0.8, run "ant jar-withouthadoop"
> and use the the resulting jar.
>
>
> On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury <su...@gmail.com>wrote:
>
>> Dmitriy...
>> I just found out that after 20 mins I get throws errors
>>
>> 2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
>> org.apache.zookeeper.ClientCnxn - Server connection successful
>> 2011-03-15 11:16:39,186 [Thread-4] INFO
>> org.apache.hadoop.mapred.JobClient - Cleaning up the staging area
>> hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
>> 2011-03-15 11:16:39,188 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - job null has failed! Stop running all dependent jobs
>> 2011-03-15 11:16:39,194 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> - 100% complete
>> 2011-03-15 11:16:39,203 [main] ERROR
>> org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to recreate
>> exception from backend error:
>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
>> trying to locate root region
>> 2011-03-15 11:16:39,204 [main] ERROR
>> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
>>
>> Are the previous errors related or I am making heads-up and start looking
>> into different direction now ?
>>
>> On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury <su...@gmail.com>wrote:
>>
>>> I am a little confused here....
>>> Can you elaborate a little more on how to achieve the above ( Pig jar
>>> from the pig-0.8 branch) and also then how to use the same to make it work
>>> with Hbase 0.90 ?
>>>
>>> On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>
>>>> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall
>>>> correctly.
>>>> You can compile a Pig jar from the pig-0.8 branch, and it should work
>>>> with 90+
>>>>
>>>> D
>>>>
>>>>
>>>> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com>wrote:
>>>>
>>>>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
>>>>> distribution.
>>>>> Yes it does not crash it just hangs after the
>>>>> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>>>>>
>>>>> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
>>>>> all the Hunks failed...
>>>>>
>>>>> Also I see that
>>>>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
>>>>> connection to server localhost/127.0.0.1:2181
>>>>> So that means that it is trying to connect to the correct (2181)
>>>>> zookeeper port...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>>>
>>>>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes
>>>>>> out pretty fast.
>>>>>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>>>>>
>>>>>> D
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>>>>>
>>>>>>> What I'm saying is that if your version of HBase is 0.89.0 or
>>>>>>> greater,
>>>>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>>>>>> and rebuild.
>>>>>>>
>>>>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>>>>>> can try applying the patch to that version, but I'd expect you'd run
>>>>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>>>>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>>>>>
>>>>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>>>>>>> wrote:
>>>>>>> > Thanks Bill for the response.
>>>>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no
>>>>>>> other
>>>>>>> > version, even 0.70 ?
>>>>>>> >
>>>>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>>>>>> >> you'll need PIG-1680:
>>>>>>> >>
>>>>>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>>>>>> >>
>>>>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com>
>>>>>>> wrote:
>>>>>>> >> > Hi,
>>>>>>> >> >
>>>>>>> >> > I am trying to extract data stored in HBase using Pig.
>>>>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job
>>>>>>> and got
>>>>>>> >> > correct results.
>>>>>>> >> > Now when I try to access HBase using the HbaseStorage the
>>>>>>> compiler just
>>>>>>> >> > hangs after
>>>>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>>>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>>>>>> >> > Even after waiting for several minutes it just remains there.
>>>>>>> There is
>>>>>>> >> > no
>>>>>>> >> > information in the log files too.
>>>>>>> >> >
>>>>>>> >> > Please help!
>>>>>>> >> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>> Sulabh Choudhury
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> SC
>>>
>>>
>>
>>
>> --
>>
>> SC
>>
>
>
--
Re: Cannot connect HBase to Pig
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Anything interesting in Map task logs?
As for your previous question -- just check out the branch-0.8, run
"ant jar-withouthadoop"
and use the the resulting jar.
On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury <su...@gmail.com>wrote:
> Dmitriy...
> I just found out that after 20 mins I get throws errors
>
> 2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
> org.apache.zookeeper.ClientCnxn - Server connection successful
> 2011-03-15 11:16:39,186 [Thread-4] INFO org.apache.hadoop.mapred.JobClient
> - Cleaning up the staging area
> hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
> 2011-03-15 11:16:39,188 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job null has failed! Stop running all dependent jobs
> 2011-03-15 11:16:39,194 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2011-03-15 11:16:39,203 [main] ERROR org.apache.pig.tools.pigstats.PigStats
> - ERROR 2997: Unable to recreate exception from backend error:
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
> trying to locate root region
> 2011-03-15 11:16:39,204 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
>
> Are the previous errors related or I am making heads-up and start looking
> into different direction now ?
>
> On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury <su...@gmail.com>wrote:
>
>> I am a little confused here....
>> Can you elaborate a little more on how to achieve the above ( Pig jar from
>> the pig-0.8 branch) and also then how to use the same to make it work with
>> Hbase 0.90 ?
>>
>> On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>
>>> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
>>> You can compile a Pig jar from the pig-0.8 branch, and it should work
>>> with 90+
>>>
>>> D
>>>
>>>
>>> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com>wrote:
>>>
>>>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
>>>> distribution.
>>>> Yes it does not crash it just hangs after the
>>>> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>>>>
>>>> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
>>>> all the Hunks failed...
>>>>
>>>> Also I see that
>>>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
>>>> connection to server localhost/127.0.0.1:2181
>>>> So that means that it is trying to connect to the correct (2181)
>>>> zookeeper port...
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>>
>>>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
>>>>> pretty fast.
>>>>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>>>>
>>>>> D
>>>>>
>>>>>
>>>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>>>>
>>>>>> What I'm saying is that if your version of HBase is 0.89.0 or greater,
>>>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>>>>> and rebuild.
>>>>>>
>>>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>>>>> can try applying the patch to that version, but I'd expect you'd run
>>>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>>>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>>>>
>>>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>>>>>> wrote:
>>>>>> > Thanks Bill for the response.
>>>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no
>>>>>> other
>>>>>> > version, even 0.70 ?
>>>>>> >
>>>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>>>>> >> you'll need PIG-1680:
>>>>>> >>
>>>>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>>>>> >>
>>>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>>>>>> >> > Hi,
>>>>>> >> >
>>>>>> >> > I am trying to extract data stored in HBase using Pig.
>>>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job
>>>>>> and got
>>>>>> >> > correct results.
>>>>>> >> > Now when I try to access HBase using the HbaseStorage the
>>>>>> compiler just
>>>>>> >> > hangs after
>>>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>>>>> >> > Even after waiting for several minutes it just remains there.
>>>>>> There is
>>>>>> >> > no
>>>>>> >> > information in the log files too.
>>>>>> >> >
>>>>>> >> > Please help!
>>>>>> >> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Sulabh Choudhury
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> SC
>>
>>
>
>
> --
>
> SC
>
Re: Cannot connect HBase to Pig
Posted by sulabh choudhury <su...@gmail.com>.
Dmitriy...
I just found out that after 20 mins I get throws errors
2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
org.apache.zookeeper.ClientCnxn - Server connection successful
2011-03-15 11:16:39,186 [Thread-4] INFO org.apache.hadoop.mapred.JobClient
- Cleaning up the staging area
hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
2011-03-15 11:16:39,188 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2011-03-15 11:16:39,194 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2011-03-15 11:16:39,203 [main] ERROR org.apache.pig.tools.pigstats.PigStats
- ERROR 2997: Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
trying to locate root region
2011-03-15 11:16:39,204 [main] ERROR
org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
Are the previous errors related or I am making heads-up and start looking
into different direction now ?
On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury <su...@gmail.com>wrote:
> I am a little confused here....
> Can you elaborate a little more on how to achieve the above ( Pig jar from
> the pig-0.8 branch) and also then how to use the same to make it work with
> Hbase 0.90 ?
>
> On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>
>> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
>> You can compile a Pig jar from the pig-0.8 branch, and it should work with
>> 90+
>>
>> D
>>
>>
>> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com>wrote:
>>
>>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
>>> distribution.
>>> Yes it does not crash it just hangs after the
>>> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>>>
>>> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
>>> all the Hunks failed...
>>>
>>> Also I see that
>>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
>>> connection to server localhost/127.0.0.1:2181
>>> So that means that it is trying to connect to the correct (2181)
>>> zookeeper port...
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>>
>>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
>>>> pretty fast.
>>>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>>>
>>>> D
>>>>
>>>>
>>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>>>
>>>>> What I'm saying is that if your version of HBase is 0.89.0 or greater,
>>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>>>> and rebuild.
>>>>>
>>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>>>> can try applying the patch to that version, but I'd expect you'd run
>>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>>>
>>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>>>>> wrote:
>>>>> > Thanks Bill for the response.
>>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no
>>>>> other
>>>>> > version, even 0.70 ?
>>>>> >
>>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>>>> >> you'll need PIG-1680:
>>>>> >>
>>>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>>>> >>
>>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>>>>> >> > Hi,
>>>>> >> >
>>>>> >> > I am trying to extract data stored in HBase using Pig.
>>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job and
>>>>> got
>>>>> >> > correct results.
>>>>> >> > Now when I try to access HBase using the HbaseStorage the compiler
>>>>> just
>>>>> >> > hangs after
>>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>>>> >> > Even after waiting for several minutes it just remains there.
>>>>> There is
>>>>> >> > no
>>>>> >> > information in the log files too.
>>>>> >> >
>>>>> >> > Please help!
>>>>> >> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> --
>>> Thanks and Regards,
>>> Sulabh Choudhury
>>>
>>>
>>
>
>
> --
>
> SC
>
>
--
SC
Re: Cannot connect HBase to Pig
Posted by sulabh choudhury <su...@gmail.com>.
I am a little confused here....
Can you elaborate a little more on how to achieve the above ( Pig jar from
the pig-0.8 branch) and also then how to use the same to make it work with
Hbase 0.90 ?
On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
> You can compile a Pig jar from the pig-0.8 branch, and it should work with
> 90+
>
> D
>
>
> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com>wrote:
>
>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
>> distribution.
>> Yes it does not crash it just hangs after the
>> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>>
>> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
>> all the Hunks failed...
>>
>> Also I see that
>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
>> connection to server localhost/127.0.0.1:2181
>> So that means that it is trying to connect to the correct (2181) zookeeper
>> port...
>>
>>
>>
>>
>>
>>
>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>>
>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
>>> pretty fast.
>>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>>
>>> D
>>>
>>>
>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>>
>>>> What I'm saying is that if your version of HBase is 0.89.0 or greater,
>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>>> and rebuild.
>>>>
>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>>> can try applying the patch to that version, but I'd expect you'd run
>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>>
>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>>>> wrote:
>>>> > Thanks Bill for the response.
>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
>>>> > version, even 0.70 ?
>>>> >
>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>>> >> you'll need PIG-1680:
>>>> >>
>>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>>> >>
>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>>>> >> > Hi,
>>>> >> >
>>>> >> > I am trying to extract data stored in HBase using Pig.
>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job and
>>>> got
>>>> >> > correct results.
>>>> >> > Now when I try to access HBase using the HbaseStorage the compiler
>>>> just
>>>> >> > hangs after
>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>>> >> > Even after waiting for several minutes it just remains there. There
>>>> is
>>>> >> > no
>>>> >> > information in the log files too.
>>>> >> >
>>>> >> > Please help!
>>>> >> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>>
>> --
>>
>> --
>> Thanks and Regards,
>> Sulabh Choudhury
>>
>>
>
--
SC
Re: Cannot connect HBase to Pig
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
You can compile a Pig jar from the pig-0.8 branch, and it should work with
90+
D
On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury <su...@gmail.com> wrote:
> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
> distribution.
> Yes it does not crash it just hangs after the
> "org.apache.zookeeper.ClientCnxn - Server connection successful"
>
> When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
> all the Hunks failed...
>
> Also I see that
> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
> connection to server localhost/127.0.0.1:2181
> So that means that it is trying to connect to the correct (2181) zookeeper
> port...
>
>
>
>
>
>
> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com>wrote:
>
>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
>> pretty fast.
>> Sulabh, are you sure that your hbase conf is on the Pig path?
>>
>> D
>>
>>
>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>>
>>> What I'm saying is that if your version of HBase is 0.89.0 or greater,
>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>>> and rebuild.
>>>
>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>>> can try applying the patch to that version, but I'd expect you'd run
>>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>>
>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>>> wrote:
>>> > Thanks Bill for the response.
>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
>>> > version, even 0.70 ?
>>> >
>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>>> wrote:
>>> >>
>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>>> >> you'll need PIG-1680:
>>> >>
>>> >> https://issues.apache.org/jira/browse/PIG-1680
>>> >>
>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>>> >> > Hi,
>>> >> >
>>> >> > I am trying to extract data stored in HBase using Pig.
>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job and
>>> got
>>> >> > correct results.
>>> >> > Now when I try to access HBase using the HbaseStorage the compiler
>>> just
>>> >> > hangs after
>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>>> >> > Even after waiting for several minutes it just remains there. There
>>> is
>>> >> > no
>>> >> > information in the log files too.
>>> >> >
>>> >> > Please help!
>>> >> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>
>>
>
>
> --
>
> --
> Thanks and Regards,
> Sulabh Choudhury
>
>
Re: Cannot connect HBase to Pig
Posted by sulabh choudhury <su...@gmail.com>.
I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
distribution.
Yes it does not crash it just hangs after the
"org.apache.zookeeper.ClientCnxn - Server connection successful"
When I apply the latest patch
(pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
all the Hunks failed...
Also I see that
Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
connection to server localhost/127.0.0.1:2181
So that means that it is trying to connect to the correct (2181) zookeeper
port...
On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
> pretty fast.
> Sulabh, are you sure that your hbase conf is on the Pig path?
>
> D
>
>
> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com>wrote:
>
>> What I'm saying is that if your version of HBase is 0.89.0 or greater,
>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
>> and rebuild.
>>
>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
>> can try applying the patch to that version, but I'd expect you'd run
>> into a lot of conflicts. You'd be better off to just upgrade Pig to
>> 0.8.0 and apply the patch, or just build Pig from the trunk.
>>
>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
>> wrote:
>> > Thanks Bill for the response.
>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
>> > version, even 0.70 ?
>> >
>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
>> wrote:
>> >>
>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
>> >> you'll need PIG-1680:
>> >>
>> >> https://issues.apache.org/jira/browse/PIG-1680
>> >>
>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > I am trying to extract data stored in HBase using Pig.
>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job and
>> got
>> >> > correct results.
>> >> > Now when I try to access HBase using the HbaseStorage the compiler
>> just
>> >> > hangs after
>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
>> >> > Even after waiting for several minutes it just remains there. There
>> is
>> >> > no
>> >> > information in the log files too.
>> >> >
>> >> > Please help!
>> >> >
>> >
>> >
>> >
>> >
>> >
>>
>
>
--
--
Thanks and Regards,
Sulabh Choudhury
Re: Cannot connect HBase to Pig
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
pretty fast.
Sulabh, are you sure that your hbase conf is on the Pig path?
D
On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham <bi...@gmail.com> wrote:
> What I'm saying is that if your version of HBase is 0.89.0 or greater,
> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
> and rebuild.
>
> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
> can try applying the patch to that version, but I'd expect you'd run
> into a lot of conflicts. You'd be better off to just upgrade Pig to
> 0.8.0 and apply the patch, or just build Pig from the trunk.
>
> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com>
> wrote:
> > Thanks Bill for the response.
> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
> > version, even 0.70 ?
> >
> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com>
> wrote:
> >>
> >> What version of Pig and what version of HBase? If HBase >= 0.89.0
> >> you'll need PIG-1680:
> >>
> >> https://issues.apache.org/jira/browse/PIG-1680
> >>
> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I am trying to extract data stored in HBase using Pig.
> >> > I got Pig to work on Hadoop and then also ran a sample M/R job and got
> >> > correct results.
> >> > Now when I try to access HBase using the HbaseStorage the compiler
> just
> >> > hangs after
> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
> >> > org.apache.zookeeper.ClientCnxn - Server connection successful
> >> > Even after waiting for several minutes it just remains there. There is
> >> > no
> >> > information in the log files too.
> >> >
> >> > Please help!
> >> >
> >
> >
> >
> >
> >
>
Re: Cannot connect HBase to Pig
Posted by Bill Graham <bi...@gmail.com>.
What I'm saying is that if your version of HBase is 0.89.0 or greater,
you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
and rebuild.
I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
can try applying the patch to that version, but I'd expect you'd run
into a lot of conflicts. You'd be better off to just upgrade Pig to
0.8.0 and apply the patch, or just build Pig from the trunk.
On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <su...@gmail.com> wrote:
> Thanks Bill for the response.
> Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
> version, even 0.70 ?
>
> On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com> wrote:
>>
>> What version of Pig and what version of HBase? If HBase >= 0.89.0
>> you'll need PIG-1680:
>>
>> https://issues.apache.org/jira/browse/PIG-1680
>>
>> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am trying to extract data stored in HBase using Pig.
>> > I got Pig to work on Hadoop and then also ran a sample M/R job and got
>> > correct results.
>> > Now when I try to access HBase using the HbaseStorage the compiler just
>> > hangs after
>> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
>> > org.apache.zookeeper.ClientCnxn - Server connection successful
>> > Even after waiting for several minutes it just remains there. There is
>> > no
>> > information in the log files too.
>> >
>> > Please help!
>> >
>
>
>
>
>
Re: Cannot connect HBase to Pig
Posted by sulabh choudhury <su...@gmail.com>.
Thanks Bill for the response.
Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
version, even 0.70 ?
On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <bi...@gmail.com> wrote:
> What version of Pig and what version of HBase? If HBase >= 0.89.0
> you'll need PIG-1680:
>
> https://issues.apache.org/jira/browse/PIG-1680
>
> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
> > Hi,
> >
> > I am trying to extract data stored in HBase using Pig.
> > I got Pig to work on Hadoop and then also ran a sample M/R job and got
> > correct results.
> > Now when I try to access HBase using the HbaseStorage the compiler just
> > hangs after
> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
> > org.apache.zookeeper.ClientCnxn - Server connection successful
> > Even after waiting for several minutes it just remains there. There is no
> > information in the log files too.
> >
> > Please help!
> >
>
Re: Cannot connect HBase to Pig
Posted by Bill Graham <bi...@gmail.com>.
What version of Pig and what version of HBase? If HBase >= 0.89.0
you'll need PIG-1680:
https://issues.apache.org/jira/browse/PIG-1680
On Mon, Mar 14, 2011 at 5:52 PM, SULABH <su...@gmail.com> wrote:
> Hi,
>
> I am trying to extract data stored in HBase using Pig.
> I got Pig to work on Hadoop and then also ran a sample M/R job and got
> correct results.
> Now when I try to access HBase using the HbaseStorage the compiler just
> hangs after
> 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
> org.apache.zookeeper.ClientCnxn - Server connection successful
> Even after waiting for several minutes it just remains there. There is no
> information in the log files too.
>
> Please help!
>