You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Suraj Nayak <sn...@gmail.com> on 2015/05/08 21:35:57 UTC
Help needed in debugging HCatLoader for Pig
Hi All,
I have set up hive source code as per
HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse
<https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?>
documentation
and successfully imported the project into Eclipse.
Now,If I want to debug the hive-hcatalog-pig-adapter/
*org.apache.hive.hcatalog.pig.HCatLoader* class, I tried to
run /hive-hcatalog-pig-adapter/*org.apache.hive.hcatalog.pig.TestHCatLoader*
by enabling few debug points.
Am getting following error when I Right-Click on *TestHCatLoader* and
Select *JUnit Test:*
FAILED: NullPointerException Non-local session path expected to be non-null
15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local
session path expected to be non-null
java.lang.NullPointerException: Non-local session path expected to be
non-null
While being in eclipse am able to run the goal "*test
-Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*" from itests
directory successfully. But the debug point is never caught during the
testing. How can I line by line debug any class such as this HCatLoader
loader class from eclipse?
Any help is appreciated!
--
Thanks
Suraj Nayak M
Re: Help needed in debugging HCatLoader for Pig
Posted by Suraj Nayak <sn...@gmail.com>.
Adding screenshot of Maven Goal Debug configuration for reference:
[image: Inline image 1]
Thanks!
On Fri, May 8, 2015 at 12:35 PM, Suraj Nayak <sn...@gmail.com> wrote:
> Hi All,
>
> I have set up hive source code as per
> HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse
> <https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?> documentation
> and successfully imported the project into Eclipse.
>
> Now,If I want to debug the hive-hcatalog-pig-adapter/
> *org.apache.hive.hcatalog.pig.HCatLoader* class, I tried to
> run /hive-hcatalog-pig-adapter/
> *org.apache.hive.hcatalog.pig.TestHCatLoader* by enabling few debug
> points.
>
> Am getting following error when I Right-Click on *TestHCatLoader* and
> Select *JUnit Test:*
>
> FAILED: NullPointerException Non-local session path expected to be
> non-null
> 15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local
> session path expected to be non-null
> java.lang.NullPointerException: Non-local session path expected to be
> non-null
>
> While being in eclipse am able to run the goal "*test
> -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*" from
> itests directory successfully. But the debug point is never caught during
> the testing. How can I line by line debug any class such as this HCatLoader
> loader class from eclipse?
>
> Any help is appreciated!
>
> --
> Thanks
> Suraj Nayak M
>
--
Thanks
Suraj Nayak M
Re: Help needed in debugging HCatLoader for Pig
Posted by Suraj Nayak <sn...@gmail.com>.
Thanks a lot Hari. Am able to debug the code now through eclipse. Yes, it
was setup issue in my Mac. Worked fine after I set up Hive source in
CentOS-6.
On Fri, May 8, 2015 at 2:13 PM, Hari Subramaniyan <
hsubramaniyan@hortonworks.com> wrote:
> Hi Suraj,
>
> These tests have been executed successfully of late in the precommit build
> runs, for e.g.
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/lastStableBuild/testReport/org.apache.hive.hcatalog.pig/TestHCatLoader/ and
> in my local machine.
>
> So most likely, this is a setup issue.
>
>
> Thanks
>
> Hari
> ------------------------------
> *From:* Suraj Nayak <sn...@gmail.com>
> *Sent:* Friday, May 08, 2015 1:14 PM
> *To:* user@hive.apache.org; Hari Subramaniyan
> *Subject:* Re: Help needed in debugging HCatLoader for Pig
>
> Thanks a lot Hari,
>
> I ran the below command from hive-project root:
>
> *mvn -Dmaven.surefire.debug="-Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent
> -Djava.compiler=NONE" test
> -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*
>
> and connected from eclipse via remote debugging option successfully.
> This solved the problem of debugging.
>
> Now am facing one more issue. There were some test failures. Am looking
> at it, however, I will paste the error below if it is common issue(so I
> don't reinvent the wheel again):
>
> Listening for transport dt_socket at address: 8000
> Running org.apache.hive.hcatalog.pig.TestHCatLoader
> Tests run: 60, Failures: 9, Errors: 22, Skipped: 3, Time elapsed:
> 1,428.185 sec <<< FAILURE! - in org.apache.hive.hcatalog.pig.TestHCatLoader
> testProjectionsBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time
> elapsed: 8.82 sec <<< ERROR!
> org.apache.pig.impl.logicalLayer.FrontendException: Unable to open
> iterator for alias Y2
> at org.apache.pig.PigServer.openIterator(PigServer.java:872)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic(TestHCatLoader.java:462)
>
> testConvertBooleanToInt[0](org.apache.hive.hcatalog.pig.TestHCatLoader)
> Time elapsed: 4.926 sec <<< ERROR!
> org.apache.pig.impl.logicalLayer.FrontendException: Unable to open
> iterator for alias data
> at org.apache.pig.PigServer.openIterator(PigServer.java:872)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt(TestHCatLoader.java:571)
>
> testReadDataBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time
> elapsed: 22.188 sec <<< ERROR!
> org.apache.pig.impl.logicalLayer.FrontendException: Unable to open
> iterator for alias X
> at org.apache.pig.PigServer.openIterator(PigServer.java:872)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic(TestHCatLoader.java:294)
>
> testColumnarStorePushdown[0](org.apache.hive.hcatalog.pig.TestHCatLoader)
> Time elapsed: 3.953 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown(TestHCatLoader.java:504)
>
> Thanks!
>
> On Fri, May 8, 2015 at 12:40 PM, Hari Subramaniyan <
> hsubramaniyan@hortonworks.com> wrote:
>
>> Use -Dmaven.surefire.debug option and attach to port 5005 (default),
>> more info can be found here :
>>
>>
>> http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html
>>
>>
>>
>> Thanks
>>
>> Hari
>> ------------------------------
>> *From:* Suraj Nayak <sn...@gmail.com>
>> *Sent:* Friday, May 08, 2015 12:35 PM
>> *To:* user@hive.apache.org
>> *Subject:* Help needed in debugging HCatLoader for Pig
>>
>> Hi All,
>>
>> I have set up hive source code as per
>> HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse
>> <https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?> documentation
>> and successfully imported the project into Eclipse.
>>
>> Now,If I want to debug the hive-hcatalog-pig-adapter/
>> *org.apache.hive.hcatalog.pig.HCatLoader* class, I tried to
>> run /hive-hcatalog-pig-adapter/
>> *org.apache.hive.hcatalog.pig.TestHCatLoader* by enabling few debug
>> points.
>>
>> Am getting following error when I Right-Click on *TestHCatLoader* and
>> Select *JUnit Test:*
>>
>> FAILED: NullPointerException Non-local session path expected to be
>> non-null
>> 15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local
>> session path expected to be non-null
>> java.lang.NullPointerException: Non-local session path expected to be
>> non-null
>>
>> While being in eclipse am able to run the goal "*test
>> -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*" from
>> itests directory successfully. But the debug point is never caught during
>> the testing. How can I line by line debug any class such as this HCatLoader
>> loader class from eclipse?
>>
>> Any help is appreciated!
>>
>> --
>> Thanks
>> Suraj Nayak M
>>
>>
>>
>
>
> --
> Thanks
> Suraj Nayak M
>
--
Thanks
Suraj Nayak M
Re: Help needed in debugging HCatLoader for Pig
Posted by Hari Subramaniyan <hs...@hortonworks.com>.
Hi Suraj,
These tests have been executed successfully of late in the precommit build runs, for e.g. http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/lastStableBuild/testReport/org.apache.hive.hcatalog.pig/TestHCatLoader/ and in my local machine.
So most likely, this is a setup issue.
Thanks
Hari
________________________________
From: Suraj Nayak <sn...@gmail.com>
Sent: Friday, May 08, 2015 1:14 PM
To: user@hive.apache.org; Hari Subramaniyan
Subject: Re: Help needed in debugging HCatLoader for Pig
Thanks a lot Hari,
I ran the below command from hive-project root:
mvn -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE" test -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2
and connected from eclipse via remote debugging option successfully. This solved the problem of debugging.
Now am facing one more issue. There were some test failures. Am looking at it, however, I will paste the error below if it is common issue(so I don't reinvent the wheel again):
Listening for transport dt_socket at address: 8000
Running org.apache.hive.hcatalog.pig.TestHCatLoader
Tests run: 60, Failures: 9, Errors: 22, Skipped: 3, Time elapsed: 1,428.185 sec <<< FAILURE! - in org.apache.hive.hcatalog.pig.TestHCatLoader
testProjectionsBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time elapsed: 8.82 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator for alias Y2
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic(TestHCatLoader.java:462)
testConvertBooleanToInt[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time elapsed: 4.926 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator for alias data
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt(TestHCatLoader.java:571)
testReadDataBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time elapsed: 22.188 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator for alias X
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic(TestHCatLoader.java:294)
testColumnarStorePushdown[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time elapsed: 3.953 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown(TestHCatLoader.java:504)
Thanks!
On Fri, May 8, 2015 at 12:40 PM, Hari Subramaniyan <hs...@hortonworks.com>> wrote:
Use -Dmaven.surefire.debug option and attach to port 5005 (default), more info can be found here :
http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html
Thanks
Hari
________________________________
From: Suraj Nayak <sn...@gmail.com>>
Sent: Friday, May 08, 2015 12:35 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Help needed in debugging HCatLoader for Pig
Hi All,
I have set up hive source code as per HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse<https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?> documentation and successfully imported the project into Eclipse.
Now,If I want to debug the hive-hcatalog-pig-adapter/org.apache.hive.hcatalog.pig.HCatLoader class, I tried to run /hive-hcatalog-pig-adapter/org.apache.hive.hcatalog.pig.TestHCatLoader by enabling few debug points.
Am getting following error when I Right-Click on TestHCatLoader and Select JUnit Test:
FAILED: NullPointerException Non-local session path expected to be non-null
15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local session path expected to be non-null
java.lang.NullPointerException: Non-local session path expected to be non-null
While being in eclipse am able to run the goal "test -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2" from itests directory successfully. But the debug point is never caught during the testing. How can I line by line debug any class such as this HCatLoader loader class from eclipse?
Any help is appreciated!
--
Thanks
Suraj Nayak M
--
Thanks
Suraj Nayak M
Re: Help needed in debugging HCatLoader for Pig
Posted by Suraj Nayak <sn...@gmail.com>.
Thanks a lot Hari,
I ran the below command from hive-project root:
*mvn -Dmaven.surefire.debug="-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent
-Djava.compiler=NONE" test
-Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*
and connected from eclipse via remote debugging option successfully. This
solved the problem of debugging.
Now am facing one more issue. There were some test failures. Am looking at
it, however, I will paste the error below if it is common issue(so I don't
reinvent the wheel again):
Listening for transport dt_socket at address: 8000
Running org.apache.hive.hcatalog.pig.TestHCatLoader
Tests run: 60, Failures: 9, Errors: 22, Skipped: 3, Time elapsed: 1,428.185
sec <<< FAILURE! - in org.apache.hive.hcatalog.pig.TestHCatLoader
testProjectionsBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time
elapsed: 8.82 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator
for alias Y2
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at
org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic(TestHCatLoader.java:462)
testConvertBooleanToInt[0](org.apache.hive.hcatalog.pig.TestHCatLoader)
Time elapsed: 4.926 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator
for alias data
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at
org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt(TestHCatLoader.java:571)
testReadDataBasic[0](org.apache.hive.hcatalog.pig.TestHCatLoader) Time
elapsed: 22.188 sec <<< ERROR!
org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator
for alias X
at org.apache.pig.PigServer.openIterator(PigServer.java:872)
at
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic(TestHCatLoader.java:294)
testColumnarStorePushdown[0](org.apache.hive.hcatalog.pig.TestHCatLoader)
Time elapsed: 3.953 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown(TestHCatLoader.java:504)
Thanks!
On Fri, May 8, 2015 at 12:40 PM, Hari Subramaniyan <
hsubramaniyan@hortonworks.com> wrote:
> Use -Dmaven.surefire.debug option and attach to port 5005 (default),
> more info can be found here :
>
>
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html
>
>
>
> Thanks
>
> Hari
> ------------------------------
> *From:* Suraj Nayak <sn...@gmail.com>
> *Sent:* Friday, May 08, 2015 12:35 PM
> *To:* user@hive.apache.org
> *Subject:* Help needed in debugging HCatLoader for Pig
>
> Hi All,
>
> I have set up hive source code as per
> HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse
> <https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?> documentation
> and successfully imported the project into Eclipse.
>
> Now,If I want to debug the hive-hcatalog-pig-adapter/
> *org.apache.hive.hcatalog.pig.HCatLoader* class, I tried to
> run /hive-hcatalog-pig-adapter/
> *org.apache.hive.hcatalog.pig.TestHCatLoader* by enabling few debug
> points.
>
> Am getting following error when I Right-Click on *TestHCatLoader* and
> Select *JUnit Test:*
>
> FAILED: NullPointerException Non-local session path expected to be
> non-null
> 15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local
> session path expected to be non-null
> java.lang.NullPointerException: Non-local session path expected to be
> non-null
>
> While being in eclipse am able to run the goal "*test
> -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2*" from
> itests directory successfully. But the debug point is never caught during
> the testing. How can I line by line debug any class such as this HCatLoader
> loader class from eclipse?
>
> Any help is appreciated!
>
> --
> Thanks
> Suraj Nayak M
>
>
>
--
Thanks
Suraj Nayak M
Re: Help needed in debugging HCatLoader for Pig
Posted by Hari Subramaniyan <hs...@hortonworks.com>.
Use ?-Dmaven.surefire.debug option and attach to port 5005 (default), more info can be found here :
http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html?
Thanks
Hari
________________________________
From: Suraj Nayak <sn...@gmail.com>
Sent: Friday, May 08, 2015 12:35 PM
To: user@hive.apache.org
Subject: Help needed in debugging HCatLoader for Pig
Hi All,
I have set up hive source code as per HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse<https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIimportintoEclipse?> documentation and successfully imported the project into Eclipse.
Now,If I want to debug the hive-hcatalog-pig-adapter/org.apache.hive.hcatalog.pig.HCatLoader class, I tried to run /hive-hcatalog-pig-adapter/org.apache.hive.hcatalog.pig.TestHCatLoader by enabling few debug points.
Am getting following error when I Right-Click on TestHCatLoader and Select JUnit Test:
FAILED: NullPointerException Non-local session path expected to be non-null
15/05/08 12:24:37 ERROR ql.Driver: FAILED: NullPointerException Non-local session path expected to be non-null
java.lang.NullPointerException: Non-local session path expected to be non-null
While being in eclipse am able to run the goal "test -Dtest=org.apache.hive.hcatalog.pig.TestHCatLoader -Phadoop-2" from itests directory successfully. But the debug point is never caught during the testing. How can I line by line debug any class such as this HCatLoader loader class from eclipse?
Any help is appreciated!
--
Thanks
Suraj Nayak M