You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sergio Peña (JIRA)" <ji...@apache.org> on 2015/02/10 16:29:12 UTC
[jira] [Updated] (HIVE-9612) Turn off DEBUG logging for Lazy
Objects for tests
[ https://issues.apache.org/jira/browse/HIVE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergio Peña updated HIVE-9612:
------------------------------
Attachment: HIVE-9612.1.patch
> Turn off DEBUG logging for Lazy Objects for tests
> -------------------------------------------------
>
> Key: HIVE-9612
> URL: https://issues.apache.org/jira/browse/HIVE-9612
> Project: Hive
> Issue Type: Bug
> Reporter: Brock Noland
> Assignee: Sergio Peña
> Attachments: HIVE-9612.1.patch
>
>
> Our tests are collecting a tremendous amounts of logs:
> {noformat}
> [root@ip-10-152-185-204 TestRCFile]# pwd
> /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2709/succeeded/TestRCFile
> [root@ip-10-152-185-204 TestRCFile]# ls -lh hive.log
> -rw-r--r-- 1 hiveptest hiveptest 143M Feb 8 03:54 hive.log
> {noformat}
> Much of this logging is due to stack traces printed at DEBUG.
> {noformat}
> 2015-02-08 00:54:07,942 DEBUG [main]: lazy.LazyDouble (LazyDouble.java:init(55)) - Data not in the Double data type range so converted to null. Given data is :
> java.lang.NumberFormatException: empty String
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1011)
> at java.lang.Double.parseDouble(Double.java:540)
> at org.apache.hadoop.hive.serde2.lazy.LazyDouble.init(LazyDouble.java:51)
> at org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase$FieldInfo.uncheckedGetField(ColumnarStructBase.java:111)
> at org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase.getFieldsAsList(ColumnarStructBase.java:224)
> at org.apache.hadoop.hive.serde2.objectinspector.ColumnarStructObjectInspector.getStructFieldsDataAsList(ColumnarStructObjectInspector.java:76)
> at org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe.serialize(ColumnarSerDe.java:144)
> at org.apache.hadoop.hive.ql.io.TestRCFile.partialReadTest(TestRCFile.java:598)
> at org.apache.hadoop.hive.ql.io.TestRCFile.testWriteAndPartialRead(TestRCFile.java:417)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
> at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
> at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> 2015-02-08 00:54:17,992 DEBUG [main]: lazy.LazyPrimitive (LazyPrimitive.java:logExceptionMessage(81)) - Data not in the INT data type range so converted to null. Given data is :
> java.lang.Exception: For debugging purposes
> at org.apache.hadoop.hive.serde2.lazy.LazyPrimitive.logExceptionMessage(LazyPrimitive.java:81)
> at org.apache.hadoop.hive.serde2.lazy.LazyInteger.init(LazyInteger.java:59)
> at org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase$FieldInfo.uncheckedGetField(ColumnarStructBase.java:111)
> at org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase.getField(ColumnarStructBase.java:172)
> at org.apache.hadoop.hive.serde2.objectinspector.ColumnarStructObjectInspector.getStructFieldData(ColumnarStructObjectInspector.java:67)
> at org.apache.hadoop.hive.ql.io.TestRCFile.testSimpleReadAndWrite(TestRCFile.java:232)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
> at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
> at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> {noformat}
> I think we should disable logging for LazyPrimitive, LazyDouble, and friends for tests. I think we'd do that here:
> https://github.com/apache/hive/blob/trunk/ql/src/main/resources/hive-exec-log4j.properties
> and
> https://github.com/apache/hive/blob/trunk/data/conf/hive-log4j.properties
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)