You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kirill Kozlov (Jira)" <ji...@apache.org> on 2019/10/15 23:59:00 UTC
[jira] [Updated] (BEAM-8407) [SQL] Some Hive tests throw
NullPointerException, but get marked as passing (Direct Runner)
[ https://issues.apache.org/jira/browse/BEAM-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kirill Kozlov updated BEAM-8407:
--------------------------------
Description:
One of the tests with this issue:
{code:java}
./gradlew -p sdks/java/extensions/sql/ test --tests org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb --info
{code}
Stack trace:
{code:java}
[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException nulljava.lang.NullPointerException
at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77) ~[hive-exec-2.1.0.jar:2.1.0]
at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54) ~[hive-exec-2.1.0.jar:2.1.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126) [hive-metastore-2.1.0.jar:2.1.0]
at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369) [hive-hcatalog-core-2.1.0.jar:2.1.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150) [hive-metastore-2.1.0.jar:2.1.0]
at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?]
at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) [hive-hcatalog-core-2.1.0.jar:2.1.0]
at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) [hive-hcatalog-core-2.1.0.jar:2.1.0]
at org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?]
at org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?]
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
... more NullPointerExceptions{code}
was:
One of the tests with this issue:
{code:java}
./gradlew -p sdks/java/extensions/sql/ test --tests org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb --info
{code}
Stack trace:
{code:java}
[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException nulljava.lang.NullPointerException at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77) ~[hive-exec-2.1.0.jar:2.1.0] at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54) ~[hive-exec-2.1.0.jar:2.1.0] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126) [hive-metastore-2.1.0.jar:2.1.0] at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369) [hive-hcatalog-core-2.1.0.jar:2.1.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150) [hive-metastore-2.1.0.jar:2.1.0] at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?] at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) [hive-hcatalog-core-2.1.0.jar:2.1.0] at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) [hive-hcatalog-core-2.1.0.jar:2.1.0] at org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] at org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
... more NullPointerExceptions{code}
> [SQL] Some Hive tests throw NullPointerException, but get marked as passing (Direct Runner)
> -------------------------------------------------------------------------------------------
>
> Key: BEAM-8407
> URL: https://issues.apache.org/jira/browse/BEAM-8407
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Affects Versions: 2.15.0
> Reporter: Kirill Kozlov
> Priority: Major
> Labels: test
>
> One of the tests with this issue:
> {code:java}
> ./gradlew -p sdks/java/extensions/sql/ test --tests org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb --info
> {code}
> Stack trace:
> {code:java}
> [direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException nulljava.lang.NullPointerException
> at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77) ~[hive-exec-2.1.0.jar:2.1.0]
> at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54) ~[hive-exec-2.1.0.jar:2.1.0]
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126) [hive-metastore-2.1.0.jar:2.1.0]
> at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369) [hive-hcatalog-core-2.1.0.jar:2.1.0]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222]
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150) [hive-metastore-2.1.0.jar:2.1.0]
> at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?]
> at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) [hive-hcatalog-core-2.1.0.jar:2.1.0]
> at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) [hive-hcatalog-core-2.1.0.jar:2.1.0]
> at org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?]
> at org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?]
> at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
> at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
> at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
> at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
> ... more NullPointerExceptions{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)