You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2017/11/20 17:32:00 UTC
[jira] [Commented] (HIVE-18091) Failing tests of itests/qtest-spark
and itests/hive-unit on branch-1
[ https://issues.apache.org/jira/browse/HIVE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259526#comment-16259526 ]
Hive QA commented on HIVE-18091:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898492/HIVE-18091.2-branch-1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 62 failed/errored test(s), 9119 tests executed
*Failed tests:*
{noformat}
TestHs2HooksWithMiniKdc - did not produce a TEST-*.xml file (likely timed out) (batchId=451)
TestJdbcWithMiniKdc - did not produce a TEST-*.xml file (likely timed out) (batchId=448)
TestJdbcWithMiniKdcCookie - did not produce a TEST-*.xml file (likely timed out) (batchId=447)
TestJdbcWithMiniKdcSQLAuthBinary - did not produce a TEST-*.xml file (likely timed out) (batchId=445)
TestJdbcWithMiniKdcSQLAuthHttp - did not produce a TEST-*.xml file (likely timed out) (batchId=450)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=201)
[multi_insert_mixed.q,smb_mapjoin_4.q,timestamp_comparison.q,join_cond_pushdown_3.q,insert1.q,union_remove_10.q,mapreduce2.q,bucketmapjoin_negative.q,udf_in_file.q,skewjoinopt5.q,auto_join12.q,skewjoin.q,vector_count_distinct.q,smb_mapjoin_3.q,stats10.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=204)
[auto_join30.q,union_remove_1.q,ppd_outer_join2.q,date_udf.q,join16.q,smb_mapjoin_13.q,bucketmapjoin7.q,smb_mapjoin_18.q,join19.q,metadata_only_queries.q,union6.q,cbo_subq_in.q,vectorization_part.q,groupby3_map_multi_distinct.q,vectorized_timestamp_funcs.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=205)
[bucketmapjoin12.q,auto_join10.q,ptf_rcfile.q,join20.q,vector_elt.q,multi_insert.q,groupby_rollup1.q,ppd_join5.q,join_filters_overlap.q,vector_string_concat.q,join_empty.q,smb_mapjoin_6.q,auto_sortmerge_join_12.q,groupby_bigdata.q,innerjoin.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=206)
[tez_joins_explain.q,input17.q,ppd_gby_join.q,vectorized_rcfile_columnar.q,ppd_join.q,join_cond_pushdown_1.q,union_remove_6_subq.q,timestamp_3.q,load_dyn_part6.q,load_dyn_part9.q,multi_insert_gby2.q,vectorization_11.q,avro_compression_enabled_native.q,stats_noscan_2.q,transform1.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=210)
[escape_distributeby1.q,join9.q,groupby2.q,vectorization_pushdown.q,union_date.q,join_cond_pushdown_unqual3.q,join8.q,sample10.q,ppd_outer_join3.q,cross_product_check_1.q,statsfs.q,auto_sortmerge_join_2.q,auto_join_stats.q,input_part2.q,groupby_multi_single_reducer3.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=214)
[ppd_join4.q,load_dyn_part2.q,smb_mapjoin_7.q,vectorization_5.q,smb_mapjoin_2.q,ppd_join_filter.q,column_access_stats.q,stats0.q,vector_between_in.q,vectorized_string_funcs.q,vectorization_1.q,bucket_map_join_2.q,temp_table_join1.q,vectorized_case.q,stats_noscan_1.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=216)
[stats12.q,groupby4.q,union_top_level.q,groupby10.q,subquery_in.q,mapjoin_filter_on_outerjoin.q,stats14.q,auto_sortmerge_join_4.q,limit_partition_metadataonly.q,load_dyn_part4.q,union3.q,smb_mapjoin_14.q,groupby3_noskew_multi_distinct.q,stats18.q,union_remove_21.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=220)
[table_access_keys_stats.q,bucketmapjoin11.q,union_remove_9.q,mergejoins_mixed.q,join_nullsafe.q,stats8.q,auto_join28.q,skewjoinopt14.q,union17.q,vectorized_shufflejoin.q,groupby8_noskew.q,auto_sortmerge_join_10.q,groupby11.q,skewjoinopt11.q,load_dyn_part11.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=221)
[script_pipe.q,auto_join24.q,ptf_seqfile.q,union_remove_23.q,filter_join_breaktask.q,parallel_join0.q,join_thrift.q,vectorized_mapjoin.q,groupby8.q,union4.q,auto_join5.q,smb_mapjoin_20.q,groupby_multi_insert_common_distinct.q,join6.q,union_remove_16.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
[join36.q,union_remove_15.q,smb_mapjoin_10.q,bucket_map_join_tez1.q,temp_table.q,union_remove_13.q,auto_join8.q,auto_join6.q,auto_join0.q,vectorization_17.q,auto_join_stats2.q,skewjoin_union_remove_1.q,union16.q,auto_join32.q,union_remove_20.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_mapjoin (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_drop_database_removes_partition_dirs (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_drop_table_removes_partition_dirs (batchId=132)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization_acid (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_dependency (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_dependency2 (batchId=109)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby10 (batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby11 (batchId=111)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby8 (batchId=114)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin5 (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table_gb1 (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_fast_stats (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_string_decimal (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing (batchId=82)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_drop_partition (batchId=437)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join1 (batchId=452)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join4 (batchId=452)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join5 (batchId=452)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynamic_partition_pruning (batchId=175)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_groupby2 (batchId=180)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_stats_only_null (batchId=174)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_smb_empty (batchId=176)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union_fast_stats (batchId=181)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning (batchId=175)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_groupby2 (batchId=143)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_index_bitmap_auto (batchId=138)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx_cbo_2 (batchId=147)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import (batchId=153)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_right_side_join (batchId=153)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dynamic_rdd_cache (batchId=213)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby1 (batchId=198)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats9 (batchId=202)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_only_null (batchId=203)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_13 (batchId=211)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_short_regress (batchId=211)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=380)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testSplitGenFailure (batchId=594)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testTableProperties (batchId=379)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=269)
org.apache.hive.jdbc.TestSSL.testSSLVersion (batchId=408)
org.apache.hive.minikdc.TestHiveAuthFactory.testStartTokenManagerForDBTokenStore (batchId=449)
org.apache.hive.minikdc.TestHiveAuthFactory.testStartTokenManagerForMemoryTokenStore (batchId=449)
org.apache.hive.minikdc.TestMiniHiveKdc.testLogin (batchId=446)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7922/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7922/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7922/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 62 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898492 - PreCommit-HIVE-Build
> Failing tests of itests/qtest-spark and itests/hive-unit on branch-1
> --------------------------------------------------------------------
>
> Key: HIVE-18091
> URL: https://issues.apache.org/jira/browse/HIVE-18091
> Project: Hive
> Issue Type: Bug
> Components: Test, Testing Infrastructure
> Reporter: Daniel Voros
> Assignee: Daniel Voros
> Attachments: HIVE-18091.1-branch-1.patch, HIVE-18091.2-branch-1.patch
>
>
> Seen this while looking at ptest results for HIVE-17947 but is probably an older issue.
> Tests under itests/qtest-spark and itests/hive-unit fail when trying to execute the download-spark plugin with:
> {code}
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Hive Integration - Unit Tests 1.3.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-it-unit ---
> [INFO]
> [INFO] --- maven-antrun-plugin:1.7:run (download-spark) @ hive-it-unit ---
> [INFO] Executing tasks
> main:
> [exec] + /bin/pwd
> [exec] + BASE_DIR=./target
> [exec] + HIVE_ROOT=./target/../../../
> [exec] + DOWNLOAD_DIR=./../thirdparty
> [exec] + mkdir -p ./../thirdparty
> [exec] /home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit
> [exec] + download http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz spark
> [exec] + url=http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz
> [exec] + finalName=spark
> [exec] ++ basename http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.5.0-bin-hadoop2-without-hive.tgz
> [exec] + tarName=spark-1.5.0-bin-hadoop2-without-hive.tgz
> [exec] + rm -rf ./target/spark
> [exec] + [[ ! -f ./../thirdparty/spark-1.5.0-bin-hadoop2-without-hive.tgz ]]
> [exec] + tar -zxf ./../thirdparty/spark-1.5.0-bin-hadoop2-without-hive.tgz -C ./target
> [exec] + mv ./target/spark-1.5.0-bin-hadoop2-without-hive ./target/spark
> [exec] + cp -f ./target/../../..//data/conf/spark/log4j.properties ./target/spark/conf/
> [exec] + sed '/package /d' /data/hiveptest/working/apache-github-branch-1-source/itests/../contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleAdd.java
> [exec] sed: can't read /data/hiveptest/working/apache-github-branch-1-source/itests/../contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleAdd.java: No such file or directory
> [exec] + javac -cp /data/hiveptest/working/maven/org/apache/hive/hive-exec/1.3.0-SNAPSHOT/hive-exec-1.3.0-SNAPSHOT.jar /tmp/UDFExampleAdd.java -d /tmp
> [exec] + jar -cf /tmp/udfexampleadd-1.0.jar -C /tmp UDFExampleAdd.class
> [exec] /tmp/UDFExampleAdd.class : no such file or directory
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 8.376s
> [INFO] Finished at: Mon Nov 06 22:29:39 UTC 2017
> [INFO] Final Memory: 18M/241M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (download-spark) on project hive-it-unit: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec dir="/home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit" executable="bash" failonerror="true">... @ 4:141 in /home/hiveptest/35.192.99.254-hiveptest-0/apache-github-branch-1-source/itests/hive-unit/target/antrun/build-main.xml
> {code}
> {{mvn antrun:run@download-spark}} passes when run locally, so I guess it might be an issue with the way wer're executing ptests.
> Full list of classes with the same error:
> {code}
> TestAcidOnTez
> TestAdminUser
> TestAuthorizationPreEventListener
> TestAuthzApiEmbedAuthorizerInEmbed
> TestAuthzApiEmbedAuthorizerInRemote
> TestBeeLineWithArgs
> TestCLIAuthzSessionContext
> TestClearDanglingScratchDir
> TestClientSideAuthorizationProvider
> TestCompactor
> TestCreateUdfEntities
> TestCustomAuthentication
> TestDBTokenStore
> TestDDLWithRemoteMetastoreSecondNamenode
> TestDynamicSerDe
> TestEmbeddedHiveMetaStore
> TestEmbeddedThriftBinaryCLIService
> TestFilterHooks
> TestFolderPermissions
> TestHS2AuthzContext
> TestHS2AuthzSessionContext
> TestHS2ClearDanglingScratchDir
> TestHS2ImpersonationWithRemoteMS
> TestHiveAuthorizerCheckInvocation
> TestHiveAuthorizerShowFilters
> TestHiveHistory
> TestHiveMetaStoreTxns
> TestHiveMetaStoreWithEnvironmentContext
> TestHiveMetaTool
> TestHiveServer2
> TestHiveServer2SessionTimeout
> TestHiveSessionImpl
> TestHs2Hooks
> TestJdbcDriver2
> TestJdbcMetadataApiAuth
> TestJdbcWithLocalClusterSpark
> TestJdbcWithMiniHS2
> TestJdbcWithMiniMr
> TestJdbcWithSQLAuthUDFBlacklist
> TestJdbcWithSQLAuthorization
> TestLocationQueries
> TestMTQueries
> TestMarkPartition
> TestMarkPartitionRemote
> TestMetaStoreAuthorization
> TestMetaStoreConnectionUrlHook
> TestMetaStoreEndFunctionListener
> TestMetaStoreEventListener
> TestMetaStoreEventListenerOnlyOnCommit
> TestMetaStoreInitListener
> TestMetaStoreListenersError
> TestMetaStoreMetrics
> TestMetaStoreUtils
> TestMetastoreAuthorizationProvider
> TestMetastoreVersion
> TestMiniHS2
> TestMiniSparkOnYarnCliDriver
> TestMultiAuthorizationPreEventListener
> TestMultiSessionsHS2WithLocalClusterSpark
> TestNoSaslAuth
> TestOperationLoggingAPIWithMr
> TestOperationLoggingAPIWithTez
> TestPartitionNameWhitelistValidation
> TestPasswordWithConfig
> TestRemoteHiveMetaStore
> TestRemoteHiveMetaStoreIpAddress
> TestRemoteUGIHiveMetaStoreIpAddress
> TestRetryingHMSHandler
> TestSSL
> TestSchemaTool
> TestSemanticAnalyzerHookLoading
> TestSerdeWithFieldComments
> TestServerSpecificConfig
> TestServiceDiscovery
> TestSetUGIOnBothClientServer
> TestSetUGIOnOnlyClient
> TestSetUGIOnOnlyServer
> TestSparkCliDriver
> TestSparkNegativeCliDriver
> TestStorageBasedClientSideAuthorizationProvider
> TestStorageBasedMetastoreAuthorizationDrops
> TestStorageBasedMetastoreAuthorizationProvider
> TestStorageBasedMetastoreAuthorizationReads
> TestThriftBinaryCLIService
> TestThriftHttpCLIService
> TestZooKeeperTokenStore
> {code}
> These are all the test classes of the two modules with the exception of TestHiveMetastore.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)