You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/05/21 20:53:00 UTC
[jira] [Resolved] (HIVE-19637) Add slow test report script to
testutils
[ https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prasanth Jayachandran resolved HIVE-19637.
------------------------------------------
Resolution: Fixed
Fix Version/s: 3.1.0
Committed to master. Thanks for the review!
> Add slow test report script to testutils
> ----------------------------------------
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
> Affects Versions: 3.1.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit test url. Would like to get that committed to testutils so that its useful for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
> ##########################################################################################################
> ██████████████████████████████████████████████████ 20806 TestCliDriver
> ███████████████████████ 9601 TestMiniLlapLocalCliDriver
> ███████████████████ 8210 TestSparkCliDriver
> ██████ 2744 TestMinimrCliDriver
> █████ 2262 TestEncryptedHDFSCliDriver
> ████ 2021 TestMiniSparkOnYarnCliDriver
> ████ 1808 TestHiveCli
> ███ 1566 TestMiniLlapCliDriver
> ███ 1345 TestReplicationScenarios
> ██ 1238 TestMiniDruidCliDriver
> ██ 940 TestNegativeCliDriver
> ██ 865 TestHBaseCliDriver
> █ 681 TestMiniTezCliDriver
> █ 555 TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374 TestBlobstoreCliDriver
> 328 TestNegativeMinimrCliDriver
> 302 TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240 TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
> ###############################################################################################################################################
> ██████████████████████████████████████████████████ 680 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █████████████████████████████████████████████ 623 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███████████████████████████████ 429 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███████████████████████████ 374 TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███████████████████████████ 374 TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> ████████████████████████ 330 TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █████████████████ 238 TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> ████████████████ 227 TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███████████████ 214 TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███████████████ 211 TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███████████████ 210 TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]
> ███████████████ 206 TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]
> ██████████████ 202 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]
> ██████████████ 198 TestCliDriver_testCliDriver[typechangetest]
> ████████████ 172 TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]
> ████████████ 164 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]
> ███████████ 158 TestCliDriver_testCliDriver[mm_all]
> ███████████ 155 TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
> ██████████ 145 TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
> ██████████ 141 TestMiniLlapCliDriver_testCliDriver[mm_all]
> ██████████ 140 TestMiniDruidCliDriver_testCliDriver[druidmini_mv]
> ██████████ 137 TestSparkCliDriver_testCliDriver[auto_join_filters]
> █████████ 135 TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]
> █████████ 124 TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]
> ████████ 121 TestCliDriver_testCliDriver[type_change_test_int_vectorized]
> {code}
> {code:title=text mode}
> $ python gen-report.py -b 11102
> Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
> TestCliDriver 20805.579
> TestMiniLlapLocalCliDriver 9601.362
> TestSparkCliDriver 8210.062
> TestMinimrCliDriver 2743.746
> TestEncryptedHDFSCliDriver 2261.866
> TestMiniSparkOnYarnCliDriver 2021.468
> TestHiveCli 1807.56
> TestMiniLlapCliDriver 1565.858
> TestReplicationScenarios 1345.344
> TestMiniDruidCliDriver 1237.776
> TestNegativeCliDriver 940.321
> TestHBaseCliDriver 864.707
> TestMiniTezCliDriver 681.457
> TestTxnCommands2WithSplitUpdateAndVectorization 555.382
> TestCompactor 543.173
> TestTxnCommands2 527.84
> TestStreaming 378.325
> TestBlobstoreCliDriver 374.134
> TestNegativeMinimrCliDriver 328.128
> TestTxnCommandsWithSplitUpdateAndVectorization 301.718
> TestHCatClient 301.324
> TestTxnCommands 298.694
> TestTxnLoadData 260.841
> TestAcidOnTez 258.262
> TestHBaseNegativeCliDriver 240.198
> Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two] 680.326
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators] 623.404
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part] 429.358
> TestSparkCliDriver_testCliDriver[vectorization_short_regress] 374.491
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress] 374.164
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition] 329.945
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5] 238.5
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert] 226.773
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables] 214.09
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1] 210.803
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5] 209.894
> TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid] 206.339
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge] 201.754
> TestCliDriver_testCliDriver[typechangetest] 198.371
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table] 172.267
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets] 163.617
> TestCliDriver_testCliDriver[mm_all] 158.401
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning] 155.255
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning] 145.481
> TestMiniLlapCliDriver_testCliDriver[mm_all] 141.369
> TestMiniDruidCliDriver_testCliDriver[druidmini_mv] 139.815
> TestSparkCliDriver_testCliDriver[auto_join_filters] 137.391
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3] 135.227
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3] 124.469
> TestCliDriver_testCliDriver[type_change_test_int_vectorized] 120.85
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)