You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/05/21 18:58:00 UTC
[jira] [Created] (HIVE-19637) Add slow test report script to
testutils
Prasanth Jayachandran created HIVE-19637:
--------------------------------------------
Summary: Add slow test report script to testutils
Key: HIVE-19637
URL: https://issues.apache.org/jira/browse/HIVE-19637
Project: Hive
Issue Type: Sub-task
Components: Test
Affects Versions: 3.1.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran
Wrote the attached utility script to find top K slow tests from precommit test url. Would like to get that committed to testutils so that its useful for everyone.
{code:title=ascii mode}
$ python gen-report.py -b 11102 -a
Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
##########################################################################################################
██████████████████████████████████████████████████ 20806 TestCliDriver
███████████████████████ 9601 TestMiniLlapLocalCliDriver
███████████████████ 8210 TestSparkCliDriver
██████ 2744 TestMinimrCliDriver
█████ 2262 TestEncryptedHDFSCliDriver
████ 2021 TestMiniSparkOnYarnCliDriver
████ 1808 TestHiveCli
███ 1566 TestMiniLlapCliDriver
███ 1345 TestReplicationScenarios
██ 1238 TestMiniDruidCliDriver
██ 940 TestNegativeCliDriver
██ 865 TestHBaseCliDriver
█ 681 TestMiniTezCliDriver
█ 555 TestTxnCommands2WithSplitUpdateAndVectorization
█ 543 TestCompactor
█ 528 TestTxnCommands2
378 TestStreaming
374 TestBlobstoreCliDriver
328 TestNegativeMinimrCliDriver
302 TestTxnCommandsWithSplitUpdateAndVectorization
301 TestHCatClient
299 TestTxnCommands
261 TestTxnLoadData
258 TestAcidOnTez
240 TestHBaseNegativeCliDriver
Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
###############################################################################################################################################
██████████████████████████████████████████████████ 680 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
█████████████████████████████████████████████ 623 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
███████████████████████████████ 429 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
███████████████████████████ 374 TestSparkCliDriver_testCliDriver[vectorization_short_regress]
███████████████████████████ 374 TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
████████████████████████ 330 TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
█████████████████ 238 TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
████████████████ 227 TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
███████████████ 214 TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
███████████████ 211 TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
███████████████ 210 TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]
███████████████ 206 TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]
██████████████ 202 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]
██████████████ 198 TestCliDriver_testCliDriver[typechangetest]
████████████ 172 TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]
████████████ 164 TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]
███████████ 158 TestCliDriver_testCliDriver[mm_all]
███████████ 155 TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
██████████ 145 TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
██████████ 141 TestMiniLlapCliDriver_testCliDriver[mm_all]
██████████ 140 TestMiniDruidCliDriver_testCliDriver[druidmini_mv]
██████████ 137 TestSparkCliDriver_testCliDriver[auto_join_filters]
█████████ 135 TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]
█████████ 124 TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]
████████ 121 TestCliDriver_testCliDriver[type_change_test_int_vectorized]
{code}
{code:title=text mode}
$ python gen-report.py -b 11102
Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
TestCliDriver 20805.579
TestMiniLlapLocalCliDriver 9601.362
TestSparkCliDriver 8210.062
TestMinimrCliDriver 2743.746
TestEncryptedHDFSCliDriver 2261.866
TestMiniSparkOnYarnCliDriver 2021.468
TestHiveCli 1807.56
TestMiniLlapCliDriver 1565.858
TestReplicationScenarios 1345.344
TestMiniDruidCliDriver 1237.776
TestNegativeCliDriver 940.321
TestHBaseCliDriver 864.707
TestMiniTezCliDriver 681.457
TestTxnCommands2WithSplitUpdateAndVectorization 555.382
TestCompactor 543.173
TestTxnCommands2 527.84
TestStreaming 378.325
TestBlobstoreCliDriver 374.134
TestNegativeMinimrCliDriver 328.128
TestTxnCommandsWithSplitUpdateAndVectorization 301.718
TestHCatClient 301.324
TestTxnCommands 298.694
TestTxnLoadData 260.841
TestAcidOnTez 258.262
TestHBaseNegativeCliDriver 240.198
Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two] 680.326
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators] 623.404
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part] 429.358
TestSparkCliDriver_testCliDriver[vectorization_short_regress] 374.491
TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress] 374.164
TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition] 329.945
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5] 238.5
TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert] 226.773
TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables] 214.09
TestMiniLlapCliDriver_testCliDriver[unionDistinct_1] 210.803
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5] 209.894
TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid] 206.339
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge] 201.754
TestCliDriver_testCliDriver[typechangetest] 198.371
TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table] 172.267
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets] 163.617
TestCliDriver_testCliDriver[mm_all] 158.401
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning] 155.255
TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning] 145.481
TestMiniLlapCliDriver_testCliDriver[mm_all] 141.369
TestMiniDruidCliDriver_testCliDriver[druidmini_mv] 139.815
TestSparkCliDriver_testCliDriver[auto_join_filters] 137.391
TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3] 135.227
TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3] 124.469
TestCliDriver_testCliDriver[type_change_test_int_vectorized] 120.85
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)