[jira] [Updated] (HIVE-14767) Migrate slow MiniMr tests to faster options

["Prasanth Jayachandran (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prasanth Jayachandran updated HIVE-14767:
-----------------------------------------
    Attachment: HIVE-14767.1.patch

> Migrate slow MiniMr tests to faster options
> -------------------------------------------
>
>                 Key: HIVE-14767
>                 URL: https://issues.apache.org/jira/browse/HIVE-14767
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14767.1.patch
>
>
> After analyzing the latest test results https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/
> there are many Minimr tests that are slow individually. There are also some tests that does not have to run in minimr, some tests are testing hdfs related features that can be migrated to MiniLlap (some are already in MiniSpark). Since we have removed hadoop20 now some MiniMr tests can be moved to TestCliDriver. We should keep the absolute minimum number of tests in minimr and move other slow test to minillap (spark already has most of these tests).
> These are the individual test runtimes for minimr tests
> {code}
> QFiles	TestMinimrCliDriver elapsed time (Build #1055)
> infer_bucket_sort_reducers_power_two.q	356.211
> bucket5.q	257.589
> infer_bucket_sort_bucketed_table.q	251.557
> bucketizedhiveinputformat.q	249.755
> infer_bucket_sort_map_operators.q	249.484
> index_bitmap_auto.q	212.9
> infer_bucket_sort_dyn_part.q	204.401
> skewjoin_onesideskew.q	198.428
> index_bitmap3.q	194.221
> truncate_column_buckets.q	193.966
> auto_sortmerge_join_16.q	171.094
> schemeAuthority.q	170.839
> leftsemijoin_mr.q	150.493
> join_acid_non_acid.q	143.232
> empty_dir_in_table.q	136.463
> bucketmapjoin6.q	129.786
> reduce_deduplicate.q	116.455
> exchgpartition2lel.q	115.678
> bucket_many.q	104.316
> infer_bucket_sort_merge.q	100.976
> quotedid_smb.q	95.801
> external_table_with_space_in_location_path.q	80.265
> schemeAuthority2.q	72.64
> root_dir_external_table.q	72.171
> parallel_orderby.q	71.662
> disable_merge_for_bucketing.q	69.25
> bucket_num_reducers.q	67.618
> remote_script.q	67.029
> groupby2.q	66.72
> bucket4.q	66.02
> udf_using.q	64.985
> file_with_header_footer.q	42.986
> infer_bucket_sort_num_buckets.q	38.795
> scriptfile1.q	35.317
> bucket6.q	34.963
> non_native_window_udf.q	34.129
> join1.q	33.831
> bucketmapjoin7.q	33.71
> bucket_num_reducers2.q	30.645
> list_bucket_dml_10.q	25.751
> insert_dir_distcp.q	25.723
> uber_reduce.q	16.481
> load_hdfs_file_with_space_in_the_name.q	3.899
> table_nonprintable.q	2.097
> load_fs2.q	1.947
> input16_cc.q	0.916
> temp_table_external.q	0.751
> import_exported_table.q	0.508
> scriptfile1_win.q	0.119
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)