[jira] [Commented] (HIVE-4750) Fix TestCliDriver.list_bucket_dml_{6,7,8}.q on 0.23

["Prasanth J (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749020#comment-13749020 ] 

Prasanth J commented on HIVE-4750:
----------------------------------

The issue is not related to inconsistent File.length() on Mac vs Linux. But it is related to FileInputFormat in map reduce. The map reduce JIRA for this https://issues.apache.org/jira/browse/MAPREDUCE-3193. 

The problem is when there is only one nested directory (under a partition directory in hive), the FileInputFormat considers it as a file and then returns the size of it (which apparently is a directory). Its true that Mac vs Linux returns different lengths for directories. But thats not the intended behavior. The intended behavior of this test is to recurse through the subdirectory of HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME and get the total file lengths of all the files contained in the sub-directory. Since the golden file is generated with hadoop version that didn't contain MAPREDUCE-3193 fix, the results were wrong and were OS dependent.

I regenerated the golden file for the failing test and tested it under Mac and CentOS 6. The test cases are passing with the expected results on both OSes. 

For further verification, the list_bucket_dml_6.q file has comments with the list of file structure/details which can be used to validate the results. 

Also, test cases list_bucket_dml_{7,8}.q are passing for me in the local setup.
                
> Fix TestCliDriver.list_bucket_dml_{6,7,8}.q on 0.23
> ---------------------------------------------------
>
>                 Key: HIVE-4750
>                 URL: https://issues.apache.org/jira/browse/HIVE-4750
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Brock Noland
>            Assignee: Prasanth J
>
> Removing 6,7,8 from the scope of HIVE-4746.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira