You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Paul Yang (JIRA)" <ji...@apache.org> on 2010/05/28 23:58:39 UTC
[jira] Updated: (HIVE-1377) getPartitionDescFromPath() in
CombeHiveInputFormat should handle matching by path
[ https://issues.apache.org/jira/browse/HIVE-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1377:
----------------------------
Description:
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}
Then calling
{code}
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}
will throw an IOException because /user/warehouse/tableName is not a prefix of hdfs://host:9000/user/warehouse/tableName/abc. Currently, this is not an issue but will come up if CombineFileInputFormat is modified so what the scheme and authority are not stripped out when generating splits (see MAPREDUCE-1806).
The proposed solution is add a case where matching is done by just the path component of the URI's.
was:
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}
Then calling
{code}
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}
will throw an IOException. Currently, this is not an issue but will come up if CombineFileInputFormat is modified to so what the scheme and authority are not stripped out in MAPREDUCE-1806.
The proposed solution is add a case where matching is done by just the path component of the URI's.
> getPartitionDescFromPath() in CombeHiveInputFormat should handle matching by path
> ---------------------------------------------------------------------------------
>
> Key: HIVE-1377
> URL: https://issues.apache.org/jira/browse/HIVE-1377
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
>
> The use case is:
> {code}
> dir = hdfs://host:9000/user/warehouse/tableName/abc
> pathToPartitionInfo = {/user/warehouse/tableName : myPart}
> {code}
> Then calling
> {code}
> getPartitionDescFromPath(dir, pathToPartitionInfo)
> {code}
> will throw an IOException because /user/warehouse/tableName is not a prefix of hdfs://host:9000/user/warehouse/tableName/abc. Currently, this is not an issue but will come up if CombineFileInputFormat is modified so what the scheme and authority are not stripped out when generating splits (see MAPREDUCE-1806).
> The proposed solution is add a case where matching is done by just the path component of the URI's.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.