You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (JIRA)" <ji...@apache.org> on 2012/09/05 03:54:07 UTC

[jira] [Created] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Kevin Wilfong created HIVE-3429:
-----------------------------------

             Summary: Bucket map join involving table with more than 1 partition column causes FileNotFoundException
                 Key: HIVE-3429
                 URL: https://issues.apache.org/jira/browse/HIVE-3429
             Project: Hive
          Issue Type: Bug
          Components: Query Processor
    Affects Versions: 0.10.0
            Reporter: Kevin Wilfong
            Assignee: Kevin Wilfong


Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   

 [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
    [junit] 	at java.io.FileInputStream.open(Native Method)
    [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
    [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
    [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
    [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
    [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
    [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
    [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
    [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
    [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
    [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
    [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
    [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
    [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
    [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
    [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
    [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
    [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
    [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
    [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
    [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
    [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
    [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
    [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
    [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
    [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
    [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
    [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
    [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
    [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
    [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
    [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
    [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
    [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
    [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
    [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
    [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
    [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
    [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Namit Jain updated HIVE-3429:
-----------------------------

       Resolution: Fixed
    Fix Version/s: 0.10.0
     Hadoop Flags: Reviewed
           Status: Resolved  (was: Patch Available)

committed. Thanks Kevin
                
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448847#comment-13448847 ] 

Hudson commented on HIVE-3429:
------------------------------

Integrated in Hive-trunk-h0.21 #1648 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1648/])
    HIVE-3429 Bucket map join involving table with more than 1 partition column causes 
FileNotFoundException (Kevin Wilfong via namit) (Revision 1381213)

     Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381213
Files : 
* /hive/trunk/build-common.xml
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BucketMapJoinContext.java
* /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin7.q
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin7.q.out

                
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448490#comment-13448490 ] 

Namit Jain commented on HIVE-3429:
----------------------------------

+1 

running tests
                
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kevin Wilfong updated HIVE-3429:
--------------------------------

    Status: Patch Available  (was: Open)
    
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kevin Wilfong updated HIVE-3429:
--------------------------------

    Attachment: HIVE-3429.1.patch.txt
    
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Navis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448400#comment-13448400 ] 

Navis commented on HIVE-3429:
-----------------------------

Oh, I've missed that. Thanks!
                
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-3429.1.patch.txt
>
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3429) Bucket map join involving table with more than 1 partition column causes FileNotFoundException

Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448398#comment-13448398 ] 

Kevin Wilfong commented on HIVE-3429:
-------------------------------------

https://reviews.facebook.net/D5151
                
> Bucket map join involving table with more than 1 partition column causes FileNotFoundException
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3429
>                 URL: https://issues.apache.org/jira/browse/HIVE-3429
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.10.0
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>
> Running a bucket map join exception on a table with more than one partition results in an exception is below.  This is because the partition spec is added to the file name, which unintentionally, produces a new subdirectory.   
>  [junit] java.io.FileNotFoundException: /Users/kevinwilfong/Documents/hive_driver_start/build/ql/scratchdir/local/hive_2012-09-04_18-35-38_679_3765928822897237252/-local-10002/HashTable-Stage-1/MapJoin-b-21-(ds=2008-04-08 (No such file or directory)
>     [junit] 	at java.io.FileInputStream.open(Native Method)
>     [junit] 	at java.io.FileInputStream.<init>(FileInputStream.java:120)
>     [junit] 	at org.apache.hadoop.hive.common.CompressionUtils.tar(CompressionUtils.java:59)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:398)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] java.lang.IllegalArgumentException: Can not create a Path from an empty string
>     [junit] 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>     [junit] 	at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:381)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:194)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:137)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:135)
>     [junit] 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1112)
>     [junit] 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     [junit] 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>     [junit] 	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:712)
>     [junit] 	at org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7(TestMinimrCliDriver.java:288)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
>     [junit] 	at junit.framework.TestCase.runTest(TestCase.java:168)
>     [junit] 	at junit.framework.TestCase.runBare(TestCase.java:134)
>     [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>     [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>     [junit] 	at junit.framework.TestResult.run(TestResult.java:113)
>     [junit] 	at junit.framework.TestCase.run(TestCase.java:124)
>     [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>     [junit] 	at junit.framework.TestSuite.run(TestSuite.java:227)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
>     [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
>     [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira