[jira] [Updated] (HBASE-8147) Add Integration Test for "The HCat Scenario"

["Nick Dimiduk (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nick Dimiduk updated HBASE-8147:
--------------------------------

    Attachment: HBASE-8147-WIP.patch

I've replicated the behavior of HCat under this scenario, but it looks like there's a problem with the way I'm attaching dependency jars in the test. Running the HCat code against patched HBase does not produce this error message. Attached is a WIP patch that introduces an integration test for the above ticket. It depends on first applying the trunk patch attached to that ticket.

{noformat}
attempt_201303191253_0001_m_000001_0: java.io.IOException: Distributed cache entry arrays have different lengths: 45, 58, 45, 45
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.JobLocalizer.downloadPrivateCacheObjects(JobLocalizer.java:316)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.JobLocalizer.downloadPrivateCache(JobLocalizer.java:351)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:388)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:202)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1228)
attempt_201303191253_0001_m_000001_0:   at java.security.AccessController.doPrivileged(Native Method)
attempt_201303191253_0001_m_000001_0:   at javax.security.auth.Subject.doAs(Subject.java:396)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1203)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1118)
attempt_201303191253_0001_m_000001_0:   at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2430)
attempt_201303191253_0001_m_000001_0:   at java.lang.Thread.run(Thread.java:680)
{noformat}

To run locally, apply the two patches, set up a pseudo-distributed cluster, and launch the job with:

{noformat}
$ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop \
  jar hbase-it/target/hbase-it-0.97-SNAPSHOT-tests.jar \
  org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv
{noformat}
                
> Add Integration Test for "The HCat Scenario"
> --------------------------------------------
>
>                 Key: HBASE-8147
>                 URL: https://issues.apache.org/jira/browse/HBASE-8147
>             Project: HBase
>          Issue Type: Test
>          Components: mapreduce, test
>    Affects Versions: 0.95.0, 0.98.0
>            Reporter: Nick Dimiduk
>         Attachments: HBASE-8147-WIP.patch
>
>
> HBASE-8140 needs an integration test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira