[jira] [Updated] (HBASE-6581) Build with hadoop.profile=3.0

["Eric Charles (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Charles updated HBASE-6581:
--------------------------------

    Attachment: HBASE-6581-20130821.patch

Attached a new patch which work with current trunk.

Upon the poms, we also have now to deal with the protobuf 2.5.0 (used by hadoop, hbase uses 2.4.1, the communication between mixed protobufs is not possible) and also the change of FSDataOutputStream#sync method (now #hsync).

1. To solve the protobuf issue, I propose to remove the generated classes from hbase-protocol/src/main/java and let the generation occur at build-time with hadoop-maven-plugin. This implies that the correct protoc version must be installed on the build machine (2.4.1 for pre-h3 to ensure backwards compatibility, 2.5.0 for h3).

2. I have added a hbase-hadoop3-compat module that reuses the hbase-hadoop2-compat to avoid duplicating all the metrics classes. To allow the build, I have quickly implemented the call the the (h)sync via introspection (IMHO subject to review/optimization - another option would be to duplicate the compat classes, or to create extra modules only for that specific case).

The patch allows me to build/assemble with the various profiles. hadoop3 profile assembly works well for me on top of an hadoop trunk build.

There are a few test failures, but I also had some failures on my env even with a vanilla trunk (will further look at these).

                
> Build with hadoop.profile=3.0
> -----------------------------
>
>                 Key: HBASE-6581
>                 URL: https://issues.apache.org/jira/browse/HBASE-6581
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eric Charles
>         Attachments: HBASE-6581-1.patch, HBASE-6581-20130821.patch, HBASE-6581-2.patch, HBASE-6581.diff, HBASE-6581.diff
>
>
> Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common).
> I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile.
> Please tell me if that's ok to go this way.
> Thx, Eric
> [1]
> $ mvn clean install -Dhadoop.profile=3.0
> [INFO] Scanning for projects...
> [ERROR] The build could not read 3 projects -> [Help 1]
> [ERROR]   
> [ERROR]   The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21
> [ERROR]   
> [ERROR]   The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21
> [ERROR]   
> [ERROR]   The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21
> [ERROR]     'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21
> [ERROR] 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira