Release 1.1.0

[Claudio Martella <cl...@gmail.com>]

Hi all,

I've talked to Roman about the reasons why 1.1.0 fell in the backlog. I
think it was quite a shame we managed the last voting round.

Roman should be capable better than me to summarize and explain the status,
but for what I understand we should be able to release with builds for
hadoop 1 without problems, and only core and examples for hadoop 2.
Roman explained that the problems with some inputformat like HBase etc. are
due to hadoop 2, and not necessarily specific to giraph. Maybe Roman can
explain better.

So, I'm proposing to vote for 1.1.0 again. What do you think?

Best,
Claudio

-- 
   Claudio Martella

Re: Release 1.1.0

[Roman Shaposhnik <ro...@shaposhnik.org>]

Hi!

On Mon, Sep 15, 2014 at 10:26 AM, Claudio Martella
<cl...@gmail.com> wrote:
> Hi all,
>
> I've talked to Roman about the reasons why 1.1.0 fell in the backlog. I
> think it was quite a shame we managed the last voting round.
>
> Roman should be capable better than me to summarize and explain the status,
> but for what I understand we should be able to release with builds for
> hadoop 1 without problems, and only core and examples for hadoop 2.

Let me try to recap what is known so far.



Here's what I'm getting when running on a freshly
booted Amazon EC2 VM of Ubuntu 14.04:

$ umask 0022
$ curl http://people.apache.org/~rvs/giraph-1.1.0-RC0/giraph-dist-1.1.0-src.tar.bz2
| tar xjvf -
$ cd giraph-1.1.0
$ rm -rf ~/.m2
$ mvn --version
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
2014-08-11T20:58:10+00:00)
Maven home: /home/ubuntu/apache-maven-3.2.3
Java version: 1.7.0_65, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-35-generic", arch: "amd64", family: "unix"
$ mvn clean install -Phadoop_1
SUCCESS

$ rm -rf ~/.m2
$ mvn clean install -Phadoop_2
[INFO] Apache Giraph Parent ............................... SUCCESS [ 19.361 s]
[INFO] Apache Giraph Core ................................. SUCCESS [09:37 min]
[INFO] Apache Giraph .............. SUCCESS [10:14 min]

For everything else tests FAIL (expected)

$ mvn clean install -Phadoop_2 -DskipTests
SUCCESS

What this means is that I can't reproduce build failures
in hadoop_1 profile. There are *expected* test failures
in hadoop_2 profile on test and I suggest we RELNOTE
those.

All the other profiles may fail or not -- that wouldn't make
a difference. Or at least it wasn't supposed to (I thought
we agreed that hadoop_[12] were the only official
ones).

To summarize, hadoop 1 works all, hadoop 2 build works,
but tests fail ONLY for some of the I/O component (core
and example tests pass just fine with hadoop 2 as well).

Another point is that things like HBase, Hive/HCatalog when
compiled against Hadoop2 manually work just fine even in tests.

Why do I think we shouldn't just disable tests in hadoop_2
is the fact that they can be useful IFF you're willing
to build dependencies for them manually. IOW,  if you go a
trouble of actually building artifacts of HBase and Hive/HCatalog
against Hadoop 2 and have *those* artifacts in your local
.m2 -- than even tests pass.

This is exactly what Bigtop does and that's how I know
this stuff works with Hadoop 2 just fine. It just requires
a really complicated build process at this point.

> Roman explained that the problems with some inputformat like HBase etc. are
> due to hadoop 2, and not necessarily specific to giraph. Maybe Roman can
> explain better.
>
> So, I'm proposing to vote for 1.1.0 again. What do you think?

I think restarting the vote would be the right thing to do. Worst case
scenario -- we will uncover real issues and will spin up a new RC.

Thanks,
Roman.