HBase and OpenJDK

[Thomas Koch <th...@koch.ro>]

Hi,

I'm working on the Debian package for HBase. On Debian, the default JVM is 
OpenJDK, since it's free and the SUN JVM isn't much loved by the debian-java 
team.
Now when running HBase on OpenJDK 1.6.0_18 the master frontend shows the 
warning:
"Your current JVM version 1.6.0_18 is known to be unstable with HBase. Please 
see the HBase wiki  for details."

Has anybody already made experiences with running Hadoop/HBase on OpenJDK? 
Does the above warning also apply for this JVM?
For a broader addoption of Hadoop/HBase I'd recommend to do also testing on 
OpenJDK since I wouldn't be sure if the proprietary SUN JVM will remain much 
longer in Debian. (Don't know about Red Hat.)

Best regards,

Thomas Koch, http://www.koch.ro

Re: HBase and OpenJDK

[Jean-Daniel Cryans <jd...@apache.org>]

> Nah, the purpose of Java is so we can all have the fun of understanding
> garbage collection algorithms!

I giggled.

J-D

Re: HBase and OpenJDK

[Todd Lipcon <to...@cloudera.com>]

On Fri, Apr 23, 2010 at 9:30 AM, tsuna <ts...@gmail.com> wrote:

> On Fri, Apr 23, 2010 at 9:06 AM, Todd Lipcon <to...@cloudera.com> wrote:
> > So, from the perspective of those developing and supporting the software,
> it makes far
> > more sense to have a single supported JVM.
>
> Isn't this defeating the whole purpose of using Java in the first place?
>  :)
>
>
Nah, the purpose of Java is so we can all have the fun of understanding
garbage collection algorithms!

-Todd


>  --
> Benoit "tsuna" Sigoure
> Software Engineer @ www.StumbleUpon.com
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: HBase and OpenJDK

[tsuna <ts...@gmail.com>]

On Fri, Apr 23, 2010 at 9:06 AM, Todd Lipcon <to...@cloudera.com> wrote:
> So, from the perspective of those developing and supporting the software, it makes far
> more sense to have a single supported JVM.

Isn't this defeating the whole purpose of using Java in the first place?  :)

-- 
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: HBase and OpenJDK

[Todd Lipcon <to...@cloudera.com>]

Hi Thomas,

While I love open source and some of the ideals that Debian tries to push,
please also understand from the perspective of those developing Hadoop and
HBase that supporting (ie guaranteeing no problems) both JDKs is a large
burden. Hadoop and HBase exercise *every* part of the JVM, and we've seen
countless cases where a specific patch update of Sun's JVM (by far the most
widely used) has caused very sporadic crashes. For example, on 1.6.0u18, DNs
and RSs seem to crash after an average of about 10 days. On 1.6.0u5 there
was a bug that Yahoo found only on sorts of >100TB.

While we can certainly run the unit tests on OpenJDK and say "hey, look, it
works", that's not the same as running a 40 node cluster for a week under
load and verifying that the JVM didn't fall apart in between. So, from the
perspective of those developing and supporting the software, it makes far
more sense to have a single supported JVM. If someone is seeing strange
stability issues on OpenJDK, the very first thing I would tell them to do is
switch to Sun's. Of course if the Sun JDK stops being supported by Oracle, I
guess we'll have to reconsider :)

(I'm not trying to discourage you from packaging HBase, I'm just warning
that if a Debian user comes to the list and is running different versions of
dependency jars or a strange JDK, the we'll probably make them switch over
to what we QA on before providing much help)

-Todd



On Fri, Apr 23, 2010 at 7:50 AM, Thomas Koch <th...@koch.ro> wrote:

> Hi,
>
> I'm working on the Debian package for HBase. On Debian, the default JVM is
> OpenJDK, since it's free and the SUN JVM isn't much loved by the
> debian-java
> team.
> Now when running HBase on OpenJDK 1.6.0_18 the master frontend shows the
> warning:
> "Your current JVM version 1.6.0_18 is known to be unstable with HBase.
> Please
> see the HBase wiki  for details."
>
> Has anybody already made experiences with running Hadoop/HBase on OpenJDK?
> Does the above warning also apply for this JVM?
> For a broader addoption of Hadoop/HBase I'd recommend to do also testing on
> OpenJDK since I wouldn't be sure if the proprietary SUN JVM will remain
> much
> longer in Debian. (Don't know about Red Hat.)
>
> Best regards,
>
> Thomas Koch, http://www.koch.ro
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: HBase and OpenJDK

[Stack <st...@duboce.net>]

Hey Thomas:

The warning is because the 1.6.0_18 SUN JDK is broke, at least for
hadoop and hbase (random JVM crashes).  I'm not sure if OpenJDK
suffers the same problems.  Let me test.  I'll report back.

St.Ack

On Fri, Apr 23, 2010 at 7:50 AM, Thomas Koch <th...@koch.ro> wrote:
> Hi,
>
> I'm working on the Debian package for HBase. On Debian, the default JVM is
> OpenJDK, since it's free and the SUN JVM isn't much loved by the debian-java
> team.
> Now when running HBase on OpenJDK 1.6.0_18 the master frontend shows the
> warning:
> "Your current JVM version 1.6.0_18 is known to be unstable with HBase. Please
> see the HBase wiki  for details."
>
> Has anybody already made experiences with running Hadoop/HBase on OpenJDK?
> Does the above warning also apply for this JVM?
> For a broader addoption of Hadoop/HBase I'd recommend to do also testing on
> OpenJDK since I wouldn't be sure if the proprietary SUN JVM will remain much
> longer in Debian. (Don't know about Red Hat.)
>
> Best regards,
>
> Thomas Koch, http://www.koch.ro
>