You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Ted Yu <yu...@gmail.com> on 2011/09/29 03:56:51 UTC
HBase trunk on 0.23 Was: Update on hadoop-0.23
Minor correction to Todd's report, currently HBase TRUNK doesn't compile
against 0.23 (
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
):
[ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
cannot find symbol
[ERROR] symbol : class FSConstants
[ERROR] location: package org.apache.hadoop.hdfs.protocol
[ERROR]
Cheers
On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi all,
>
> Just an update from the HBase side: I've run some cluster tests on
> HDFS 0.23 (as of about a month ago) and it generally works well.
> Performance for some workloads is ~2x due to HDFS-941, and can be
> improved a bit more if I finish HDFS-2080 in time. I did not do
> extensive failure testing (to stress the new append/sync code) but I
> do plan to do that in the coming months.
>
> HBase trunk can compile against 0.23 by using -Dhadoop23 on the maven
> build. Currently some 15 or so tests are failing - the following HBase
> JIRA tracks those issues:
> https://issues.apache.org/jira/browse/HBASE-4254
>
> (these may be indicative of HDFS side bugs)
>
> Any help there from the community would be appreciated!
>
> -Todd
>
> On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> > Hi Arun!
> >
> > Thanks for the quick reply!
> >
> > I'm sorry if I had too many questions in my original email, but I can't
> find
> > an answer to my "integration tests" question. Could you, please, share
> > a URL with us where I can find out more about them?
> >
> > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
> >> # We made changes to Pig - rather we got help from the Pig team,
> particularly Daniel.
> >>
> >> So, we plan to work through the rest of the stack - Hive, Oozie etc.
> very soon and we'll
> >> depend on updated releases from the individual projects.
> >
> > Do we have any kinds of commitment from downstream projects as far as
> those
> > updates are concerned? Are they targeting these changes as part of point
> (patch)
> > release of an already released version (like Pig 0.9.X for example) or
> > will it be
> > part of a brand new major release?
> >
> > Thanks,
> > Roman.
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Konstantin Boudnik <co...@apache.org>.
I don't see hadoop-hdfs artifact being downloaded during the build. I see
hadoop-hdfs-test, but not hdfs itself. This is likely to be a problem because
location of the interface in question is the same as it was back in 0.22
Cos
On Wed, Sep 28, 2011 at 07:16PM, Roman Shaposhnik wrote:
> On Wed, Sep 28, 2011 at 6:56 PM, Ted Yu <yu...@gmail.com> wrote:
> > Minor correction to Todd's report, currently HBase TRUNK doesn't compile
> > against 0.23 (
> > https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
> > ):
> >
> > [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
> > cannot find symbol
> > [ERROR] symbol ═: class FSConstants
> > [ERROR] location: package org.apache.hadoop.hdfs.protocol
> > [ERROR]
>
> Yup. That's exactly what I'm seeing:
> http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/COMPONENT=hbase,label=lucid/2/console
>
> Which is weird. The delta between .22 and .23 is not that great as far
> as HDFS APIs are concerned,
> so what makes it work for one, but not the other?
>
> Thanks,
> Roman.
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Konstantin Boudnik <co...@apache.org>.
I don't see hadoop-hdfs artifact being downloaded during the build. I see
hadoop-hdfs-test, but not hdfs itself. This is likely to be a problem because
location of the interface in question is the same as it was back in 0.22
Cos
On Wed, Sep 28, 2011 at 07:16PM, Roman Shaposhnik wrote:
> On Wed, Sep 28, 2011 at 6:56 PM, Ted Yu <yu...@gmail.com> wrote:
> > Minor correction to Todd's report, currently HBase TRUNK doesn't compile
> > against 0.23 (
> > https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
> > ):
> >
> > [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
> > cannot find symbol
> > [ERROR] symbol ═: class FSConstants
> > [ERROR] location: package org.apache.hadoop.hdfs.protocol
> > [ERROR]
>
> Yup. That's exactly what I'm seeing:
> http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/COMPONENT=hbase,label=lucid/2/console
>
> Which is weird. The delta between .22 and .23 is not that great as far
> as HDFS APIs are concerned,
> so what makes it work for one, but not the other?
>
> Thanks,
> Roman.
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Sep 28, 2011 at 6:56 PM, Ted Yu <yu...@gmail.com> wrote:
> Minor correction to Todd's report, currently HBase TRUNK doesn't compile
> against 0.23 (
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
> ):
>
> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
> cannot find symbol
> [ERROR] symbol : class FSConstants
> [ERROR] location: package org.apache.hadoop.hdfs.protocol
> [ERROR]
Yup. That's exactly what I'm seeing:
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/COMPONENT=hbase,label=lucid/2/console
Which is weird. The delta between .22 and .23 is not that great as far
as HDFS APIs are concerned,
so what makes it work for one, but not the other?
Thanks,
Roman.
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Harsh J <ha...@cloudera.com>.
I've filed https://issues.apache.org/jira/browse/HBASE-4510 to handle
this. Please leave a comment there on what of the two approaches would
be desirable by HBase?
On Thu, Sep 29, 2011 at 10:35 AM, Harsh J <ha...@cloudera.com> wrote:
> Ted,
>
> (-general@)
>
> This is due to some renames carried out and backported from
> https://issues.apache.org/jira/browse/HDFS-1620. I will file HBASE
> jiras appropriately for this (or is there one already?).
>
> I wasn't aware that the non public symbols were being used downstream.
> But nothing should be broken beyond a few renames.
>
> On Thu, Sep 29, 2011 at 7:26 AM, Ted Yu <yu...@gmail.com> wrote:
>> Minor correction to Todd's report, currently HBase TRUNK doesn't compile
>> against 0.23 (
>> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
>> ):
>>
>> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
>> cannot find symbol
>> [ERROR] symbol : class FSConstants
>> [ERROR] location: package org.apache.hadoop.hdfs.protocol
>> [ERROR]
>>
>> Cheers
>>
>> On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com> wrote:
>>
>>> Hi all,
>>>
>>> Just an update from the HBase side: I've run some cluster tests on
>>> HDFS 0.23 (as of about a month ago) and it generally works well.
>>> Performance for some workloads is ~2x due to HDFS-941, and can be
>>> improved a bit more if I finish HDFS-2080 in time. I did not do
>>> extensive failure testing (to stress the new append/sync code) but I
>>> do plan to do that in the coming months.
>>>
>>> HBase trunk can compile against 0.23 by using -Dhadoop23 on the maven
>>> build. Currently some 15 or so tests are failing - the following HBase
>>> JIRA tracks those issues:
>>> https://issues.apache.org/jira/browse/HBASE-4254
>>>
>>> (these may be indicative of HDFS side bugs)
>>>
>>> Any help there from the community would be appreciated!
>>>
>>> -Todd
>>>
>>> On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>> > Hi Arun!
>>> >
>>> > Thanks for the quick reply!
>>> >
>>> > I'm sorry if I had too many questions in my original email, but I can't
>>> find
>>> > an answer to my "integration tests" question. Could you, please, share
>>> > a URL with us where I can find out more about them?
>>> >
>>> > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy <ac...@hortonworks.com>
>>> wrote:
>>> >> # We made changes to Pig - rather we got help from the Pig team,
>>> particularly Daniel.
>>> >>
>>> >> So, we plan to work through the rest of the stack - Hive, Oozie etc.
>>> very soon and we'll
>>> >> depend on updated releases from the individual projects.
>>> >
>>> > Do we have any kinds of commitment from downstream projects as far as
>>> those
>>> > updates are concerned? Are they targeting these changes as part of point
>>> (patch)
>>> > release of an already released version (like Pig 0.9.X for example) or
>>> > will it be
>>> > part of a brand new major release?
>>> >
>>> > Thanks,
>>> > Roman.
>>> >
>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
>>>
>>
>
>
>
> --
> Harsh J
>
--
Harsh J
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Harsh J <ha...@cloudera.com>.
Ted,
(-general@)
This is due to some renames carried out and backported from
https://issues.apache.org/jira/browse/HDFS-1620. I will file HBASE
jiras appropriately for this (or is there one already?).
I wasn't aware that the non public symbols were being used downstream.
But nothing should be broken beyond a few renames.
On Thu, Sep 29, 2011 at 7:26 AM, Ted Yu <yu...@gmail.com> wrote:
> Minor correction to Todd's report, currently HBase TRUNK doesn't compile
> against 0.23 (
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
> ):
>
> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
> cannot find symbol
> [ERROR] symbol : class FSConstants
> [ERROR] location: package org.apache.hadoop.hdfs.protocol
> [ERROR]
>
> Cheers
>
> On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com> wrote:
>
>> Hi all,
>>
>> Just an update from the HBase side: I've run some cluster tests on
>> HDFS 0.23 (as of about a month ago) and it generally works well.
>> Performance for some workloads is ~2x due to HDFS-941, and can be
>> improved a bit more if I finish HDFS-2080 in time. I did not do
>> extensive failure testing (to stress the new append/sync code) but I
>> do plan to do that in the coming months.
>>
>> HBase trunk can compile against 0.23 by using -Dhadoop23 on the maven
>> build. Currently some 15 or so tests are failing - the following HBase
>> JIRA tracks those issues:
>> https://issues.apache.org/jira/browse/HBASE-4254
>>
>> (these may be indicative of HDFS side bugs)
>>
>> Any help there from the community would be appreciated!
>>
>> -Todd
>>
>> On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> > Hi Arun!
>> >
>> > Thanks for the quick reply!
>> >
>> > I'm sorry if I had too many questions in my original email, but I can't
>> find
>> > an answer to my "integration tests" question. Could you, please, share
>> > a URL with us where I can find out more about them?
>> >
>> > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >> # We made changes to Pig - rather we got help from the Pig team,
>> particularly Daniel.
>> >>
>> >> So, we plan to work through the rest of the stack - Hive, Oozie etc.
>> very soon and we'll
>> >> depend on updated releases from the individual projects.
>> >
>> > Do we have any kinds of commitment from downstream projects as far as
>> those
>> > updates are concerned? Are they targeting these changes as part of point
>> (patch)
>> > release of an already released version (like Pig 0.9.X for example) or
>> > will it be
>> > part of a brand new major release?
>> >
>> > Thanks,
>> > Roman.
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
--
Harsh J
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Ted Yu <yu...@gmail.com>.
That' right.
Harsh has logged HBASE-4510 and is in the process of helping us :-)
In the future, such change should be better communicated to downstream
projects.
Cheers
On Wed, Sep 28, 2011 at 11:40 PM, Uma Maheswara Rao G 72686 <
maheswara@huawei.com> wrote:
> Hi Ted,
>
> Recently Constants filenames has been renamed in Trunk and 23 branch.
> Looks that will be the cause for this issue.
> I remember we have done that in HDFS-1620.
>
> current codebase has FsConstants.java.
>
> Regards,
> Uma
>
> ----- Original Message -----
> From: Ted Yu <yu...@gmail.com>
> Date: Thursday, September 29, 2011 7:27 am
> Subject: HBase trunk on 0.23 Was: Update on hadoop-0.23
> To: general@hadoop.apache.org
> Cc: bigtop-dev@incubator.apache.org
>
> > Minor correction to Todd's report, currently HBase TRUNK doesn't
> > compileagainst 0.23 (
> > https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-
> > Hadoop-23/42/console
> > ):
> >
> > [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-
> >
> 23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]cannot
> find symbol
> > [ERROR] symbol : class FSConstants
> > [ERROR] location: package org.apache.hadoop.hdfs.protocol
> > [ERROR]
> >
> > Cheers
> >
> > On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com>
> > wrote:
> > > Hi all,
> > >
> > > Just an update from the HBase side: I've run some cluster tests on
> > > HDFS 0.23 (as of about a month ago) and it generally works well.
> > > Performance for some workloads is ~2x due to HDFS-941, and can be
> > > improved a bit more if I finish HDFS-2080 in time. I did not do
> > > extensive failure testing (to stress the new append/sync code)
> > but I
> > > do plan to do that in the coming months.
> > >
> > > HBase trunk can compile against 0.23 by using -Dhadoop23 on the
> > maven> build. Currently some 15 or so tests are failing - the
> > following HBase
> > > JIRA tracks those issues:
> > > https://issues.apache.org/jira/browse/HBASE-4254
> > >
> > > (these may be indicative of HDFS side bugs)
> > >
> > > Any help there from the community would be appreciated!
> > >
> > > -Todd
> > >
> > > On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik
> > <rv...@apache.org> wrote:
> > > > Hi Arun!
> > > >
> > > > Thanks for the quick reply!
> > > >
> > > > I'm sorry if I had too many questions in my original email,
> > but I can't
> > > find
> > > > an answer to my "integration tests" question. Could you,
> > please, share
> > > > a URL with us where I can find out more about them?
> > > >
> > > > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy
> > <ac...@hortonworks.com>> wrote:
> > > >> # We made changes to Pig - rather we got help from the Pig team,
> > > particularly Daniel.
> > > >>
> > > >> So, we plan to work through the rest of the stack - Hive,
> > Oozie etc.
> > > very soon and we'll
> > > >> depend on updated releases from the individual projects.
> > > >
> > > > Do we have any kinds of commitment from downstream projects as
> > far as
> > > those
> > > > updates are concerned? Are they targeting these changes as
> > part of point
> > > (patch)
> > > > release of an already released version (like Pig 0.9.X for
> > example) or
> > > > will it be
> > > > part of a brand new major release?
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> > >
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> > >
> >
>
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Ted Yu <yu...@gmail.com>.
That' right.
Harsh has logged HBASE-4510 and is in the process of helping us :-)
In the future, such change should be better communicated to downstream
projects.
Cheers
On Wed, Sep 28, 2011 at 11:40 PM, Uma Maheswara Rao G 72686 <
maheswara@huawei.com> wrote:
> Hi Ted,
>
> Recently Constants filenames has been renamed in Trunk and 23 branch.
> Looks that will be the cause for this issue.
> I remember we have done that in HDFS-1620.
>
> current codebase has FsConstants.java.
>
> Regards,
> Uma
>
> ----- Original Message -----
> From: Ted Yu <yu...@gmail.com>
> Date: Thursday, September 29, 2011 7:27 am
> Subject: HBase trunk on 0.23 Was: Update on hadoop-0.23
> To: general@hadoop.apache.org
> Cc: bigtop-dev@incubator.apache.org
>
> > Minor correction to Todd's report, currently HBase TRUNK doesn't
> > compileagainst 0.23 (
> > https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-
> > Hadoop-23/42/console
> > ):
> >
> > [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-
> >
> 23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]cannot
> find symbol
> > [ERROR] symbol : class FSConstants
> > [ERROR] location: package org.apache.hadoop.hdfs.protocol
> > [ERROR]
> >
> > Cheers
> >
> > On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com>
> > wrote:
> > > Hi all,
> > >
> > > Just an update from the HBase side: I've run some cluster tests on
> > > HDFS 0.23 (as of about a month ago) and it generally works well.
> > > Performance for some workloads is ~2x due to HDFS-941, and can be
> > > improved a bit more if I finish HDFS-2080 in time. I did not do
> > > extensive failure testing (to stress the new append/sync code)
> > but I
> > > do plan to do that in the coming months.
> > >
> > > HBase trunk can compile against 0.23 by using -Dhadoop23 on the
> > maven> build. Currently some 15 or so tests are failing - the
> > following HBase
> > > JIRA tracks those issues:
> > > https://issues.apache.org/jira/browse/HBASE-4254
> > >
> > > (these may be indicative of HDFS side bugs)
> > >
> > > Any help there from the community would be appreciated!
> > >
> > > -Todd
> > >
> > > On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik
> > <rv...@apache.org> wrote:
> > > > Hi Arun!
> > > >
> > > > Thanks for the quick reply!
> > > >
> > > > I'm sorry if I had too many questions in my original email,
> > but I can't
> > > find
> > > > an answer to my "integration tests" question. Could you,
> > please, share
> > > > a URL with us where I can find out more about them?
> > > >
> > > > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy
> > <ac...@hortonworks.com>> wrote:
> > > >> # We made changes to Pig - rather we got help from the Pig team,
> > > particularly Daniel.
> > > >>
> > > >> So, we plan to work through the rest of the stack - Hive,
> > Oozie etc.
> > > very soon and we'll
> > > >> depend on updated releases from the individual projects.
> > > >
> > > > Do we have any kinds of commitment from downstream projects as
> > far as
> > > those
> > > > updates are concerned? Are they targeting these changes as
> > part of point
> > > (patch)
> > > > release of an already released version (like Pig 0.9.X for
> > example) or
> > > > will it be
> > > > part of a brand new major release?
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> > >
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> > >
> >
>
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Uma Maheswara Rao G 72686 <ma...@huawei.com>.
Hi Ted,
Recently Constants filenames has been renamed in Trunk and 23 branch. Looks that will be the cause for this issue.
I remember we have done that in HDFS-1620.
current codebase has FsConstants.java.
Regards,
Uma
----- Original Message -----
From: Ted Yu <yu...@gmail.com>
Date: Thursday, September 29, 2011 7:27 am
Subject: HBase trunk on 0.23 Was: Update on hadoop-0.23
To: general@hadoop.apache.org
Cc: bigtop-dev@incubator.apache.org
> Minor correction to Todd's report, currently HBase TRUNK doesn't
> compileagainst 0.23 (
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-
> Hadoop-23/42/console
> ):
>
> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-
> 23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]cannot find symbol
> [ERROR] symbol : class FSConstants
> [ERROR] location: package org.apache.hadoop.hdfs.protocol
> [ERROR]
>
> Cheers
>
> On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com>
> wrote:
> > Hi all,
> >
> > Just an update from the HBase side: I've run some cluster tests on
> > HDFS 0.23 (as of about a month ago) and it generally works well.
> > Performance for some workloads is ~2x due to HDFS-941, and can be
> > improved a bit more if I finish HDFS-2080 in time. I did not do
> > extensive failure testing (to stress the new append/sync code)
> but I
> > do plan to do that in the coming months.
> >
> > HBase trunk can compile against 0.23 by using -Dhadoop23 on the
> maven> build. Currently some 15 or so tests are failing - the
> following HBase
> > JIRA tracks those issues:
> > https://issues.apache.org/jira/browse/HBASE-4254
> >
> > (these may be indicative of HDFS side bugs)
> >
> > Any help there from the community would be appreciated!
> >
> > -Todd
> >
> > On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik
> <rv...@apache.org> wrote:
> > > Hi Arun!
> > >
> > > Thanks for the quick reply!
> > >
> > > I'm sorry if I had too many questions in my original email,
> but I can't
> > find
> > > an answer to my "integration tests" question. Could you,
> please, share
> > > a URL with us where I can find out more about them?
> > >
> > > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy
> <ac...@hortonworks.com>> wrote:
> > >> # We made changes to Pig - rather we got help from the Pig team,
> > particularly Daniel.
> > >>
> > >> So, we plan to work through the rest of the stack - Hive,
> Oozie etc.
> > very soon and we'll
> > >> depend on updated releases from the individual projects.
> > >
> > > Do we have any kinds of commitment from downstream projects as
> far as
> > those
> > > updates are concerned? Are they targeting these changes as
> part of point
> > (patch)
> > > release of an already released version (like Pig 0.9.X for
> example) or
> > > will it be
> > > part of a brand new major release?
> > >
> > > Thanks,
> > > Roman.
> > >
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Uma Maheswara Rao G 72686 <ma...@huawei.com>.
Hi Ted,
Recently Constants filenames has been renamed in Trunk and 23 branch. Looks that will be the cause for this issue.
I remember we have done that in HDFS-1620.
current codebase has FsConstants.java.
Regards,
Uma
----- Original Message -----
From: Ted Yu <yu...@gmail.com>
Date: Thursday, September 29, 2011 7:27 am
Subject: HBase trunk on 0.23 Was: Update on hadoop-0.23
To: general@hadoop.apache.org
Cc: bigtop-dev@incubator.apache.org
> Minor correction to Todd's report, currently HBase TRUNK doesn't
> compileagainst 0.23 (
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-
> Hadoop-23/42/console
> ):
>
> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-
> 23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]cannot find symbol
> [ERROR] symbol : class FSConstants
> [ERROR] location: package org.apache.hadoop.hdfs.protocol
> [ERROR]
>
> Cheers
>
> On Tue, Sep 27, 2011 at 12:56 PM, Todd Lipcon <to...@cloudera.com>
> wrote:
> > Hi all,
> >
> > Just an update from the HBase side: I've run some cluster tests on
> > HDFS 0.23 (as of about a month ago) and it generally works well.
> > Performance for some workloads is ~2x due to HDFS-941, and can be
> > improved a bit more if I finish HDFS-2080 in time. I did not do
> > extensive failure testing (to stress the new append/sync code)
> but I
> > do plan to do that in the coming months.
> >
> > HBase trunk can compile against 0.23 by using -Dhadoop23 on the
> maven> build. Currently some 15 or so tests are failing - the
> following HBase
> > JIRA tracks those issues:
> > https://issues.apache.org/jira/browse/HBASE-4254
> >
> > (these may be indicative of HDFS side bugs)
> >
> > Any help there from the community would be appreciated!
> >
> > -Todd
> >
> > On Tue, Sep 27, 2011 at 12:24 PM, Roman Shaposhnik
> <rv...@apache.org> wrote:
> > > Hi Arun!
> > >
> > > Thanks for the quick reply!
> > >
> > > I'm sorry if I had too many questions in my original email,
> but I can't
> > find
> > > an answer to my "integration tests" question. Could you,
> please, share
> > > a URL with us where I can find out more about them?
> > >
> > > On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy
> <ac...@hortonworks.com>> wrote:
> > >> # We made changes to Pig - rather we got help from the Pig team,
> > particularly Daniel.
> > >>
> > >> So, we plan to work through the rest of the stack - Hive,
> Oozie etc.
> > very soon and we'll
> > >> depend on updated releases from the individual projects.
> > >
> > > Do we have any kinds of commitment from downstream projects as
> far as
> > those
> > > updates are concerned? Are they targeting these changes as
> part of point
> > (patch)
> > > release of an already released version (like Pig 0.9.X for
> example) or
> > > will it be
> > > part of a brand new major release?
> > >
> > > Thanks,
> > > Roman.
> > >
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
Re: HBase trunk on 0.23 Was: Update on hadoop-0.23
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Sep 28, 2011 at 6:56 PM, Ted Yu <yu...@gmail.com> wrote:
> Minor correction to Todd's report, currently HBase TRUNK doesn't compile
> against 0.23 (
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/42/console
> ):
>
> [ERROR] /home/hudson/hudson-slave/workspace/HBase-TRUNK-on-Hadoop-23/trunk/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java:[35,38]
> cannot find symbol
> [ERROR] symbol : class FSConstants
> [ERROR] location: package org.apache.hadoop.hdfs.protocol
> [ERROR]
Yup. That's exactly what I'm seeing:
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/COMPONENT=hbase,label=lucid/2/console
Which is weird. The delta between .22 and .23 is not that great as far
as HDFS APIs are concerned,
so what makes it work for one, but not the other?
Thanks,
Roman.