You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Karthik Kambatla <ka...@cloudera.com> on 2014/08/01 20:28:45 UTC

Re: Branching 2.5

Folks,

I think we are very close to voting on RC0. Just wanted to check one
(hopefully) last thing.

I am unable to verify the signed maven artifacts are actually deployed. To
deploy the artifacts, I did the following and it looked like it ran fine.

   1. .m2/settings.xml -> server-id is apache.staging.https
   2. mvn deploy -Psign,src,dist -Dmaven.test.skip.exec=true
   -Dcontainer-executor.conf.dir=/etc/hadoop/conf
   -Dgpg.passphrase=<my-passphrase>

However, I don't see it here - https://repository.apache.org. How do I
verify this?

Thanks
Karthik

On Wed, Jul 30, 2014 at 4:30 PM, Karthik Kambatla <ka...@cloudera.com>
wrote:

> Thanks to Andrew's patch on HADOOP-10910, I am able to build an RC.
>
>
> On Wed, Jul 30, 2014 at 1:59 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Adding builds@apache.org
>>
>> Cheers
>>
>> On Jul 30, 2014, at 12:52 AM, Andrew Wang <an...@cloudera.com>
>> wrote:
>>
>> > Alright, dug around some more and I think it's that FINDBUGS_HOME is not
>> > being set correctly. I downloaded and extracted Findbugs 1.3.9, pointed
>> > FINDBUGS_HOME at it, and the build worked after that. I don't know
>> what's
>> > up with the default maven build, it'd be great if someone could check.
>> >
>> > Can someone with access to the build machines check this?
>> >
>> > As a side note, I think 1.3.9 was released in 2009. It'd be nice to
>> catch
>> > up with the last 5 years of static analysis :)
>> >
>> >
>> > On Tue, Jul 29, 2014 at 11:36 PM, Andrew Wang <andrew.wang@cloudera.com
>> >
>> > wrote:
>> >
>> >> I looked in the log, it also looks like findbugs is OOMing:
>> >>
>> >>     [java] Exception in thread "main" java.lang.OutOfMemoryError: GC
>> overhead limit exceeded
>> >>     [java]    at edu.umd.cs.findbugs.ba.Path.grow(Path.java:263)
>> >>     [java]    at edu.umd.cs.findbugs.ba.Path.copyFrom(Path.java:113)
>> >>     [java]    at edu.umd.cs.findbugs.ba.Path.duplicate(Path.java:103)
>> >>     [java]    at
>> edu.umd.cs.findbugs.ba.obl.State.duplicate(State.java:65)
>> >>
>> >>
>> >> This is quite possibly related, since there's an error at the end like
>> >> this:
>> >>
>> >> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project
>> hadoop-hdfs: An Ant BuildException has occured: input file
>> /home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/findbugsXml.xml
>> does not exist
>> >>
>> >> [ERROR] around Ant part ...<xslt
>> >> style="/home/jenkins/tools/findbugs/latest/src/xsl/default.xsl"
>> >>
>> in="/home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/findbugsXml.xml"
>> >>
>> out="/home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/site/findbugs.html"/>...
>> >> @ 44:368 in
>> >>
>> /home/jenkins/jenkins-slave/workspace/HADOOP2_Release_Artifacts_Builder/branch-2.5.0/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml
>> >>
>> >> I'll try to figure out how to increase this, but if anyone else knows,
>> >> feel free to chime in.
>> >>
>> >>
>> >> On Tue, Jul 29, 2014 at 5:41 PM, Karthik Kambatla <ka...@cloudera.com>
>> >> wrote:
>> >>
>> >>> Devs,
>> >>>
>> >>> I created branch-2.5.0 and was trying to cut an RC, but ran into
>> issues
>> >>> with creating one. If anyone knows what is going on, please help me
>> out. I
>> >>> ll continue looking into it otherwise.
>> >>>
>> >>>
>> https://builds.apache.org/job/HADOOP2_Release_Artifacts_Builder/24/console
>> >>> is the build that failed. It appears the issue is because it can't
>> find
>> >>> Null.java. I run into the same issue locally as well, even with
>> >>> branch-2.4.1. So, I wonder if I should be doing anything else to
>> create
>> >>> the
>> >>> RC instead?
>> >>>
>> >>> Thanks
>> >>> Karthik
>> >>>
>> >>>
>> >>> On Sun, Jul 27, 2014 at 11:09 AM, Zhijie Shen <zs...@hortonworks.com>
>> >>> wrote:
>> >>>
>> >>>> I've just committed YARN-2247, which is the last 2.5 blocker from
>> YARN.
>> >>>>
>> >>>>
>> >>>> On Sat, Jul 26, 2014 at 5:02 AM, Karthik Kambatla <
>> kasha@cloudera.com>
>> >>>> wrote:
>> >>>>
>> >>>>> A quick update:
>> >>>>>
>> >>>>> All remaining blockers are on the verge of getting committed. Once
>> >>> that
>> >>>> is
>> >>>>> done, I plan to cut a branch for 2.5.0 and get an RC out hopefully
>> >>> this
>> >>>>> coming Monday.
>> >>>>>
>> >>>>>
>> >>>>> On Fri, Jul 25, 2014 at 12:32 PM, Andrew Wang <
>> >>> andrew.wang@cloudera.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> One thing I forgot, the release note activities are happening at
>> >>>>>> HADOOP-10821. If you have other things you'd like to see mentioned,
>> >>>> feel
>> >>>>>> free to leave a comment on the JIRA and I'll try to include it.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> Andrew
>> >>>>>>
>> >>>>>>
>> >>>>>> On Fri, Jul 25, 2014 at 12:28 PM, Andrew Wang <
>> >>>> andrew.wang@cloudera.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> I just went through and fixed up the HDFS and Common CHANGES.txt
>> >>> for
>> >>>>>> 2.5.0.
>> >>>>>>>
>> >>>>>>> As a friendly reminder, please try to put things under the correct
>> >>>>>> section
>> >>>>>>> :) We have subsections for the xattr changes in HDFS-2006 and
>> >>>>>> HADOOP-10514,
>> >>>>>>> and there were some unrelated JIRAs appended to the end.
>> >>>>>>>
>> >>>>>>> I'd also encourage committers to be more liberal with their use of
>> >>>> the
>> >>>>>> NEW
>> >>>>>>> FEATURES section. I'm helping Karthik write up the 2.5 release
>> >>> notes,
>> >>>>> and
>> >>>>>>> I'm using NEW FEATURES to fill it out. When looking through the
>> >>> JIRA
>> >>>>> list
>> >>>>>>> though, I decided to promote things like the SNN/DN/JN webUI
>> >>>>>> improvements,
>> >>>>>>> the HCFS specification work, and OIV read-only WebHDFS access to
>> >>> new
>> >>>>>>> features. One rule-of-thumb, if a feature required an umbrella
>> >>> JIRA,
>> >>>>> put
>> >>>>>>> the umbrella under NEW FEATURES when it's resolved.
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> Andrew
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Jul 16, 2014 at 7:59 PM, Wangda Tan <wh...@gmail.com>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Thanks Tsuyoshi for pointing me this,
>> >>>>>>>>
>> >>>>>>>> Wangda
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Thu, Jul 17, 2014 at 10:30 AM, Tsuyoshi OZAWA <
>> >>>>>>>> ozawa.tsuyoshi@gmail.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Wangda,
>> >>>>>>>>>
>> >>>>>>>>> The following link is same link as Karthik mentioned:
>> >>>
>> https://issues.apache.org/jira/browse/YARN-2247?jql=project%20in%20(Hadoop%2C%20HDFS%2C%20YARN%2C%20%22Hadoop%20Map%2FReduce%22)%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%2Fs%22%20%3D%202.5.0%20AND%20priority%20in%20(Blocker)
>> >>>>>>>>>
>> >>>>>>>>> Or, please access to http://goo.gl/FX3iWp
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> - Tsuyoshi
>> >>>>>>>>>
>> >>>>>>>>> On Thu, Jul 17, 2014 at 10:55 AM, Zhijie Shen <
>> >>>>> zshen@hortonworks.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>> I raised YARN-2247 as the blocker of 2.5.0.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Thu, Jul 17, 2014 at 9:42 AM, Wangda Tan <
>> >>>> wheeleast@gmail.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi Karthik,
>> >>>>>>>>>>> I found I cannot access the filter: http://s.apache.org/vJg
>> >>> .
>> >>>>> Could
>> >>>>>>>> you
>> >>>>>>>>>>> please check its permission? I'd like to know if there's any
>> >>>>>> related
>> >>>>>>>>> issues
>> >>>>>>>>>>> to me. :)
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> Wangda
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Thu, Jul 17, 2014 at 5:54 AM, Karthik Kambatla <
>> >>>>>>>> kasha@cloudera.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> We are down to 4 blockers and looks like they are all
>> >>>> actively
>> >>>>>>>> being
>> >>>>>>>>>>> worked
>> >>>>>>>>>>>> on. Please reconsider marking new JIRAs as blockers.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks
>> >>>>>>>>>>>> Karthik
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> PS: I moved out a couple of JIRAs that didn't seem like
>> >>> true
>> >>>>>>>> blockers
>> >>>>>>>>> to
>> >>>>>>>>>>>> 2.6.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:43 AM, Karthik Kambatla <
>> >>>>>>>> kasha@cloudera.com
>> >>>>>>>>>>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Folks,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> We have 10 blockers for 2.5. Can the people working on
>> >>> them
>> >>>>>>>> revisit
>> >>>>>>>>> and
>> >>>>>>>>>>>>> see if they are really blockers. If they are, can we
>> >>> try to
>> >>>>> get
>> >>>>>>>>> them in
>> >>>>>>>>>>>>> soon? It would be nice to get an RC out the end of this
>> >>>> week
>> >>>>> or
>> >>>>>>>> at
>> >>>>>>>>>>> least
>> >>>>>>>>>>>>> early next week?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks
>> >>>>>>>>>>>>> Karthik
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Wed, Jul 2, 2014 at 11:32 PM, Karthik Kambatla <
>> >>>>>>>>> kasha@cloudera.com>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I just
>> >>>>>>>>>>>>>> 1. moved non-blocker 2.5 JIRAs to 2.6.
>> >>>>>>>>>>>>>> 2. created branch-2.5 and added sections for 2.6.0 in
>> >>> all
>> >>>>>>>>> CHANGES.txt
>> >>>>>>>>>>> in
>> >>>>>>>>>>>>>> trunk and branch-2.
>> >>>>>>>>>>>>>> 3. Will create branch-2.5.0 when we are ready to
>> >>> create an
>> >>>>> RC
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> There are 11 pending blockers for 2.5.0:
>> >>>>>>>> http://s.apache.org/vJg
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Committers - please exercise caution when merging to
>> >>>>>> branch-2.5
>> >>>>>>>> and
>> >>>>>>>>>>>>>> target non-blockers preferably to 2.6
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Wed, Jul 2, 2014 at 10:24 PM, Karthik Kambatla <
>> >>>>>>>>> kasha@cloudera.com
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Committers,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I am working on branching 2.5. Will send an update as
>> >>>> soon
>> >>>>> as
>> >>>>>>>> I am
>> >>>>>>>>>>> done
>> >>>>>>>>>>>>>>> branching.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> Zhijie Shen
>> >>>>>>>>>> Hortonworks Inc.
>> >>>>>>>>>> http://hortonworks.com/
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> CONFIDENTIALITY NOTICE
>> >>>>>>>>>> NOTICE: This message is intended for the use of the
>> >>> individual
>> >>>> or
>> >>>>>>>> entity
>> >>>>>>>>> to
>> >>>>>>>>>> which it is addressed and may contain information that is
>> >>>>>>>> confidential,
>> >>>>>>>>>> privileged and exempt from disclosure under applicable law.
>> >>> If
>> >>>> the
>> >>>>>>>> reader
>> >>>>>>>>>> of this message is not the intended recipient, you are hereby
>> >>>>>> notified
>> >>>>>>>>> that
>> >>>>>>>>>> any printing, copying, dissemination, distribution,
>> >>> disclosure
>> >>>> or
>> >>>>>>>>>> forwarding of this communication is strictly prohibited. If
>> >>> you
>> >>>>> have
>> >>>>>>>>>> received this communication in error, please contact the
>> >>> sender
>> >>>>>>>>> immediately
>> >>>>>>>>>> and delete it from your system. Thank You.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> - Tsuyoshi
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Zhijie Shen
>> >>>> Hortonworks Inc.
>> >>>> http://hortonworks.com/
>> >>>>
>> >>>> --
>> >>>> CONFIDENTIALITY NOTICE
>> >>>> NOTICE: This message is intended for the use of the individual or
>> >>> entity to
>> >>>> which it is addressed and may contain information that is
>> confidential,
>> >>>> privileged and exempt from disclosure under applicable law. If the
>> >>> reader
>> >>>> of this message is not the intended recipient, you are hereby
>> notified
>> >>> that
>> >>>> any printing, copying, dissemination, distribution, disclosure or
>> >>>> forwarding of this communication is strictly prohibited. If you have
>> >>>> received this communication in error, please contact the sender
>> >>> immediately
>> >>>> and delete it from your system. Thank You.
>> >>
>> >>
>>
>
>

Re: Branching 2.5

Posted by Andrew Wang <an...@cloudera.com>.

Juan and Yongjun just let me know that we need HDFS-6793 because it reverts
an incompatible change. I just committed it down to branch-2.5.0.


On Fri, Aug 1, 2014 at 12:07 PM, Stack <st...@duboce.net> wrote:

> On Fri, Aug 1, 2014 at 11:28 AM, Karthik Kambatla <ka...@cloudera.com>
> wrote:
>
> > Folks,
> >
> > I think we are very close to voting on RC0. Just wanted to check one
> > (hopefully) last thing.
> >
> > I am unable to verify the signed maven artifacts are actually deployed.
> To
> > deploy the artifacts, I did the following and it looked like it ran fine.
> >
> >    1. .m2/settings.xml -> server-id is apache.staging.https
> >    2. mvn deploy -Psign,src,dist -Dmaven.test.skip.exec=true
> >    -Dcontainer-executor.conf.dir=/etc/hadoop/conf
> >    -Dgpg.passphrase=<my-passphrase>
> >
> > However, I don't see it here - https://repository.apache.org. How do I
> > verify this?
> >
>
> When deploy runs, it logs where it is uploading the artifacts too.  Do you
> have the mvn deploy output still Karthik?  Maybe that'll help track where
> mvn hid it on you?
>
> St.Ack
>

Re: Branching 2.5

Posted by Stack <st...@duboce.net>.

On Fri, Aug 1, 2014 at 11:28 AM, Karthik Kambatla <ka...@cloudera.com>
wrote:

> Folks,
>
> I think we are very close to voting on RC0. Just wanted to check one
> (hopefully) last thing.
>
> I am unable to verify the signed maven artifacts are actually deployed. To
> deploy the artifacts, I did the following and it looked like it ran fine.
>
>    1. .m2/settings.xml -> server-id is apache.staging.https
>    2. mvn deploy -Psign,src,dist -Dmaven.test.skip.exec=true
>    -Dcontainer-executor.conf.dir=/etc/hadoop/conf
>    -Dgpg.passphrase=<my-passphrase>
>
> However, I don't see it here - https://repository.apache.org. How do I
> verify this?
>

When deploy runs, it logs where it is uploading the artifacts too.  Do you
have the mvn deploy output still Karthik?  Maybe that'll help track where
mvn hid it on you?

St.Ack