You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yetus.apache.org by Enis Söztutar <en...@apache.org> on 2016/03/02 19:47:58 UTC
/bin/ls permission issue on precommit tests
HBase precommit tests have been failing on some nodes for a couple of days
related to this message:
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
Error while running command to get file permissions :
ExitCodeException exitCode=127: /bin/ls: error while loading shared
libraries: libacl.so.1: failed to map segment from shared object:
Permission denied
It is not clear to me whether this is a docker / yetus problem, or an INFRA
issue. Anybody familiar with this? Obviously, it makes it very hard to
commit a patch without the precommit tests. I've seen this on at least 2
nodes.
Sample output:
https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0_72.txt
Enis
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Please remember to only use the prerelease check box on a case-by-case
basis when checking to see if a yetus issue is resolved for hbase's
use.
On Wed, Mar 2, 2016 at 1:33 PM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the information, Chris.
>
> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> uses current
> HEAD of apache/yetus.
>
> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>
> Cheers
>
> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hi Enis,
>>
>> Hadoop pre-commit was experiencing the same problem a few weeks ago:
>>
>> https://issues.apache.org/jira/browse/YETUS-286
>>
>>
>> We resolved this as a duplicate of a patch that allows control of whether
>> or not Docker runs in privileged mode:
>>
>> https://issues.apache.org/jira/browse/YETUS-285
>>
>>
>> I have to admit I'm not enough of a Docker expert to understand this fix,
>> but you can read through the issue comments from the contributors if you
>> want details.
>>
>> Hadoop pre-commit currently runs off of Yetus master, which includes this
>> patch, and we are no longer seeing the problem. I see HBase pre-commit is
>> currently running with Yetus release 0.1.0, which does not have the fix.
>>
>> I think you have 2 options:
>>
>> 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>> be aware that new Yetus commits will go to master though.)
>> 2. If you don't like the idea of HBase pre-commit running Yetus code that
>> is constantly in motion, then you can wait a few days for sign-off of our
>> 0.2.0 release candidate, which is currently under [VOTE] and looking good
>> so far.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>>
>> >HBase precommit tests have been failing on some nodes for a couple of days
>> >related to this message:
>> >
>> >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> >Error while running command to get file permissions :
>> >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>> >libraries: libacl.so.1: failed to map segment from shared object:
>> >Permission denied
>> >
>> >
>> >It is not clear to me whether this is a docker / yetus problem, or an
>> >INFRA
>> >issue. Anybody familiar with this? Obviously, it makes it very hard to
>> >commit a patch without the precommit tests. I've seen this on at least 2
>> >nodes.
>> >
>> >Sample output:
>> >
>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>> >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>> >
>> >Enis
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Allen Wittenauer <aw...@altiscale.com>.
For the record, I’m planning on switching Hadoop to 0.2.0 as soon as we finish the vote. The results we’re getting from live testing has dried up.
Re: /bin/ls permission issue on precommit tests
Posted by Allen Wittenauer <aw...@altiscale.com>.
For the record, I’m planning on switching Hadoop to 0.2.0 as soon as we finish the vote. The results we’re getting from live testing has dried up.
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
I've changed the pre-commit job for HBase, and kicked a run for
https://issues.apache.org/jira/browse/HBASE-15295. Let's see whether that
resolves the issue for now.
Enis
On Wed, Mar 2, 2016 at 3:25 PM, Sean Busbey <bu...@apache.org> wrote:
> On Wed, Mar 2, 2016 at 4:18 PM, Enis Söztutar <en...@gmail.com> wrote:
> > I have seen it in a couple of places: HBASE-15325, HBASE-15375,
> > HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0
> and
> > ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
> > not do the exclude.
> >
> > Let's do turn off the docker mode if you think that it will solve the
> > issue. I really just want to unblock precommit builds.
> >
>
> +1 that should let us move forward, albeit with degraded capabilities
> (older shellcheck,
> maybe ruby linting won't work, we'll be able to see specifics in the
> next report)
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
I've changed the pre-commit job for HBase, and kicked a run for
https://issues.apache.org/jira/browse/HBASE-15295. Let's see whether that
resolves the issue for now.
Enis
On Wed, Mar 2, 2016 at 3:25 PM, Sean Busbey <bu...@apache.org> wrote:
> On Wed, Mar 2, 2016 at 4:18 PM, Enis Söztutar <en...@gmail.com> wrote:
> > I have seen it in a couple of places: HBASE-15325, HBASE-15375,
> > HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0
> and
> > ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
> > not do the exclude.
> >
> > Let's do turn off the docker mode if you think that it will solve the
> > issue. I really just want to unblock precommit builds.
> >
>
> +1 that should let us move forward, albeit with degraded capabilities
> (older shellcheck,
> maybe ruby linting won't work, we'll be able to see specifics in the
> next report)
>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
On Wed, Mar 2, 2016 at 4:18 PM, Enis Söztutar <en...@gmail.com> wrote:
> I have seen it in a couple of places: HBASE-15325, HBASE-15375,
> HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0 and
> ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
> not do the exclude.
>
> Let's do turn off the docker mode if you think that it will solve the
> issue. I really just want to unblock precommit builds.
>
+1 that should let us move forward, albeit with degraded capabilities
(older shellcheck,
maybe ruby linting won't work, we'll be able to see specifics in the
next report)
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
On Wed, Mar 2, 2016 at 4:18 PM, Enis Söztutar <en...@gmail.com> wrote:
> I have seen it in a couple of places: HBASE-15325, HBASE-15375,
> HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0 and
> ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
> not do the exclude.
>
> Let's do turn off the docker mode if you think that it will solve the
> issue. I really just want to unblock precommit builds.
>
+1 that should let us move forward, albeit with degraded capabilities
(older shellcheck,
maybe ruby linting won't work, we'll be able to see specifics in the
next report)
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
I have seen it in a couple of places: HBASE-15325, HBASE-15375,
HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0 and
ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
not do the exclude.
Let's do turn off the docker mode if you think that it will solve the
issue. I really just want to unblock precommit builds.
Enis
On Wed, Mar 2, 2016 at 2:10 PM, Sean Busbey <bu...@apache.org> wrote:
> How often is the problem happening? I hadn't seen it come up prior to
> this thread.
>
> What about just turning off docker mode?
>
> What about seeing if this is only happening on a single host and just
> blacklisting that host until the release?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 3:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> > Let me do another proposal (for the HBase community)
> >
> > Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
> > 0.2.0 release. Once the release is out, we can switch it back to using
> > 0.2.0 and depend on 0.2.0 until 0.3.0.
> >
> > How does this sound?
> >
> > Enis
> >
> > On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> >> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
> >> wrote:
> >>
> >>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org>
> wrote:
> >>> >
> >>> >> Such a tag would be a distribution according to ASF rules. The Yetus
> >>> >> PMC would have to vote on it just as we do releases.
> >>> >>
> >>> >
> >>> > Not necessarily. We can depend on a pre-release tag of any of our
> >>> > dependencies as long as we are not releasing them with our release.
> We
> >>> are
> >>> > not releasing yetus together with HBase. We can depend on any
> snapshot
> >>> tag
> >>> > for our precommit build. I was suggesting that the yetus community
> have
> >>> a
> >>> > guideline on what commit has is the best.
> >>> >
> >>>
> >>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
> >>> it pleases, you are correct, but ASF rules about use from those
> outside of
> >>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
> >>> project in
> >>> non-released form we need to take action to stop it.
> >>
> >>
> >>> I *really*, *really* would like to avoid going down the rabbit hole of
> >>> ASF rules lawyering on "who's using the artifact" and other ways of
> >>> attempting
> >>> to use semantics to avoid the root cause of yetus needing to release
> more
> >>> often.
> >>> That will be exhausting, a waste of time that could instead be put
> towards
> >>> actually having releases, and will doom the Yetus project to fail in
> >>> its expressed purpose of making software for the public good (rather
> than
> >>> for
> >>> ASF internal belly-itching).
> >>>
> >>
> >> ASF internal projects are the immediate consumer of yetus today and we
> >> depend on it in a critical manner in the development lifecycle. That is
> why
> >> prioritizing ASF internal needs for YETUS makes sense to me. Given
> enough
> >> time, and if yetus community keeps up with frequent releases and more
> >> stability we would not need to depend on unreleased tags. But if HBase
> and
> >> Hadoop and other projects are blocked for precommit until a release
> which
> >> may be another week or so, I would rather go with a solution that fixes
> the
> >> issue now and worry about the perfect solution later.
> >>
> >>
> >>>
> >>>
> >>> >
> >>> >>
> >>> >> The answer to the issue of blocking downstream folks is to release
> more
> >>> >> often.
> >>> >>
> >>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
> >>> >> that want access to more frequent releases can help the situation by
> >>> >> helping to vote on releases.
> >>> >>
> >>> >
> >>> > This is great, but I did not see it happen yet. Again, my main goal
> is
> >>> to
> >>> > unblock current state with the least amount of friction.
> >>> >
> >>>
> >>> As I mentioned, the best way to unblock the current state is to help
> >>> vote on releases.
> >>>
> >>> Another way to help would be to be more vocal on prioritization for
> >>> things going into a release. For example, we have the 0.2.0 vote
> closing
> >>> this coming Friday rather than the prior Monday because the Yetus
> >>> community prioritized fixing a few last sharp edges over
> >>> the weekend. More voices that state the pressing need for blocking
> >>> issues will help the community make calls that are in line with what
> you
> >>> desire.
> >>>
> >>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
> >>> had no reason to push for a sooner release with my HBase hat on.
> >>>
> >>>
> >>> --
> >>> Sean
> >>>
> >>
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
I have seen it in a couple of places: HBASE-15325, HBASE-15375,
HBASE-15181, HBASE-14949. I had checked the hosts before, I remember H0 and
ubuntu-4 as hosts, but I assumed that this is not host specific, thus did
not do the exclude.
Let's do turn off the docker mode if you think that it will solve the
issue. I really just want to unblock precommit builds.
Enis
On Wed, Mar 2, 2016 at 2:10 PM, Sean Busbey <bu...@apache.org> wrote:
> How often is the problem happening? I hadn't seen it come up prior to
> this thread.
>
> What about just turning off docker mode?
>
> What about seeing if this is only happening on a single host and just
> blacklisting that host until the release?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 3:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> > Let me do another proposal (for the HBase community)
> >
> > Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
> > 0.2.0 release. Once the release is out, we can switch it back to using
> > 0.2.0 and depend on 0.2.0 until 0.3.0.
> >
> > How does this sound?
> >
> > Enis
> >
> > On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> >> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
> >> wrote:
> >>
> >>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org>
> wrote:
> >>> >
> >>> >> Such a tag would be a distribution according to ASF rules. The Yetus
> >>> >> PMC would have to vote on it just as we do releases.
> >>> >>
> >>> >
> >>> > Not necessarily. We can depend on a pre-release tag of any of our
> >>> > dependencies as long as we are not releasing them with our release.
> We
> >>> are
> >>> > not releasing yetus together with HBase. We can depend on any
> snapshot
> >>> tag
> >>> > for our precommit build. I was suggesting that the yetus community
> have
> >>> a
> >>> > guideline on what commit has is the best.
> >>> >
> >>>
> >>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
> >>> it pleases, you are correct, but ASF rules about use from those
> outside of
> >>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
> >>> project in
> >>> non-released form we need to take action to stop it.
> >>
> >>
> >>> I *really*, *really* would like to avoid going down the rabbit hole of
> >>> ASF rules lawyering on "who's using the artifact" and other ways of
> >>> attempting
> >>> to use semantics to avoid the root cause of yetus needing to release
> more
> >>> often.
> >>> That will be exhausting, a waste of time that could instead be put
> towards
> >>> actually having releases, and will doom the Yetus project to fail in
> >>> its expressed purpose of making software for the public good (rather
> than
> >>> for
> >>> ASF internal belly-itching).
> >>>
> >>
> >> ASF internal projects are the immediate consumer of yetus today and we
> >> depend on it in a critical manner in the development lifecycle. That is
> why
> >> prioritizing ASF internal needs for YETUS makes sense to me. Given
> enough
> >> time, and if yetus community keeps up with frequent releases and more
> >> stability we would not need to depend on unreleased tags. But if HBase
> and
> >> Hadoop and other projects are blocked for precommit until a release
> which
> >> may be another week or so, I would rather go with a solution that fixes
> the
> >> issue now and worry about the perfect solution later.
> >>
> >>
> >>>
> >>>
> >>> >
> >>> >>
> >>> >> The answer to the issue of blocking downstream folks is to release
> more
> >>> >> often.
> >>> >>
> >>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
> >>> >> that want access to more frequent releases can help the situation by
> >>> >> helping to vote on releases.
> >>> >>
> >>> >
> >>> > This is great, but I did not see it happen yet. Again, my main goal
> is
> >>> to
> >>> > unblock current state with the least amount of friction.
> >>> >
> >>>
> >>> As I mentioned, the best way to unblock the current state is to help
> >>> vote on releases.
> >>>
> >>> Another way to help would be to be more vocal on prioritization for
> >>> things going into a release. For example, we have the 0.2.0 vote
> closing
> >>> this coming Friday rather than the prior Monday because the Yetus
> >>> community prioritized fixing a few last sharp edges over
> >>> the weekend. More voices that state the pressing need for blocking
> >>> issues will help the community make calls that are in line with what
> you
> >>> desire.
> >>>
> >>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
> >>> had no reason to push for a sooner release with my HBase hat on.
> >>>
> >>>
> >>> --
> >>> Sean
> >>>
> >>
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
How often is the problem happening? I hadn't seen it come up prior to
this thread.
What about just turning off docker mode?
What about seeing if this is only happening on a single host and just
blacklisting that host until the release?
-Sean
On Wed, Mar 2, 2016 at 3:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> Let me do another proposal (for the HBase community)
>
> Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
> 0.2.0 release. Once the release is out, we can switch it back to using
> 0.2.0 and depend on 0.2.0 until 0.3.0.
>
> How does this sound?
>
> Enis
>
> On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com> wrote:
>
>> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
>> wrote:
>>
>>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
>>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>>> >
>>> >> Such a tag would be a distribution according to ASF rules. The Yetus
>>> >> PMC would have to vote on it just as we do releases.
>>> >>
>>> >
>>> > Not necessarily. We can depend on a pre-release tag of any of our
>>> > dependencies as long as we are not releasing them with our release. We
>>> are
>>> > not releasing yetus together with HBase. We can depend on any snapshot
>>> tag
>>> > for our precommit build. I was suggesting that the yetus community have
>>> a
>>> > guideline on what commit has is the best.
>>> >
>>>
>>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
>>> it pleases, you are correct, but ASF rules about use from those outside of
>>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
>>> project in
>>> non-released form we need to take action to stop it.
>>
>>
>>> I *really*, *really* would like to avoid going down the rabbit hole of
>>> ASF rules lawyering on "who's using the artifact" and other ways of
>>> attempting
>>> to use semantics to avoid the root cause of yetus needing to release more
>>> often.
>>> That will be exhausting, a waste of time that could instead be put towards
>>> actually having releases, and will doom the Yetus project to fail in
>>> its expressed purpose of making software for the public good (rather than
>>> for
>>> ASF internal belly-itching).
>>>
>>
>> ASF internal projects are the immediate consumer of yetus today and we
>> depend on it in a critical manner in the development lifecycle. That is why
>> prioritizing ASF internal needs for YETUS makes sense to me. Given enough
>> time, and if yetus community keeps up with frequent releases and more
>> stability we would not need to depend on unreleased tags. But if HBase and
>> Hadoop and other projects are blocked for precommit until a release which
>> may be another week or so, I would rather go with a solution that fixes the
>> issue now and worry about the perfect solution later.
>>
>>
>>>
>>>
>>> >
>>> >>
>>> >> The answer to the issue of blocking downstream folks is to release more
>>> >> often.
>>> >>
>>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
>>> >> that want access to more frequent releases can help the situation by
>>> >> helping to vote on releases.
>>> >>
>>> >
>>> > This is great, but I did not see it happen yet. Again, my main goal is
>>> to
>>> > unblock current state with the least amount of friction.
>>> >
>>>
>>> As I mentioned, the best way to unblock the current state is to help
>>> vote on releases.
>>>
>>> Another way to help would be to be more vocal on prioritization for
>>> things going into a release. For example, we have the 0.2.0 vote closing
>>> this coming Friday rather than the prior Monday because the Yetus
>>> community prioritized fixing a few last sharp edges over
>>> the weekend. More voices that state the pressing need for blocking
>>> issues will help the community make calls that are in line with what you
>>> desire.
>>>
>>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
>>> had no reason to push for a sooner release with my HBase hat on.
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
How often is the problem happening? I hadn't seen it come up prior to
this thread.
What about just turning off docker mode?
What about seeing if this is only happening on a single host and just
blacklisting that host until the release?
-Sean
On Wed, Mar 2, 2016 at 3:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> Let me do another proposal (for the HBase community)
>
> Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
> 0.2.0 release. Once the release is out, we can switch it back to using
> 0.2.0 and depend on 0.2.0 until 0.3.0.
>
> How does this sound?
>
> Enis
>
> On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com> wrote:
>
>> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
>> wrote:
>>
>>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
>>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>>> >
>>> >> Such a tag would be a distribution according to ASF rules. The Yetus
>>> >> PMC would have to vote on it just as we do releases.
>>> >>
>>> >
>>> > Not necessarily. We can depend on a pre-release tag of any of our
>>> > dependencies as long as we are not releasing them with our release. We
>>> are
>>> > not releasing yetus together with HBase. We can depend on any snapshot
>>> tag
>>> > for our precommit build. I was suggesting that the yetus community have
>>> a
>>> > guideline on what commit has is the best.
>>> >
>>>
>>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
>>> it pleases, you are correct, but ASF rules about use from those outside of
>>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
>>> project in
>>> non-released form we need to take action to stop it.
>>
>>
>>> I *really*, *really* would like to avoid going down the rabbit hole of
>>> ASF rules lawyering on "who's using the artifact" and other ways of
>>> attempting
>>> to use semantics to avoid the root cause of yetus needing to release more
>>> often.
>>> That will be exhausting, a waste of time that could instead be put towards
>>> actually having releases, and will doom the Yetus project to fail in
>>> its expressed purpose of making software for the public good (rather than
>>> for
>>> ASF internal belly-itching).
>>>
>>
>> ASF internal projects are the immediate consumer of yetus today and we
>> depend on it in a critical manner in the development lifecycle. That is why
>> prioritizing ASF internal needs for YETUS makes sense to me. Given enough
>> time, and if yetus community keeps up with frequent releases and more
>> stability we would not need to depend on unreleased tags. But if HBase and
>> Hadoop and other projects are blocked for precommit until a release which
>> may be another week or so, I would rather go with a solution that fixes the
>> issue now and worry about the perfect solution later.
>>
>>
>>>
>>>
>>> >
>>> >>
>>> >> The answer to the issue of blocking downstream folks is to release more
>>> >> often.
>>> >>
>>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
>>> >> that want access to more frequent releases can help the situation by
>>> >> helping to vote on releases.
>>> >>
>>> >
>>> > This is great, but I did not see it happen yet. Again, my main goal is
>>> to
>>> > unblock current state with the least amount of friction.
>>> >
>>>
>>> As I mentioned, the best way to unblock the current state is to help
>>> vote on releases.
>>>
>>> Another way to help would be to be more vocal on prioritization for
>>> things going into a release. For example, we have the 0.2.0 vote closing
>>> this coming Friday rather than the prior Monday because the Yetus
>>> community prioritized fixing a few last sharp edges over
>>> the weekend. More voices that state the pressing need for blocking
>>> issues will help the community make calls that are in line with what you
>>> desire.
>>>
>>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
>>> had no reason to push for a sooner release with my HBase hat on.
>>>
>>>
>>> --
>>> Sean
>>>
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
Let me do another proposal (for the HBase community)
Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
0.2.0 release. Once the release is out, we can switch it back to using
0.2.0 and depend on 0.2.0 until 0.3.0.
How does this sound?
Enis
On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
> wrote:
>
>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>> >
>> >> Such a tag would be a distribution according to ASF rules. The Yetus
>> >> PMC would have to vote on it just as we do releases.
>> >>
>> >
>> > Not necessarily. We can depend on a pre-release tag of any of our
>> > dependencies as long as we are not releasing them with our release. We
>> are
>> > not releasing yetus together with HBase. We can depend on any snapshot
>> tag
>> > for our precommit build. I was suggesting that the yetus community have
>> a
>> > guideline on what commit has is the best.
>> >
>>
>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
>> it pleases, you are correct, but ASF rules about use from those outside of
>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
>> project in
>> non-released form we need to take action to stop it.
>
>
>> I *really*, *really* would like to avoid going down the rabbit hole of
>> ASF rules lawyering on "who's using the artifact" and other ways of
>> attempting
>> to use semantics to avoid the root cause of yetus needing to release more
>> often.
>> That will be exhausting, a waste of time that could instead be put towards
>> actually having releases, and will doom the Yetus project to fail in
>> its expressed purpose of making software for the public good (rather than
>> for
>> ASF internal belly-itching).
>>
>
> ASF internal projects are the immediate consumer of yetus today and we
> depend on it in a critical manner in the development lifecycle. That is why
> prioritizing ASF internal needs for YETUS makes sense to me. Given enough
> time, and if yetus community keeps up with frequent releases and more
> stability we would not need to depend on unreleased tags. But if HBase and
> Hadoop and other projects are blocked for precommit until a release which
> may be another week or so, I would rather go with a solution that fixes the
> issue now and worry about the perfect solution later.
>
>
>>
>>
>> >
>> >>
>> >> The answer to the issue of blocking downstream folks is to release more
>> >> often.
>> >>
>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
>> >> that want access to more frequent releases can help the situation by
>> >> helping to vote on releases.
>> >>
>> >
>> > This is great, but I did not see it happen yet. Again, my main goal is
>> to
>> > unblock current state with the least amount of friction.
>> >
>>
>> As I mentioned, the best way to unblock the current state is to help
>> vote on releases.
>>
>> Another way to help would be to be more vocal on prioritization for
>> things going into a release. For example, we have the 0.2.0 vote closing
>> this coming Friday rather than the prior Monday because the Yetus
>> community prioritized fixing a few last sharp edges over
>> the weekend. More voices that state the pressing need for blocking
>> issues will help the community make calls that are in line with what you
>> desire.
>>
>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
>> had no reason to push for a sooner release with my HBase hat on.
>>
>>
>> --
>> Sean
>>
>
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
Let me do another proposal (for the HBase community)
Let's set USE_YETUS_PRERELEASE in the HBase precommit job for now until
0.2.0 release. Once the release is out, we can switch it back to using
0.2.0 and depend on 0.2.0 until 0.3.0.
How does this sound?
Enis
On Wed, Mar 2, 2016 at 1:33 PM, Enis Söztutar <en...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com>
> wrote:
>
>> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
>> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>> >
>> >> Such a tag would be a distribution according to ASF rules. The Yetus
>> >> PMC would have to vote on it just as we do releases.
>> >>
>> >
>> > Not necessarily. We can depend on a pre-release tag of any of our
>> > dependencies as long as we are not releasing them with our release. We
>> are
>> > not releasing yetus together with HBase. We can depend on any snapshot
>> tag
>> > for our precommit build. I was suggesting that the yetus community have
>> a
>> > guideline on what commit has is the best.
>> >
>>
>> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
>> it pleases, you are correct, but ASF rules about use from those outside of
>> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
>> project in
>> non-released form we need to take action to stop it.
>
>
>> I *really*, *really* would like to avoid going down the rabbit hole of
>> ASF rules lawyering on "who's using the artifact" and other ways of
>> attempting
>> to use semantics to avoid the root cause of yetus needing to release more
>> often.
>> That will be exhausting, a waste of time that could instead be put towards
>> actually having releases, and will doom the Yetus project to fail in
>> its expressed purpose of making software for the public good (rather than
>> for
>> ASF internal belly-itching).
>>
>
> ASF internal projects are the immediate consumer of yetus today and we
> depend on it in a critical manner in the development lifecycle. That is why
> prioritizing ASF internal needs for YETUS makes sense to me. Given enough
> time, and if yetus community keeps up with frequent releases and more
> stability we would not need to depend on unreleased tags. But if HBase and
> Hadoop and other projects are blocked for precommit until a release which
> may be another week or so, I would rather go with a solution that fixes the
> issue now and worry about the perfect solution later.
>
>
>>
>>
>> >
>> >>
>> >> The answer to the issue of blocking downstream folks is to release more
>> >> often.
>> >>
>> >> The Yetus PMC has a release cadence goal of weekly. Other projects
>> >> that want access to more frequent releases can help the situation by
>> >> helping to vote on releases.
>> >>
>> >
>> > This is great, but I did not see it happen yet. Again, my main goal is
>> to
>> > unblock current state with the least amount of friction.
>> >
>>
>> As I mentioned, the best way to unblock the current state is to help
>> vote on releases.
>>
>> Another way to help would be to be more vocal on prioritization for
>> things going into a release. For example, we have the 0.2.0 vote closing
>> this coming Friday rather than the prior Monday because the Yetus
>> community prioritized fixing a few last sharp edges over
>> the weekend. More voices that state the pressing need for blocking
>> issues will help the community make calls that are in line with what you
>> desire.
>>
>> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
>> had no reason to push for a sooner release with my HBase hat on.
>>
>>
>> --
>> Sean
>>
>
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
> >
> >> Such a tag would be a distribution according to ASF rules. The Yetus
> >> PMC would have to vote on it just as we do releases.
> >>
> >
> > Not necessarily. We can depend on a pre-release tag of any of our
> > dependencies as long as we are not releasing them with our release. We
> are
> > not releasing yetus together with HBase. We can depend on any snapshot
> tag
> > for our precommit build. I was suggesting that the yetus community have a
> > guideline on what commit has is the best.
> >
>
> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
> it pleases, you are correct, but ASF rules about use from those outside of
> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
> project in
> non-released form we need to take action to stop it.
> I *really*, *really* would like to avoid going down the rabbit hole of
> ASF rules lawyering on "who's using the artifact" and other ways of
> attempting
> to use semantics to avoid the root cause of yetus needing to release more
> often.
> That will be exhausting, a waste of time that could instead be put towards
> actually having releases, and will doom the Yetus project to fail in
> its expressed purpose of making software for the public good (rather than
> for
> ASF internal belly-itching).
>
ASF internal projects are the immediate consumer of yetus today and we
depend on it in a critical manner in the development lifecycle. That is why
prioritizing ASF internal needs for YETUS makes sense to me. Given enough
time, and if yetus community keeps up with frequent releases and more
stability we would not need to depend on unreleased tags. But if HBase and
Hadoop and other projects are blocked for precommit until a release which
may be another week or so, I would rather go with a solution that fixes the
issue now and worry about the perfect solution later.
>
>
> >
> >>
> >> The answer to the issue of blocking downstream folks is to release more
> >> often.
> >>
> >> The Yetus PMC has a release cadence goal of weekly. Other projects
> >> that want access to more frequent releases can help the situation by
> >> helping to vote on releases.
> >>
> >
> > This is great, but I did not see it happen yet. Again, my main goal is to
> > unblock current state with the least amount of friction.
> >
>
> As I mentioned, the best way to unblock the current state is to help
> vote on releases.
>
> Another way to help would be to be more vocal on prioritization for
> things going into a release. For example, we have the 0.2.0 vote closing
> this coming Friday rather than the prior Monday because the Yetus
> community prioritized fixing a few last sharp edges over
> the weekend. More voices that state the pressing need for blocking
> issues will help the community make calls that are in line with what you
> desire.
>
> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
> had no reason to push for a sooner release with my HBase hat on.
>
>
> --
> Sean
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
On Wed, Mar 2, 2016 at 12:24 PM, Sean Busbey <se...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
> > On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
> >
> >> Such a tag would be a distribution according to ASF rules. The Yetus
> >> PMC would have to vote on it just as we do releases.
> >>
> >
> > Not necessarily. We can depend on a pre-release tag of any of our
> > dependencies as long as we are not releasing them with our release. We
> are
> > not releasing yetus together with HBase. We can depend on any snapshot
> tag
> > for our precommit build. I was suggesting that the yetus community have a
> > guideline on what commit has is the best.
> >
>
> It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
> it pleases, you are correct, but ASF rules about use from those outside of
> dev@yetus is clear: If the Yetus PMC is aware of regular use of our
> project in
> non-released form we need to take action to stop it.
> I *really*, *really* would like to avoid going down the rabbit hole of
> ASF rules lawyering on "who's using the artifact" and other ways of
> attempting
> to use semantics to avoid the root cause of yetus needing to release more
> often.
> That will be exhausting, a waste of time that could instead be put towards
> actually having releases, and will doom the Yetus project to fail in
> its expressed purpose of making software for the public good (rather than
> for
> ASF internal belly-itching).
>
ASF internal projects are the immediate consumer of yetus today and we
depend on it in a critical manner in the development lifecycle. That is why
prioritizing ASF internal needs for YETUS makes sense to me. Given enough
time, and if yetus community keeps up with frequent releases and more
stability we would not need to depend on unreleased tags. But if HBase and
Hadoop and other projects are blocked for precommit until a release which
may be another week or so, I would rather go with a solution that fixes the
issue now and worry about the perfect solution later.
>
>
> >
> >>
> >> The answer to the issue of blocking downstream folks is to release more
> >> often.
> >>
> >> The Yetus PMC has a release cadence goal of weekly. Other projects
> >> that want access to more frequent releases can help the situation by
> >> helping to vote on releases.
> >>
> >
> > This is great, but I did not see it happen yet. Again, my main goal is to
> > unblock current state with the least amount of friction.
> >
>
> As I mentioned, the best way to unblock the current state is to help
> vote on releases.
>
> Another way to help would be to be more vocal on prioritization for
> things going into a release. For example, we have the 0.2.0 vote closing
> this coming Friday rather than the prior Monday because the Yetus
> community prioritized fixing a few last sharp edges over
> the weekend. More voices that state the pressing need for blocking
> issues will help the community make calls that are in line with what you
> desire.
>
> FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
> had no reason to push for a sooner release with my HBase hat on.
>
>
> --
> Sean
>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <se...@gmail.com>.
On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>
>> Such a tag would be a distribution according to ASF rules. The Yetus
>> PMC would have to vote on it just as we do releases.
>>
>
> Not necessarily. We can depend on a pre-release tag of any of our
> dependencies as long as we are not releasing them with our release. We are
> not releasing yetus together with HBase. We can depend on any snapshot tag
> for our precommit build. I was suggesting that the yetus community have a
> guideline on what commit has is the best.
>
It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
it pleases, you are correct, but ASF rules about use from those outside of
dev@yetus is clear: If the Yetus PMC is aware of regular use of our project in
non-released form we need to take action to stop it.
I *really*, *really* would like to avoid going down the rabbit hole of
ASF rules lawyering on "who's using the artifact" and other ways of attempting
to use semantics to avoid the root cause of yetus needing to release more often.
That will be exhausting, a waste of time that could instead be put towards
actually having releases, and will doom the Yetus project to fail in
its expressed purpose of making software for the public good (rather than for
ASF internal belly-itching).
>
>>
>> The answer to the issue of blocking downstream folks is to release more
>> often.
>>
>> The Yetus PMC has a release cadence goal of weekly. Other projects
>> that want access to more frequent releases can help the situation by
>> helping to vote on releases.
>>
>
> This is great, but I did not see it happen yet. Again, my main goal is to
> unblock current state with the least amount of friction.
>
As I mentioned, the best way to unblock the current state is to help
vote on releases.
Another way to help would be to be more vocal on prioritization for
things going into a release. For example, we have the 0.2.0 vote closing
this coming Friday rather than the prior Monday because the Yetus
community prioritized fixing a few last sharp edges over
the weekend. More voices that state the pressing need for blocking
issues will help the community make calls that are in line with what you desire.
FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
had no reason to push for a sooner release with my HBase hat on.
--
Sean
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <se...@gmail.com>.
On Wed, Mar 2, 2016 at 2:04 PM, Enis Söztutar <en...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
>
>> Such a tag would be a distribution according to ASF rules. The Yetus
>> PMC would have to vote on it just as we do releases.
>>
>
> Not necessarily. We can depend on a pre-release tag of any of our
> dependencies as long as we are not releasing them with our release. We are
> not releasing yetus together with HBase. We can depend on any snapshot tag
> for our precommit build. I was suggesting that the yetus community have a
> guideline on what commit has is the best.
>
It would be a distribution for _the_Yetus_PMC_. HBase is free to do as
it pleases, you are correct, but ASF rules about use from those outside of
dev@yetus is clear: If the Yetus PMC is aware of regular use of our project in
non-released form we need to take action to stop it.
I *really*, *really* would like to avoid going down the rabbit hole of
ASF rules lawyering on "who's using the artifact" and other ways of attempting
to use semantics to avoid the root cause of yetus needing to release more often.
That will be exhausting, a waste of time that could instead be put towards
actually having releases, and will doom the Yetus project to fail in
its expressed purpose of making software for the public good (rather than for
ASF internal belly-itching).
>
>>
>> The answer to the issue of blocking downstream folks is to release more
>> often.
>>
>> The Yetus PMC has a release cadence goal of weekly. Other projects
>> that want access to more frequent releases can help the situation by
>> helping to vote on releases.
>>
>
> This is great, but I did not see it happen yet. Again, my main goal is to
> unblock current state with the least amount of friction.
>
As I mentioned, the best way to unblock the current state is to help
vote on releases.
Another way to help would be to be more vocal on prioritization for
things going into a release. For example, we have the 0.2.0 vote closing
this coming Friday rather than the prior Monday because the Yetus
community prioritized fixing a few last sharp edges over
the weekend. More voices that state the pressing need for blocking
issues will help the community make calls that are in line with what you desire.
FWIW, I had no visibility that HBase was hitting YETUS-285 and so I
had no reason to push for a sooner release with my HBase hat on.
--
Sean
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
> Such a tag would be a distribution according to ASF rules. The Yetus
> PMC would have to vote on it just as we do releases.
>
Not necessarily. We can depend on a pre-release tag of any of our
dependencies as long as we are not releasing them with our release. We are
not releasing yetus together with HBase. We can depend on any snapshot tag
for our precommit build. I was suggesting that the yetus community have a
guideline on what commit has is the best.
>
> The answer to the issue of blocking downstream folks is to release more
> often.
>
> The Yetus PMC has a release cadence goal of weekly. Other projects
> that want access to more frequent releases can help the situation by
> helping to vote on releases.
>
This is great, but I did not see it happen yet. Again, my main goal is to
unblock current state with the least amount of friction.
>
> On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> > Thanks Chris,
> >
> > This came up before whether to run against a released yetus or not.
> > Personally, I don't care a bit about whether the yetus release we are
> using
> > by default is a released one or trunk or a tag. We are only depending on
> > yetus for the precommit job and not for our distribution artifacts. I can
> > understand the yetus release is taking some time, and voting etc, but we
> > are affectively blocking progress on downstream projects, or making life
> > very inconvenient at the least. The problem is that EVERY hadoopqa run
> has
> > to be re-kicked, and it requires a committer with jenkins access to do
> it.
> >
> > Can we do something like this: yetus community can maintain a stable tag
> > (not necessarily a release) where the tag is used by default for
> downstream
> > projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> > USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in
> precommit
> > jobs. For example, when the docker issue is fixed, the yetus community
> will
> > just change the stable tag to be that point in time.
> >
> > Enis
> >
> > On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Thanks for the information, Chris.
> >>
> >> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> >> uses current
> >> HEAD of apache/yetus.
> >>
> >> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
> >>
> >> Cheers
> >>
> >> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >> wrote:
> >>
> >> > Hi Enis,
> >> >
> >> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >> >
> >> > https://issues.apache.org/jira/browse/YETUS-286
> >> >
> >> >
> >> > We resolved this as a duplicate of a patch that allows control of
> whether
> >> > or not Docker runs in privileged mode:
> >> >
> >> > https://issues.apache.org/jira/browse/YETUS-285
> >> >
> >> >
> >> > I have to admit I'm not enough of a Docker expert to understand this
> fix,
> >> > but you can read through the issue comments from the contributors if
> you
> >> > want details.
> >> >
> >> > Hadoop pre-commit currently runs off of Yetus master, which includes
> this
> >> > patch, and we are no longer seeing the problem. I see HBase
> pre-commit
> >> is
> >> > currently running with Yetus release 0.1.0, which does not have the
> fix.
> >> >
> >> > I think you have 2 options:
> >> >
> >> > 1. Switch HBase pre-commit to run Yetus from the master branch.
> (Please
> >> > be aware that new Yetus commits will go to master though.)
> >> > 2. If you don't like the idea of HBase pre-commit running Yetus code
> that
> >> > is constantly in motion, then you can wait a few days for sign-off of
> our
> >> > 0.2.0 release candidate, which is currently under [VOTE] and looking
> good
> >> > so far.
> >> >
> >> > --Chris Nauroth
> >> >
> >> >
> >> >
> >> >
> >> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >> >
> >> > >HBase precommit tests have been failing on some nodes for a couple of
> >> days
> >> > >related to this message:
> >> > >
> >> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >> > >Error while running command to get file permissions :
> >> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >> > >libraries: libacl.so.1: failed to map segment from shared object:
> >> > >Permission denied
> >> > >
> >> > >
> >> > >It is not clear to me whether this is a docker / yetus problem, or an
> >> > >INFRA
> >> > >issue. Anybody familiar with this? Obviously, it makes it very hard
> to
> >> > >commit a patch without the precommit tests. I've seen this on at
> least 2
> >> > >nodes.
> >> > >
> >> > >Sample output:
> >> > >
> >> >
> >>
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >> > >
> >> > >Enis
> >> >
> >> >
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
On Wed, Mar 2, 2016 at 11:53 AM, Sean Busbey <bu...@apache.org> wrote:
> Such a tag would be a distribution according to ASF rules. The Yetus
> PMC would have to vote on it just as we do releases.
>
Not necessarily. We can depend on a pre-release tag of any of our
dependencies as long as we are not releasing them with our release. We are
not releasing yetus together with HBase. We can depend on any snapshot tag
for our precommit build. I was suggesting that the yetus community have a
guideline on what commit has is the best.
>
> The answer to the issue of blocking downstream folks is to release more
> often.
>
> The Yetus PMC has a release cadence goal of weekly. Other projects
> that want access to more frequent releases can help the situation by
> helping to vote on releases.
>
This is great, but I did not see it happen yet. Again, my main goal is to
unblock current state with the least amount of friction.
>
> On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> > Thanks Chris,
> >
> > This came up before whether to run against a released yetus or not.
> > Personally, I don't care a bit about whether the yetus release we are
> using
> > by default is a released one or trunk or a tag. We are only depending on
> > yetus for the precommit job and not for our distribution artifacts. I can
> > understand the yetus release is taking some time, and voting etc, but we
> > are affectively blocking progress on downstream projects, or making life
> > very inconvenient at the least. The problem is that EVERY hadoopqa run
> has
> > to be re-kicked, and it requires a committer with jenkins access to do
> it.
> >
> > Can we do something like this: yetus community can maintain a stable tag
> > (not necessarily a release) where the tag is used by default for
> downstream
> > projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> > USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in
> precommit
> > jobs. For example, when the docker issue is fixed, the yetus community
> will
> > just change the stable tag to be that point in time.
> >
> > Enis
> >
> > On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Thanks for the information, Chris.
> >>
> >> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> >> uses current
> >> HEAD of apache/yetus.
> >>
> >> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
> >>
> >> Cheers
> >>
> >> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >> wrote:
> >>
> >> > Hi Enis,
> >> >
> >> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >> >
> >> > https://issues.apache.org/jira/browse/YETUS-286
> >> >
> >> >
> >> > We resolved this as a duplicate of a patch that allows control of
> whether
> >> > or not Docker runs in privileged mode:
> >> >
> >> > https://issues.apache.org/jira/browse/YETUS-285
> >> >
> >> >
> >> > I have to admit I'm not enough of a Docker expert to understand this
> fix,
> >> > but you can read through the issue comments from the contributors if
> you
> >> > want details.
> >> >
> >> > Hadoop pre-commit currently runs off of Yetus master, which includes
> this
> >> > patch, and we are no longer seeing the problem. I see HBase
> pre-commit
> >> is
> >> > currently running with Yetus release 0.1.0, which does not have the
> fix.
> >> >
> >> > I think you have 2 options:
> >> >
> >> > 1. Switch HBase pre-commit to run Yetus from the master branch.
> (Please
> >> > be aware that new Yetus commits will go to master though.)
> >> > 2. If you don't like the idea of HBase pre-commit running Yetus code
> that
> >> > is constantly in motion, then you can wait a few days for sign-off of
> our
> >> > 0.2.0 release candidate, which is currently under [VOTE] and looking
> good
> >> > so far.
> >> >
> >> > --Chris Nauroth
> >> >
> >> >
> >> >
> >> >
> >> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >> >
> >> > >HBase precommit tests have been failing on some nodes for a couple of
> >> days
> >> > >related to this message:
> >> > >
> >> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >> > >Error while running command to get file permissions :
> >> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >> > >libraries: libacl.so.1: failed to map segment from shared object:
> >> > >Permission denied
> >> > >
> >> > >
> >> > >It is not clear to me whether this is a docker / yetus problem, or an
> >> > >INFRA
> >> > >issue. Anybody familiar with this? Obviously, it makes it very hard
> to
> >> > >commit a patch without the precommit tests. I've seen this on at
> least 2
> >> > >nodes.
> >> > >
> >> > >Sample output:
> >> > >
> >> >
> >>
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >> > >
> >> > >Enis
> >> >
> >> >
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Andrew Bayer <an...@gmail.com>.
Yup.
On Mar 2, 2016 12:26, "Sean Busbey" <bu...@apache.org> wrote:
> The conversation around the demand for such a service should come from who?
>
> PMCs that want to use Yetus, I think?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com>
> wrote:
> > Speaking with my infra hat on -
> >
> > I'd like to see us end up going in that direction. But yeah, it'll need
> > committed resources from infra, meaning cycles from paid infra staff and
> > those staff being able to handle the work without needing to resort to
> > volunteer assistance for day to day activities. Starting a discussion
> with
> > infra about the possibility would be a good idea.
> >
> > A.
> > On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> >
> >> Another option would be to request that infra@asf provide "yetus based
> >> precommit testing" as an SLA governed service. Then they could
> >> maintain configuration management of running the tests with known good
> >> versions and rolling them out across projects.
> >>
> >> AFAIK, the backlog on infra is pretty severe though, so requests are
> >> likely to get backlogged if they don't come with a volunteer.
> >>
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Andrew Bayer <an...@gmail.com>.
Yup.
On Mar 2, 2016 12:26, "Sean Busbey" <bu...@apache.org> wrote:
> The conversation around the demand for such a service should come from who?
>
> PMCs that want to use Yetus, I think?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com>
> wrote:
> > Speaking with my infra hat on -
> >
> > I'd like to see us end up going in that direction. But yeah, it'll need
> > committed resources from infra, meaning cycles from paid infra staff and
> > those staff being able to handle the work without needing to resort to
> > volunteer assistance for day to day activities. Starting a discussion
> with
> > infra about the possibility would be a good idea.
> >
> > A.
> > On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> >
> >> Another option would be to request that infra@asf provide "yetus based
> >> precommit testing" as an SLA governed service. Then they could
> >> maintain configuration management of running the tests with known good
> >> versions and rolling them out across projects.
> >>
> >> AFAIK, the backlog on infra is pretty severe though, so requests are
> >> likely to get backlogged if they don't come with a volunteer.
> >>
> >>
>
Re: /bin/ls permission issue on precommit tests
Posted by Mikhail Antonov <ol...@gmail.com>.
"If the Yetus PMC is aware of regular use of our project in
non-released form we need to take action to stop it."
I'm not sure I understand that part though - is that specific to Yetus, or
is it something ASF-level? Let's take Hadoop as an example. I could build
it from trunk's HEAD once a week and deploy on production cluster, nobody
can/should stop me from doing that, right?
On Wed, Mar 2, 2016 at 12:26 PM, Sean Busbey <bu...@apache.org> wrote:
> The conversation around the demand for such a service should come from who?
>
> PMCs that want to use Yetus, I think?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com>
> wrote:
> > Speaking with my infra hat on -
> >
> > I'd like to see us end up going in that direction. But yeah, it'll need
> > committed resources from infra, meaning cycles from paid infra staff and
> > those staff being able to handle the work without needing to resort to
> > volunteer assistance for day to day activities. Starting a discussion
> with
> > infra about the possibility would be a good idea.
> >
> > A.
> > On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> >
> >> Another option would be to request that infra@asf provide "yetus based
> >> precommit testing" as an SLA governed service. Then they could
> >> maintain configuration management of running the tests with known good
> >> versions and rolling them out across projects.
> >>
> >> AFAIK, the backlog on infra is pretty severe though, so requests are
> >> likely to get backlogged if they don't come with a volunteer.
> >>
> >>
>
--
Thanks,
Michael Antonov
Re: /bin/ls permission issue on precommit tests
Posted by Mikhail Antonov <ol...@gmail.com>.
"If the Yetus PMC is aware of regular use of our project in
non-released form we need to take action to stop it."
I'm not sure I understand that part though - is that specific to Yetus, or
is it something ASF-level? Let's take Hadoop as an example. I could build
it from trunk's HEAD once a week and deploy on production cluster, nobody
can/should stop me from doing that, right?
On Wed, Mar 2, 2016 at 12:26 PM, Sean Busbey <bu...@apache.org> wrote:
> The conversation around the demand for such a service should come from who?
>
> PMCs that want to use Yetus, I think?
>
> -Sean
>
> On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com>
> wrote:
> > Speaking with my infra hat on -
> >
> > I'd like to see us end up going in that direction. But yeah, it'll need
> > committed resources from infra, meaning cycles from paid infra staff and
> > those staff being able to handle the work without needing to resort to
> > volunteer assistance for day to day activities. Starting a discussion
> with
> > infra about the possibility would be a good idea.
> >
> > A.
> > On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> >
> >> Another option would be to request that infra@asf provide "yetus based
> >> precommit testing" as an SLA governed service. Then they could
> >> maintain configuration management of running the tests with known good
> >> versions and rolling them out across projects.
> >>
> >> AFAIK, the backlog on infra is pretty severe though, so requests are
> >> likely to get backlogged if they don't come with a volunteer.
> >>
> >>
>
--
Thanks,
Michael Antonov
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
The conversation around the demand for such a service should come from who?
PMCs that want to use Yetus, I think?
-Sean
On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com> wrote:
> Speaking with my infra hat on -
>
> I'd like to see us end up going in that direction. But yeah, it'll need
> committed resources from infra, meaning cycles from paid infra staff and
> those staff being able to handle the work without needing to resort to
> volunteer assistance for day to day activities. Starting a discussion with
> infra about the possibility would be a good idea.
>
> A.
> On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
>
>> Another option would be to request that infra@asf provide "yetus based
>> precommit testing" as an SLA governed service. Then they could
>> maintain configuration management of running the tests with known good
>> versions and rolling them out across projects.
>>
>> AFAIK, the backlog on infra is pretty severe though, so requests are
>> likely to get backlogged if they don't come with a volunteer.
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
The conversation around the demand for such a service should come from who?
PMCs that want to use Yetus, I think?
-Sean
On Wed, Mar 2, 2016 at 2:05 PM, Andrew Bayer <an...@gmail.com> wrote:
> Speaking with my infra hat on -
>
> I'd like to see us end up going in that direction. But yeah, it'll need
> committed resources from infra, meaning cycles from paid infra staff and
> those staff being able to handle the work without needing to resort to
> volunteer assistance for day to day activities. Starting a discussion with
> infra about the possibility would be a good idea.
>
> A.
> On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
>
>> Another option would be to request that infra@asf provide "yetus based
>> precommit testing" as an SLA governed service. Then they could
>> maintain configuration management of running the tests with known good
>> versions and rolling them out across projects.
>>
>> AFAIK, the backlog on infra is pretty severe though, so requests are
>> likely to get backlogged if they don't come with a volunteer.
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Andrew Bayer <an...@gmail.com>.
Speaking with my infra hat on -
I'd like to see us end up going in that direction. But yeah, it'll need
committed resources from infra, meaning cycles from paid infra staff and
those staff being able to handle the work without needing to resort to
volunteer assistance for day to day activities. Starting a discussion with
infra about the possibility would be a good idea.
A.
On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> Another option would be to request that infra@asf provide "yetus based
> precommit testing" as an SLA governed service. Then they could
> maintain configuration management of running the tests with known good
> versions and rolling them out across projects.
>
> AFAIK, the backlog on infra is pretty severe though, so requests are
> likely to get backlogged if they don't come with a volunteer.
>
> On Wed, Mar 2, 2016 at 1:53 PM, Sean Busbey <bu...@apache.org> wrote:
> > Such a tag would be a distribution according to ASF rules. The Yetus
> > PMC would have to vote on it just as we do releases.
> >
> > The answer to the issue of blocking downstream folks is to release more
> often.
> >
> > The Yetus PMC has a release cadence goal of weekly. Other projects
> > that want access to more frequent releases can help the situation by
> > helping to vote on releases.
> >
> > On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >> Thanks Chris,
> >>
> >> This came up before whether to run against a released yetus or not.
> >> Personally, I don't care a bit about whether the yetus release we are
> using
> >> by default is a released one or trunk or a tag. We are only depending on
> >> yetus for the precommit job and not for our distribution artifacts. I
> can
> >> understand the yetus release is taking some time, and voting etc, but we
> >> are affectively blocking progress on downstream projects, or making life
> >> very inconvenient at the least. The problem is that EVERY hadoopqa run
> has
> >> to be re-kicked, and it requires a committer with jenkins access to do
> it.
> >>
> >> Can we do something like this: yetus community can maintain a stable tag
> >> (not necessarily a release) where the tag is used by default for
> downstream
> >> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> >> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in
> precommit
> >> jobs. For example, when the docker issue is fixed, the yetus community
> will
> >> just change the stable tag to be that point in time.
> >>
> >> Enis
> >>
> >> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >>> Thanks for the information, Chris.
> >>>
> >>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> >>> uses current
> >>> HEAD of apache/yetus.
> >>>
> >>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
> >>>
> >>> Cheers
> >>>
> >>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >>> wrote:
> >>>
> >>> > Hi Enis,
> >>> >
> >>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >>> >
> >>> > https://issues.apache.org/jira/browse/YETUS-286
> >>> >
> >>> >
> >>> > We resolved this as a duplicate of a patch that allows control of
> whether
> >>> > or not Docker runs in privileged mode:
> >>> >
> >>> > https://issues.apache.org/jira/browse/YETUS-285
> >>> >
> >>> >
> >>> > I have to admit I'm not enough of a Docker expert to understand this
> fix,
> >>> > but you can read through the issue comments from the contributors if
> you
> >>> > want details.
> >>> >
> >>> > Hadoop pre-commit currently runs off of Yetus master, which includes
> this
> >>> > patch, and we are no longer seeing the problem. I see HBase
> pre-commit
> >>> is
> >>> > currently running with Yetus release 0.1.0, which does not have the
> fix.
> >>> >
> >>> > I think you have 2 options:
> >>> >
> >>> > 1. Switch HBase pre-commit to run Yetus from the master branch.
> (Please
> >>> > be aware that new Yetus commits will go to master though.)
> >>> > 2. If you don't like the idea of HBase pre-commit running Yetus code
> that
> >>> > is constantly in motion, then you can wait a few days for sign-off
> of our
> >>> > 0.2.0 release candidate, which is currently under [VOTE] and looking
> good
> >>> > so far.
> >>> >
> >>> > --Chris Nauroth
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >>> >
> >>> > >HBase precommit tests have been failing on some nodes for a couple
> of
> >>> days
> >>> > >related to this message:
> >>> > >
> >>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >>> > >Error while running command to get file permissions :
> >>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >>> > >libraries: libacl.so.1: failed to map segment from shared object:
> >>> > >Permission denied
> >>> > >
> >>> > >
> >>> > >It is not clear to me whether this is a docker / yetus problem, or
> an
> >>> > >INFRA
> >>> > >issue. Anybody familiar with this? Obviously, it makes it very hard
> to
> >>> > >commit a patch without the precommit tests. I've seen this on at
> least 2
> >>> > >nodes.
> >>> > >
> >>> > >Sample output:
> >>> > >
> >>> >
> >>>
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >>> > >
> >>> > >Enis
> >>> >
> >>> >
> >>>
>
Re: /bin/ls permission issue on precommit tests
Posted by Andrew Bayer <an...@gmail.com>.
Speaking with my infra hat on -
I'd like to see us end up going in that direction. But yeah, it'll need
committed resources from infra, meaning cycles from paid infra staff and
those staff being able to handle the work without needing to resort to
volunteer assistance for day to day activities. Starting a discussion with
infra about the possibility would be a good idea.
A.
On Mar 2, 2016 12:00, "Sean Busbey" <bu...@apache.org> wrote:
> Another option would be to request that infra@asf provide "yetus based
> precommit testing" as an SLA governed service. Then they could
> maintain configuration management of running the tests with known good
> versions and rolling them out across projects.
>
> AFAIK, the backlog on infra is pretty severe though, so requests are
> likely to get backlogged if they don't come with a volunteer.
>
> On Wed, Mar 2, 2016 at 1:53 PM, Sean Busbey <bu...@apache.org> wrote:
> > Such a tag would be a distribution according to ASF rules. The Yetus
> > PMC would have to vote on it just as we do releases.
> >
> > The answer to the issue of blocking downstream folks is to release more
> often.
> >
> > The Yetus PMC has a release cadence goal of weekly. Other projects
> > that want access to more frequent releases can help the situation by
> > helping to vote on releases.
> >
> > On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >> Thanks Chris,
> >>
> >> This came up before whether to run against a released yetus or not.
> >> Personally, I don't care a bit about whether the yetus release we are
> using
> >> by default is a released one or trunk or a tag. We are only depending on
> >> yetus for the precommit job and not for our distribution artifacts. I
> can
> >> understand the yetus release is taking some time, and voting etc, but we
> >> are affectively blocking progress on downstream projects, or making life
> >> very inconvenient at the least. The problem is that EVERY hadoopqa run
> has
> >> to be re-kicked, and it requires a committer with jenkins access to do
> it.
> >>
> >> Can we do something like this: yetus community can maintain a stable tag
> >> (not necessarily a release) where the tag is used by default for
> downstream
> >> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> >> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in
> precommit
> >> jobs. For example, when the docker issue is fixed, the yetus community
> will
> >> just change the stable tag to be that point in time.
> >>
> >> Enis
> >>
> >> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >>> Thanks for the information, Chris.
> >>>
> >>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> >>> uses current
> >>> HEAD of apache/yetus.
> >>>
> >>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
> >>>
> >>> Cheers
> >>>
> >>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <
> cnauroth@hortonworks.com>
> >>> wrote:
> >>>
> >>> > Hi Enis,
> >>> >
> >>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >>> >
> >>> > https://issues.apache.org/jira/browse/YETUS-286
> >>> >
> >>> >
> >>> > We resolved this as a duplicate of a patch that allows control of
> whether
> >>> > or not Docker runs in privileged mode:
> >>> >
> >>> > https://issues.apache.org/jira/browse/YETUS-285
> >>> >
> >>> >
> >>> > I have to admit I'm not enough of a Docker expert to understand this
> fix,
> >>> > but you can read through the issue comments from the contributors if
> you
> >>> > want details.
> >>> >
> >>> > Hadoop pre-commit currently runs off of Yetus master, which includes
> this
> >>> > patch, and we are no longer seeing the problem. I see HBase
> pre-commit
> >>> is
> >>> > currently running with Yetus release 0.1.0, which does not have the
> fix.
> >>> >
> >>> > I think you have 2 options:
> >>> >
> >>> > 1. Switch HBase pre-commit to run Yetus from the master branch.
> (Please
> >>> > be aware that new Yetus commits will go to master though.)
> >>> > 2. If you don't like the idea of HBase pre-commit running Yetus code
> that
> >>> > is constantly in motion, then you can wait a few days for sign-off
> of our
> >>> > 0.2.0 release candidate, which is currently under [VOTE] and looking
> good
> >>> > so far.
> >>> >
> >>> > --Chris Nauroth
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >>> >
> >>> > >HBase precommit tests have been failing on some nodes for a couple
> of
> >>> days
> >>> > >related to this message:
> >>> > >
> >>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >>> > >Error while running command to get file permissions :
> >>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >>> > >libraries: libacl.so.1: failed to map segment from shared object:
> >>> > >Permission denied
> >>> > >
> >>> > >
> >>> > >It is not clear to me whether this is a docker / yetus problem, or
> an
> >>> > >INFRA
> >>> > >issue. Anybody familiar with this? Obviously, it makes it very hard
> to
> >>> > >commit a patch without the precommit tests. I've seen this on at
> least 2
> >>> > >nodes.
> >>> > >
> >>> > >Sample output:
> >>> > >
> >>> >
> >>>
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >>> > >
> >>> > >Enis
> >>> >
> >>> >
> >>>
>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Another option would be to request that infra@asf provide "yetus based
precommit testing" as an SLA governed service. Then they could
maintain configuration management of running the tests with known good
versions and rolling them out across projects.
AFAIK, the backlog on infra is pretty severe though, so requests are
likely to get backlogged if they don't come with a volunteer.
On Wed, Mar 2, 2016 at 1:53 PM, Sean Busbey <bu...@apache.org> wrote:
> Such a tag would be a distribution according to ASF rules. The Yetus
> PMC would have to vote on it just as we do releases.
>
> The answer to the issue of blocking downstream folks is to release more often.
>
> The Yetus PMC has a release cadence goal of weekly. Other projects
> that want access to more frequent releases can help the situation by
> helping to vote on releases.
>
> On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
>> Thanks Chris,
>>
>> This came up before whether to run against a released yetus or not.
>> Personally, I don't care a bit about whether the yetus release we are using
>> by default is a released one or trunk or a tag. We are only depending on
>> yetus for the precommit job and not for our distribution artifacts. I can
>> understand the yetus release is taking some time, and voting etc, but we
>> are affectively blocking progress on downstream projects, or making life
>> very inconvenient at the least. The problem is that EVERY hadoopqa run has
>> to be re-kicked, and it requires a committer with jenkins access to do it.
>>
>> Can we do something like this: yetus community can maintain a stable tag
>> (not necessarily a release) where the tag is used by default for downstream
>> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
>> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
>> jobs. For example, when the docker issue is fixed, the yetus community will
>> just change the stable tag to be that point in time.
>>
>> Enis
>>
>> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Thanks for the information, Chris.
>>>
>>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
>>> uses current
>>> HEAD of apache/yetus.
>>>
>>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>>>
>>> Cheers
>>>
>>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>> > Hi Enis,
>>> >
>>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
>>> >
>>> > https://issues.apache.org/jira/browse/YETUS-286
>>> >
>>> >
>>> > We resolved this as a duplicate of a patch that allows control of whether
>>> > or not Docker runs in privileged mode:
>>> >
>>> > https://issues.apache.org/jira/browse/YETUS-285
>>> >
>>> >
>>> > I have to admit I'm not enough of a Docker expert to understand this fix,
>>> > but you can read through the issue comments from the contributors if you
>>> > want details.
>>> >
>>> > Hadoop pre-commit currently runs off of Yetus master, which includes this
>>> > patch, and we are no longer seeing the problem. I see HBase pre-commit
>>> is
>>> > currently running with Yetus release 0.1.0, which does not have the fix.
>>> >
>>> > I think you have 2 options:
>>> >
>>> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>>> > be aware that new Yetus commits will go to master though.)
>>> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
>>> > is constantly in motion, then you can wait a few days for sign-off of our
>>> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
>>> > so far.
>>> >
>>> > --Chris Nauroth
>>> >
>>> >
>>> >
>>> >
>>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>>> >
>>> > >HBase precommit tests have been failing on some nodes for a couple of
>>> days
>>> > >related to this message:
>>> > >
>>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>>> > >Error while running command to get file permissions :
>>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>>> > >libraries: libacl.so.1: failed to map segment from shared object:
>>> > >Permission denied
>>> > >
>>> > >
>>> > >It is not clear to me whether this is a docker / yetus problem, or an
>>> > >INFRA
>>> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
>>> > >commit a patch without the precommit tests. I've seen this on at least 2
>>> > >nodes.
>>> > >
>>> > >Sample output:
>>> > >
>>> >
>>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>>> > >
>>> > >Enis
>>> >
>>> >
>>>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Another option would be to request that infra@asf provide "yetus based
precommit testing" as an SLA governed service. Then they could
maintain configuration management of running the tests with known good
versions and rolling them out across projects.
AFAIK, the backlog on infra is pretty severe though, so requests are
likely to get backlogged if they don't come with a volunteer.
On Wed, Mar 2, 2016 at 1:53 PM, Sean Busbey <bu...@apache.org> wrote:
> Such a tag would be a distribution according to ASF rules. The Yetus
> PMC would have to vote on it just as we do releases.
>
> The answer to the issue of blocking downstream folks is to release more often.
>
> The Yetus PMC has a release cadence goal of weekly. Other projects
> that want access to more frequent releases can help the situation by
> helping to vote on releases.
>
> On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
>> Thanks Chris,
>>
>> This came up before whether to run against a released yetus or not.
>> Personally, I don't care a bit about whether the yetus release we are using
>> by default is a released one or trunk or a tag. We are only depending on
>> yetus for the precommit job and not for our distribution artifacts. I can
>> understand the yetus release is taking some time, and voting etc, but we
>> are affectively blocking progress on downstream projects, or making life
>> very inconvenient at the least. The problem is that EVERY hadoopqa run has
>> to be re-kicked, and it requires a committer with jenkins access to do it.
>>
>> Can we do something like this: yetus community can maintain a stable tag
>> (not necessarily a release) where the tag is used by default for downstream
>> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
>> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
>> jobs. For example, when the docker issue is fixed, the yetus community will
>> just change the stable tag to be that point in time.
>>
>> Enis
>>
>> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Thanks for the information, Chris.
>>>
>>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
>>> uses current
>>> HEAD of apache/yetus.
>>>
>>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>>>
>>> Cheers
>>>
>>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>> > Hi Enis,
>>> >
>>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
>>> >
>>> > https://issues.apache.org/jira/browse/YETUS-286
>>> >
>>> >
>>> > We resolved this as a duplicate of a patch that allows control of whether
>>> > or not Docker runs in privileged mode:
>>> >
>>> > https://issues.apache.org/jira/browse/YETUS-285
>>> >
>>> >
>>> > I have to admit I'm not enough of a Docker expert to understand this fix,
>>> > but you can read through the issue comments from the contributors if you
>>> > want details.
>>> >
>>> > Hadoop pre-commit currently runs off of Yetus master, which includes this
>>> > patch, and we are no longer seeing the problem. I see HBase pre-commit
>>> is
>>> > currently running with Yetus release 0.1.0, which does not have the fix.
>>> >
>>> > I think you have 2 options:
>>> >
>>> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>>> > be aware that new Yetus commits will go to master though.)
>>> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
>>> > is constantly in motion, then you can wait a few days for sign-off of our
>>> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
>>> > so far.
>>> >
>>> > --Chris Nauroth
>>> >
>>> >
>>> >
>>> >
>>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>>> >
>>> > >HBase precommit tests have been failing on some nodes for a couple of
>>> days
>>> > >related to this message:
>>> > >
>>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>>> > >Error while running command to get file permissions :
>>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>>> > >libraries: libacl.so.1: failed to map segment from shared object:
>>> > >Permission denied
>>> > >
>>> > >
>>> > >It is not clear to me whether this is a docker / yetus problem, or an
>>> > >INFRA
>>> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
>>> > >commit a patch without the precommit tests. I've seen this on at least 2
>>> > >nodes.
>>> > >
>>> > >Sample output:
>>> > >
>>> >
>>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>>> > >
>>> > >Enis
>>> >
>>> >
>>>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Such a tag would be a distribution according to ASF rules. The Yetus
PMC would have to vote on it just as we do releases.
The answer to the issue of blocking downstream folks is to release more often.
The Yetus PMC has a release cadence goal of weekly. Other projects
that want access to more frequent releases can help the situation by
helping to vote on releases.
On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> Thanks Chris,
>
> This came up before whether to run against a released yetus or not.
> Personally, I don't care a bit about whether the yetus release we are using
> by default is a released one or trunk or a tag. We are only depending on
> yetus for the precommit job and not for our distribution artifacts. I can
> understand the yetus release is taking some time, and voting etc, but we
> are affectively blocking progress on downstream projects, or making life
> very inconvenient at the least. The problem is that EVERY hadoopqa run has
> to be re-kicked, and it requires a committer with jenkins access to do it.
>
> Can we do something like this: yetus community can maintain a stable tag
> (not necessarily a release) where the tag is used by default for downstream
> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
> jobs. For example, when the docker issue is fixed, the yetus community will
> just change the stable tag to be that point in time.
>
> Enis
>
> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Thanks for the information, Chris.
>>
>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
>> uses current
>> HEAD of apache/yetus.
>>
>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>>
>> Cheers
>>
>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>> > Hi Enis,
>> >
>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
>> >
>> > https://issues.apache.org/jira/browse/YETUS-286
>> >
>> >
>> > We resolved this as a duplicate of a patch that allows control of whether
>> > or not Docker runs in privileged mode:
>> >
>> > https://issues.apache.org/jira/browse/YETUS-285
>> >
>> >
>> > I have to admit I'm not enough of a Docker expert to understand this fix,
>> > but you can read through the issue comments from the contributors if you
>> > want details.
>> >
>> > Hadoop pre-commit currently runs off of Yetus master, which includes this
>> > patch, and we are no longer seeing the problem. I see HBase pre-commit
>> is
>> > currently running with Yetus release 0.1.0, which does not have the fix.
>> >
>> > I think you have 2 options:
>> >
>> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>> > be aware that new Yetus commits will go to master though.)
>> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
>> > is constantly in motion, then you can wait a few days for sign-off of our
>> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
>> > so far.
>> >
>> > --Chris Nauroth
>> >
>> >
>> >
>> >
>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>> >
>> > >HBase precommit tests have been failing on some nodes for a couple of
>> days
>> > >related to this message:
>> > >
>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> > >Error while running command to get file permissions :
>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>> > >libraries: libacl.so.1: failed to map segment from shared object:
>> > >Permission denied
>> > >
>> > >
>> > >It is not clear to me whether this is a docker / yetus problem, or an
>> > >INFRA
>> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
>> > >commit a patch without the precommit tests. I've seen this on at least 2
>> > >nodes.
>> > >
>> > >Sample output:
>> > >
>> >
>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>> > >
>> > >Enis
>> >
>> >
>>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Such a tag would be a distribution according to ASF rules. The Yetus
PMC would have to vote on it just as we do releases.
The answer to the issue of blocking downstream folks is to release more often.
The Yetus PMC has a release cadence goal of weekly. Other projects
that want access to more frequent releases can help the situation by
helping to vote on releases.
On Wed, Mar 2, 2016 at 1:49 PM, Enis Söztutar <en...@gmail.com> wrote:
> Thanks Chris,
>
> This came up before whether to run against a released yetus or not.
> Personally, I don't care a bit about whether the yetus release we are using
> by default is a released one or trunk or a tag. We are only depending on
> yetus for the precommit job and not for our distribution artifacts. I can
> understand the yetus release is taking some time, and voting etc, but we
> are affectively blocking progress on downstream projects, or making life
> very inconvenient at the least. The problem is that EVERY hadoopqa run has
> to be re-kicked, and it requires a committer with jenkins access to do it.
>
> Can we do something like this: yetus community can maintain a stable tag
> (not necessarily a release) where the tag is used by default for downstream
> projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
> USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
> jobs. For example, when the docker issue is fixed, the yetus community will
> just change the stable tag to be that point in time.
>
> Enis
>
> On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Thanks for the information, Chris.
>>
>> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
>> uses current
>> HEAD of apache/yetus.
>>
>> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>>
>> Cheers
>>
>> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>> > Hi Enis,
>> >
>> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
>> >
>> > https://issues.apache.org/jira/browse/YETUS-286
>> >
>> >
>> > We resolved this as a duplicate of a patch that allows control of whether
>> > or not Docker runs in privileged mode:
>> >
>> > https://issues.apache.org/jira/browse/YETUS-285
>> >
>> >
>> > I have to admit I'm not enough of a Docker expert to understand this fix,
>> > but you can read through the issue comments from the contributors if you
>> > want details.
>> >
>> > Hadoop pre-commit currently runs off of Yetus master, which includes this
>> > patch, and we are no longer seeing the problem. I see HBase pre-commit
>> is
>> > currently running with Yetus release 0.1.0, which does not have the fix.
>> >
>> > I think you have 2 options:
>> >
>> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>> > be aware that new Yetus commits will go to master though.)
>> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
>> > is constantly in motion, then you can wait a few days for sign-off of our
>> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
>> > so far.
>> >
>> > --Chris Nauroth
>> >
>> >
>> >
>> >
>> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>> >
>> > >HBase precommit tests have been failing on some nodes for a couple of
>> days
>> > >related to this message:
>> > >
>> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> > >Error while running command to get file permissions :
>> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>> > >libraries: libacl.so.1: failed to map segment from shared object:
>> > >Permission denied
>> > >
>> > >
>> > >It is not clear to me whether this is a docker / yetus problem, or an
>> > >INFRA
>> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
>> > >commit a patch without the precommit tests. I've seen this on at least 2
>> > >nodes.
>> > >
>> > >Sample output:
>> > >
>> >
>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>> > >
>> > >Enis
>> >
>> >
>>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
Thanks Chris,
This came up before whether to run against a released yetus or not.
Personally, I don't care a bit about whether the yetus release we are using
by default is a released one or trunk or a tag. We are only depending on
yetus for the precommit job and not for our distribution artifacts. I can
understand the yetus release is taking some time, and voting etc, but we
are affectively blocking progress on downstream projects, or making life
very inconvenient at the least. The problem is that EVERY hadoopqa run has
to be re-kicked, and it requires a committer with jenkins access to do it.
Can we do something like this: yetus community can maintain a stable tag
(not necessarily a release) where the tag is used by default for downstream
projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
jobs. For example, when the docker issue is fixed, the yetus community will
just change the stable tag to be that point in time.
Enis
On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the information, Chris.
>
> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> uses current
> HEAD of apache/yetus.
>
> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>
> Cheers
>
> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
> > Hi Enis,
> >
> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >
> > https://issues.apache.org/jira/browse/YETUS-286
> >
> >
> > We resolved this as a duplicate of a patch that allows control of whether
> > or not Docker runs in privileged mode:
> >
> > https://issues.apache.org/jira/browse/YETUS-285
> >
> >
> > I have to admit I'm not enough of a Docker expert to understand this fix,
> > but you can read through the issue comments from the contributors if you
> > want details.
> >
> > Hadoop pre-commit currently runs off of Yetus master, which includes this
> > patch, and we are no longer seeing the problem. I see HBase pre-commit
> is
> > currently running with Yetus release 0.1.0, which does not have the fix.
> >
> > I think you have 2 options:
> >
> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
> > be aware that new Yetus commits will go to master though.)
> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
> > is constantly in motion, then you can wait a few days for sign-off of our
> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
> > so far.
> >
> > --Chris Nauroth
> >
> >
> >
> >
> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >
> > >HBase precommit tests have been failing on some nodes for a couple of
> days
> > >related to this message:
> > >
> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> > >Error while running command to get file permissions :
> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> > >libraries: libacl.so.1: failed to map segment from shared object:
> > >Permission denied
> > >
> > >
> > >It is not clear to me whether this is a docker / yetus problem, or an
> > >INFRA
> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
> > >commit a patch without the precommit tests. I've seen this on at least 2
> > >nodes.
> > >
> > >Sample output:
> > >
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> > >
> > >Enis
> >
> >
>
Re: /bin/ls permission issue on precommit tests
Posted by Sean Busbey <bu...@apache.org>.
Please remember to only use the prerelease check box on a case-by-case
basis when checking to see if a yetus issue is resolved for hbase's
use.
On Wed, Mar 2, 2016 at 1:33 PM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the information, Chris.
>
> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> uses current
> HEAD of apache/yetus.
>
> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>
> Cheers
>
> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hi Enis,
>>
>> Hadoop pre-commit was experiencing the same problem a few weeks ago:
>>
>> https://issues.apache.org/jira/browse/YETUS-286
>>
>>
>> We resolved this as a duplicate of a patch that allows control of whether
>> or not Docker runs in privileged mode:
>>
>> https://issues.apache.org/jira/browse/YETUS-285
>>
>>
>> I have to admit I'm not enough of a Docker expert to understand this fix,
>> but you can read through the issue comments from the contributors if you
>> want details.
>>
>> Hadoop pre-commit currently runs off of Yetus master, which includes this
>> patch, and we are no longer seeing the problem. I see HBase pre-commit is
>> currently running with Yetus release 0.1.0, which does not have the fix.
>>
>> I think you have 2 options:
>>
>> 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
>> be aware that new Yetus commits will go to master though.)
>> 2. If you don't like the idea of HBase pre-commit running Yetus code that
>> is constantly in motion, then you can wait a few days for sign-off of our
>> 0.2.0 release candidate, which is currently under [VOTE] and looking good
>> so far.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>>
>> >HBase precommit tests have been failing on some nodes for a couple of days
>> >related to this message:
>> >
>> >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> >Error while running command to get file permissions :
>> >ExitCodeException exitCode=127: /bin/ls: error while loading shared
>> >libraries: libacl.so.1: failed to map segment from shared object:
>> >Permission denied
>> >
>> >
>> >It is not clear to me whether this is a docker / yetus problem, or an
>> >INFRA
>> >issue. Anybody familiar with this? Obviously, it makes it very hard to
>> >commit a patch without the precommit tests. I've seen this on at least 2
>> >nodes.
>> >
>> >Sample output:
>> >
>> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>> >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>> >
>> >Enis
>>
>>
Re: /bin/ls permission issue on precommit tests
Posted by Enis Söztutar <en...@gmail.com>.
Thanks Chris,
This came up before whether to run against a released yetus or not.
Personally, I don't care a bit about whether the yetus release we are using
by default is a released one or trunk or a tag. We are only depending on
yetus for the precommit job and not for our distribution artifacts. I can
understand the yetus release is taking some time, and voting etc, but we
are affectively blocking progress on downstream projects, or making life
very inconvenient at the least. The problem is that EVERY hadoopqa run has
to be re-kicked, and it requires a committer with jenkins access to do it.
Can we do something like this: yetus community can maintain a stable tag
(not necessarily a release) where the tag is used by default for downstream
projects. We can have USE_YETUS_RELEASE, USE_STABLE_YETUS and
USE_YETUS_PRERELEASE, and have USE_STABLE_YETUS as the default in precommit
jobs. For example, when the docker issue is fixed, the yetus community will
just change the stable tag to be that point in time.
Enis
On Wed, Mar 2, 2016 at 11:33 AM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the information, Chris.
>
> For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
> uses current
> HEAD of apache/yetus.
>
> Before Yetus 0.2.0 comes out, we can utilize this checkbox.
>
> Cheers
>
> On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
> > Hi Enis,
> >
> > Hadoop pre-commit was experiencing the same problem a few weeks ago:
> >
> > https://issues.apache.org/jira/browse/YETUS-286
> >
> >
> > We resolved this as a duplicate of a patch that allows control of whether
> > or not Docker runs in privileged mode:
> >
> > https://issues.apache.org/jira/browse/YETUS-285
> >
> >
> > I have to admit I'm not enough of a Docker expert to understand this fix,
> > but you can read through the issue comments from the contributors if you
> > want details.
> >
> > Hadoop pre-commit currently runs off of Yetus master, which includes this
> > patch, and we are no longer seeing the problem. I see HBase pre-commit
> is
> > currently running with Yetus release 0.1.0, which does not have the fix.
> >
> > I think you have 2 options:
> >
> > 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
> > be aware that new Yetus commits will go to master though.)
> > 2. If you don't like the idea of HBase pre-commit running Yetus code that
> > is constantly in motion, then you can wait a few days for sign-off of our
> > 0.2.0 release candidate, which is currently under [VOTE] and looking good
> > so far.
> >
> > --Chris Nauroth
> >
> >
> >
> >
> > On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
> >
> > >HBase precommit tests have been failing on some nodes for a couple of
> days
> > >related to this message:
> > >
> > >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> > >Error while running command to get file permissions :
> > >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> > >libraries: libacl.so.1: failed to map segment from shared object:
> > >Permission denied
> > >
> > >
> > >It is not clear to me whether this is a docker / yetus problem, or an
> > >INFRA
> > >issue. Anybody familiar with this? Obviously, it makes it very hard to
> > >commit a patch without the precommit tests. I've seen this on at least 2
> > >nodes.
> > >
> > >Sample output:
> > >
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> > >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> > >
> > >Enis
> >
> >
>
Re: /bin/ls permission issue on precommit tests
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the information, Chris.
For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
uses current
HEAD of apache/yetus.
Before Yetus 0.2.0 comes out, we can utilize this checkbox.
Cheers
On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hi Enis,
>
> Hadoop pre-commit was experiencing the same problem a few weeks ago:
>
> https://issues.apache.org/jira/browse/YETUS-286
>
>
> We resolved this as a duplicate of a patch that allows control of whether
> or not Docker runs in privileged mode:
>
> https://issues.apache.org/jira/browse/YETUS-285
>
>
> I have to admit I'm not enough of a Docker expert to understand this fix,
> but you can read through the issue comments from the contributors if you
> want details.
>
> Hadoop pre-commit currently runs off of Yetus master, which includes this
> patch, and we are no longer seeing the problem. I see HBase pre-commit is
> currently running with Yetus release 0.1.0, which does not have the fix.
>
> I think you have 2 options:
>
> 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
> be aware that new Yetus commits will go to master though.)
> 2. If you don't like the idea of HBase pre-commit running Yetus code that
> is constantly in motion, then you can wait a few days for sign-off of our
> 0.2.0 release candidate, which is currently under [VOTE] and looking good
> so far.
>
> --Chris Nauroth
>
>
>
>
> On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>
> >HBase precommit tests have been failing on some nodes for a couple of days
> >related to this message:
> >
> >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >Error while running command to get file permissions :
> >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >libraries: libacl.so.1: failed to map segment from shared object:
> >Permission denied
> >
> >
> >It is not clear to me whether this is a docker / yetus problem, or an
> >INFRA
> >issue. Anybody familiar with this? Obviously, it makes it very hard to
> >commit a patch without the precommit tests. I've seen this on at least 2
> >nodes.
> >
> >Sample output:
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >
> >Enis
>
>
Re: /bin/ls permission issue on precommit tests
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the information, Chris.
For HBase precommit, there is checkbox 'USE_YETUS_PRERELEASE' which
uses current
HEAD of apache/yetus.
Before Yetus 0.2.0 comes out, we can utilize this checkbox.
Cheers
On Wed, Mar 2, 2016 at 11:30 AM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hi Enis,
>
> Hadoop pre-commit was experiencing the same problem a few weeks ago:
>
> https://issues.apache.org/jira/browse/YETUS-286
>
>
> We resolved this as a duplicate of a patch that allows control of whether
> or not Docker runs in privileged mode:
>
> https://issues.apache.org/jira/browse/YETUS-285
>
>
> I have to admit I'm not enough of a Docker expert to understand this fix,
> but you can read through the issue comments from the contributors if you
> want details.
>
> Hadoop pre-commit currently runs off of Yetus master, which includes this
> patch, and we are no longer seeing the problem. I see HBase pre-commit is
> currently running with Yetus release 0.1.0, which does not have the fix.
>
> I think you have 2 options:
>
> 1. Switch HBase pre-commit to run Yetus from the master branch. (Please
> be aware that new Yetus commits will go to master though.)
> 2. If you don't like the idea of HBase pre-commit running Yetus code that
> is constantly in motion, then you can wait a few days for sign-off of our
> 0.2.0 release candidate, which is currently under [VOTE] and looking good
> so far.
>
> --Chris Nauroth
>
>
>
>
> On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>
> >HBase precommit tests have been failing on some nodes for a couple of days
> >related to this message:
> >
> >java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> >Error while running command to get file permissions :
> >ExitCodeException exitCode=127: /bin/ls: error while loading shared
> >libraries: libacl.so.1: failed to map segment from shared object:
> >Permission denied
> >
> >
> >It is not clear to me whether this is a docker / yetus problem, or an
> >INFRA
> >issue. Anybody familiar with this? Obviously, it makes it very hard to
> >commit a patch without the precommit tests. I've seen this on at least 2
> >nodes.
> >
> >Sample output:
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
> >ess/patch-unit-hbase-server-jdk1.8.0_72.txt
> >
> >Enis
>
>
Re: /bin/ls permission issue on precommit tests
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Enis,
Hadoop pre-commit was experiencing the same problem a few weeks ago:
https://issues.apache.org/jira/browse/YETUS-286
We resolved this as a duplicate of a patch that allows control of whether
or not Docker runs in privileged mode:
https://issues.apache.org/jira/browse/YETUS-285
I have to admit I'm not enough of a Docker expert to understand this fix,
but you can read through the issue comments from the contributors if you
want details.
Hadoop pre-commit currently runs off of Yetus master, which includes this
patch, and we are no longer seeing the problem. I see HBase pre-commit is
currently running with Yetus release 0.1.0, which does not have the fix.
I think you have 2 options:
1. Switch HBase pre-commit to run Yetus from the master branch. (Please
be aware that new Yetus commits will go to master though.)
2. If you don't like the idea of HBase pre-commit running Yetus code that
is constantly in motion, then you can wait a few days for sign-off of our
0.2.0 release candidate, which is currently under [VOTE] and looking good
so far.
--Chris Nauroth
On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>HBase precommit tests have been failing on some nodes for a couple of days
>related to this message:
>
>java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>Error while running command to get file permissions :
>ExitCodeException exitCode=127: /bin/ls: error while loading shared
>libraries: libacl.so.1: failed to map segment from shared object:
>Permission denied
>
>
>It is not clear to me whether this is a docker / yetus problem, or an
>INFRA
>issue. Anybody familiar with this? Obviously, it makes it very hard to
>commit a patch without the precommit tests. I've seen this on at least 2
>nodes.
>
>Sample output:
>https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>
>Enis
Re: /bin/ls permission issue on precommit tests
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Enis,
Hadoop pre-commit was experiencing the same problem a few weeks ago:
https://issues.apache.org/jira/browse/YETUS-286
We resolved this as a duplicate of a patch that allows control of whether
or not Docker runs in privileged mode:
https://issues.apache.org/jira/browse/YETUS-285
I have to admit I'm not enough of a Docker expert to understand this fix,
but you can read through the issue comments from the contributors if you
want details.
Hadoop pre-commit currently runs off of Yetus master, which includes this
patch, and we are no longer seeing the problem. I see HBase pre-commit is
currently running with Yetus release 0.1.0, which does not have the fix.
I think you have 2 options:
1. Switch HBase pre-commit to run Yetus from the master branch. (Please
be aware that new Yetus commits will go to master though.)
2. If you don't like the idea of HBase pre-commit running Yetus code that
is constantly in motion, then you can wait a few days for sign-off of our
0.2.0 release candidate, which is currently under [VOTE] and looking good
so far.
--Chris Nauroth
On 3/2/16, 10:47 AM, "Enis Söztutar" <en...@apache.org> wrote:
>HBase precommit tests have been failing on some nodes for a couple of days
>related to this message:
>
>java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>Error while running command to get file permissions :
>ExitCodeException exitCode=127: /bin/ls: error while loading shared
>libraries: libacl.so.1: failed to map segment from shared object:
>Permission denied
>
>
>It is not clear to me whether this is a docker / yetus problem, or an
>INFRA
>issue. Anybody familiar with this? Obviously, it makes it very hard to
>commit a patch without the precommit tests. I've seen this on at least 2
>nodes.
>
>Sample output:
>https://builds.apache.org/job/PreCommit-HBASE-Build/789/artifact/patchproc
>ess/patch-unit-hbase-server-jdk1.8.0_72.txt
>
>Enis