You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Keith Turner <ke...@deenlo.com> on 2019/07/11 00:06:09 UTC
Re: [RESULT][VOTE] Apache Accumulo 2.0.0-rc1
I reran the bulk import+agitation test that found #1231 after applying
fixes for it. During that test I found #1260 [1], which I am looking
into fixing. For the latest test I imported 1,000 dirs each with 10M
entries and 48 files. One bulk imports failed because of #1260. I
ran verification and all other bulk imports were ok. The only missing
data was from the single bulk import that failed. I confirmed this
by determining that all missing data had a UUID related to the failed
bulk import.
Hopefully I can figure out #1260 soon. I would like to fix it before
RC2 if possible.
[1] : https://github.com/apache/accumulo/issues/1260
On Tue, Jun 25, 2019 at 3:58 PM Mike Miller <mi...@gmail.com> wrote:
>
> Successfully ran "mvn clean verify -Psunny" on the 2.0 Branch.
>
> I have been running Random walk tests against 2.0 branch and found
> something strange with one of the tests. I opened an issue in the testing
> repo:
> https://github.com/apache/accumulo-testing/issues/83
>
> Hopefully it is just something with the test itself but it has me worried.
> The one Bulk Import test tries to create directories in HDFS before calling
> bulk import on them. The bulk import then fails because the directory
> doesn't exist yet. This happens after I confirm with FS that the file
> exists in HDFS. I will keep digging but it may have something to do with
> the changes from the new Bulk import method and extra validation that was
> added.
>
> On Mon, Jun 24, 2019 at 6:52 PM Christopher <ct...@apache.org> wrote:
>
> > This vote fails with: 4 binding -1s.
> >
> > Thanks everyone who voted and tested (whether your vote was binding or
> > not). I'll do another release candidate after #1231 is fixed, but
> > please feel free to continue testing and identifying more issues that
> > should be fixed for the next release candidate.
> >
> > (Note: There's a U.S. holiday next week, which may impact some
> > people's availability and the timing of release voting. The minimum of
> > 72 hours for any release votes still strictly applies[always], but if
> > the vote isn't closed right away, it's probably because folks are
> > spending time with friends and family.)
> >
> >
> > On Mon, Jun 24, 2019 at 6:38 PM Christopher <ct...@apache.org> wrote:
> > >
> > > -1 because of #1231 also.
> > >
> > > I will investigate
> > https://github.com/apache/accumulo-maven-plugin/issues/4
> > >
> > > On Mon, Jun 24, 2019 at 9:00 AM Owens, Mark <jm...@evoforge.org>
> > wrote:
> > > >
> > > > -1
> > > > Although my local unit and integration tests passed, I agree that
> > #1231 should be fixed before releasing publicly.
> > > >
> > > > -----Original Message-----
> > > > From: Mike Miller <mi...@gmail.com>
> > > > Sent: Monday, June 24, 2019 8:47 AM
> > > > To: dev@accumulo.apache.org
> > > > Subject: Re: [VOTE] Apache Accumulo 2.0.0-rc1
> > > >
> > > > -1
> > > > So we can fix #1231
> > > > I was able to do some Random Walk testing using Uno on my laptop last
> > week on the 2.0 branch. It was mostly successful.
> > > > I found this RW bug I plan on fixing this week:
> > > > https://github.com/apache/accumulo-testing/issues/83
> > > > I also plan on taking a look at #1231.
> > > >
> > > > On Sat, Jun 22, 2019 at 10:09 AM Jeffrey Manno <
> > jeffreymanno15@gmail.com>
> > > > wrote:
> > > >
> > > > > Just giving an update on where my testing is at in case I forget.
> > > > >
> > > > > So far haven't noticed any errors. I have been running continuous
> > > > > ingest on both a 10 tserver EC2 cluster and on my machine for roughly
> > > > > ~17 hours. I do get the tracing spans being dropped warning on the
> > Ec2
> > > > > side. Not sure of that cause of that as of yet.
> > > > >
> > > > > All the usual mvn build profiles have passed without error.
> > > > >
> > > > >
> > > > > On Fri, Jun 21, 2019 at 4:19 PM Keith Turner <ke...@deenlo.com>
> > wrote:
> > > > >
> > > > > > -1 mainly because of #1231. Also would really like to be able to
> > > > > > validate external projects can build against RC, but was unable to.
> > > > > >
> > > > > > I have been doing a lot testing this past week on EC2 using
> > > > > > 2.0.0-SNAPSHOT built from 2ac7c7e.
> > > > > >
> > > > > > I successfully ran continuous ingest with agitation and multiple
> > > > > > ingest patterns for 18 hrs on a 11 node cluster. I used the
> > > > > > following instructions
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > https://github.com/apache/accumulo-testing/blob/master/docs/ingest-tes
> > > > > t.md
> > > > > >
> > > > > > Then I ran the new bulk version of continuous ingest. I generated
> > a
> > > > > > lot of files to bulk import, started agitation, and then started
> > > > > > importing. I was using the following instructions
> > > > > >
> > > > > >
> > > > >
> > https://github.com/apache/accumulo-testing/blob/master/docs/bulk-test.
> > > > > md
> > > > > >
> > > > > > While running this I found the following bug.
> > > > > >
> > > > > > https://github.com/apache/accumulo/issues/1231
> > > > > >
> > > > > > I ran the new garbage collection simulation on a 6 node cluster
> > with
> > > > > > 15 generators and 1 collector for ~18hrs. I ran agitation for this
> > > > > > test. I need to document how I ran this using docker swarm.
> > > > > >
> > > > > >
> > https://github.com/apache/accumulo-testing/blob/master/docs/gcs.md
> > > > > >
> > > > > > I tried building Fluo against this RC, but was unable to because of
> > > > > > the following bug.
> > > > > >
> > > > > > https://github.com/apache/accumulo-maven-plugin/issues/4
> > > > > >
> > > > > > I found the following minor issues while testing.
> > > > > >
> > > > > > https://github.com/apache/accumulo-testing/issues/84
> > > > > > https://github.com/apache/accumulo/issues/1230
> > > > > > https://github.com/apache/accumulo/issues/1223
> > > > > >
> > > > > > Thanks for putting this RC together Christopher.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jun 20, 2019 at 11:42 PM Christopher <ct...@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > > Accumulo Developers,
> > > > > > >
> > > > > > > Well, this is exciting, and a long time coming!
> > > > > > > Please consider the following candidate for Apache Accumulo
> > 2.0.0.
> > > > > > >
> > > > > > > Git Commit:
> > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98
> > > > > > > Branch:
> > > > > > > 2.0.0-rc1
> > > > > > >
> > > > > > > If this vote passes, a gpg-signed tag will be created using:
> > > > > > > git tag -f -m 'Apache Accumulo 2.0.0' -s rel/2.0.0 \
> > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98
> > > > > > >
> > > > > > > Staging repo:
> > > > > >
> > > > >
> > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > 080
> > > > > > > Source (official release artifact):
> > > > > > >
> > > > > >
> > > > >
> > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-src.tar.gz
> > > > > > > Binary:
> > > > > >
> > > > >
> > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-bin.tar.gz
> > > > > > >
> > > > > > > Append ".asc" to download the cryptographic signature for a given
> > > > > > artifact.
> > > > > > > (You can also append ".sha1" or ".md5" instead in order to verify
> > > > > > > the
> > > > > > checksums
> > > > > > > generated by Maven to verify the integrity of the Nexus
> > repository
> > > > > > > staging area.)
> > > > > > >
> > > > > > > Signing keys are available at
> > > > > https://www.apache.org/dist/accumulo/KEYS
> > > > > > > (Expected fingerprint: 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D)
> > > > > > >
> > > > > > > In addition to the tarballs and their signatures, the following
> > > > > checksum
> > > > > > > files will be added to the dist/release SVN area after release:
> > > > > > > accumulo-2.0.0-src.tar.gz.sha512 will contain:
> > > > > > > SHA512 (accumulo-2.0.0-src.tar.gz) =
> > > > > > >
> > > > > >
> > > > >
> > 92b7a0910b2fd9c28f47af5ce66b52b515c5affb16ada0da7f20499fd552f647e0841e
> > > > > 2c61e67b609016bb3702888e42646176742082cc9ead33f782a83d021d
> > > > > > > accumulo-2.0.0-bin.tar.gz.sha512 will contain:
> > > > > > > SHA512 (accumulo-2.0.0-bin.tar.gz) =
> > > > > > >
> > > > > >
> > > > >
> > f85e60c38d883045bd797e75288900408d85aee16ae53eabc76d0bb3f0daea6f51b9e3
> > > > > 5396c7a6c8ff5e19a4bcd668793f08772b42e6cf41e29b3d70476b95c1
> > > > > > >
> > > > > > > Release notes (in progress) can be found at:
> > > > > > > https://accumulo.apache.org/release/accumulo-2.0.0/
> > > > > > >
> > > > > > > Release testing instructions:
> > > > > > > https://accumulo.apache.org/contributor/verifying-release
> > > > > > >
> > > > > > > Please vote one of:
> > > > > > > [ ] +1 - I have verified and accept...
> > > > > > > [ ] +0 - I have reservations, but not strong enough to vote
> > against...
> > > > > > > [ ] -1 - Because..., I do not accept...
> > > > > > > ... these artifacts as the 2.0.0 release of Apache Accumulo.
> > > > > > >
> > > > > > > This vote will remain open until at least Mon 24 Jun 2019
> > 04:00:00
> > > > > > > AM
> > > > > > UTC.
> > > > > > > (Mon 24 Jun 2019 12:00:00 AM EDT / Sun 23 Jun 2019 09:00:00 PM
> > > > > > > PDT) Voting can continue after this deadline until the release
> > > > > > > manager sends an email ending the vote.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > P.S. Hint: download the whole staging repo with
> > > > > > > wget -erobots=off -r -l inf -np -nH \
> > > > > > >
> > > > > >
> > > > >
> > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > 080/
> > > > > > > # note the trailing slash is needed
> > > > > >
> > > > >
> >
Re: [RESULT][VOTE] Apache Accumulo 2.0.0-rc1
Posted by Christopher <ct...@apache.org>.
I can wait a few days for RC2 so you can address that bug.
On Wed, Jul 10, 2019 at 8:06 PM Keith Turner <ke...@deenlo.com> wrote:
>
> I reran the bulk import+agitation test that found #1231 after applying
> fixes for it. During that test I found #1260 [1], which I am looking
> into fixing. For the latest test I imported 1,000 dirs each with 10M
> entries and 48 files. One bulk imports failed because of #1260. I
> ran verification and all other bulk imports were ok. The only missing
> data was from the single bulk import that failed. I confirmed this
> by determining that all missing data had a UUID related to the failed
> bulk import.
>
> Hopefully I can figure out #1260 soon. I would like to fix it before
> RC2 if possible.
>
> [1] : https://github.com/apache/accumulo/issues/1260
>
> On Tue, Jun 25, 2019 at 3:58 PM Mike Miller <mi...@gmail.com> wrote:
> >
> > Successfully ran "mvn clean verify -Psunny" on the 2.0 Branch.
> >
> > I have been running Random walk tests against 2.0 branch and found
> > something strange with one of the tests. I opened an issue in the testing
> > repo:
> > https://github.com/apache/accumulo-testing/issues/83
> >
> > Hopefully it is just something with the test itself but it has me worried.
> > The one Bulk Import test tries to create directories in HDFS before calling
> > bulk import on them. The bulk import then fails because the directory
> > doesn't exist yet. This happens after I confirm with FS that the file
> > exists in HDFS. I will keep digging but it may have something to do with
> > the changes from the new Bulk import method and extra validation that was
> > added.
> >
> > On Mon, Jun 24, 2019 at 6:52 PM Christopher <ct...@apache.org> wrote:
> >
> > > This vote fails with: 4 binding -1s.
> > >
> > > Thanks everyone who voted and tested (whether your vote was binding or
> > > not). I'll do another release candidate after #1231 is fixed, but
> > > please feel free to continue testing and identifying more issues that
> > > should be fixed for the next release candidate.
> > >
> > > (Note: There's a U.S. holiday next week, which may impact some
> > > people's availability and the timing of release voting. The minimum of
> > > 72 hours for any release votes still strictly applies[always], but if
> > > the vote isn't closed right away, it's probably because folks are
> > > spending time with friends and family.)
> > >
> > >
> > > On Mon, Jun 24, 2019 at 6:38 PM Christopher <ct...@apache.org> wrote:
> > > >
> > > > -1 because of #1231 also.
> > > >
> > > > I will investigate
> > > https://github.com/apache/accumulo-maven-plugin/issues/4
> > > >
> > > > On Mon, Jun 24, 2019 at 9:00 AM Owens, Mark <jm...@evoforge.org>
> > > wrote:
> > > > >
> > > > > -1
> > > > > Although my local unit and integration tests passed, I agree that
> > > #1231 should be fixed before releasing publicly.
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Miller <mi...@gmail.com>
> > > > > Sent: Monday, June 24, 2019 8:47 AM
> > > > > To: dev@accumulo.apache.org
> > > > > Subject: Re: [VOTE] Apache Accumulo 2.0.0-rc1
> > > > >
> > > > > -1
> > > > > So we can fix #1231
> > > > > I was able to do some Random Walk testing using Uno on my laptop last
> > > week on the 2.0 branch. It was mostly successful.
> > > > > I found this RW bug I plan on fixing this week:
> > > > > https://github.com/apache/accumulo-testing/issues/83
> > > > > I also plan on taking a look at #1231.
> > > > >
> > > > > On Sat, Jun 22, 2019 at 10:09 AM Jeffrey Manno <
> > > jeffreymanno15@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Just giving an update on where my testing is at in case I forget.
> > > > > >
> > > > > > So far haven't noticed any errors. I have been running continuous
> > > > > > ingest on both a 10 tserver EC2 cluster and on my machine for roughly
> > > > > > ~17 hours. I do get the tracing spans being dropped warning on the
> > > Ec2
> > > > > > side. Not sure of that cause of that as of yet.
> > > > > >
> > > > > > All the usual mvn build profiles have passed without error.
> > > > > >
> > > > > >
> > > > > > On Fri, Jun 21, 2019 at 4:19 PM Keith Turner <ke...@deenlo.com>
> > > wrote:
> > > > > >
> > > > > > > -1 mainly because of #1231. Also would really like to be able to
> > > > > > > validate external projects can build against RC, but was unable to.
> > > > > > >
> > > > > > > I have been doing a lot testing this past week on EC2 using
> > > > > > > 2.0.0-SNAPSHOT built from 2ac7c7e.
> > > > > > >
> > > > > > > I successfully ran continuous ingest with agitation and multiple
> > > > > > > ingest patterns for 18 hrs on a 11 node cluster. I used the
> > > > > > > following instructions
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > https://github.com/apache/accumulo-testing/blob/master/docs/ingest-tes
> > > > > > t.md
> > > > > > >
> > > > > > > Then I ran the new bulk version of continuous ingest. I generated
> > > a
> > > > > > > lot of files to bulk import, started agitation, and then started
> > > > > > > importing. I was using the following instructions
> > > > > > >
> > > > > > >
> > > > > >
> > > https://github.com/apache/accumulo-testing/blob/master/docs/bulk-test.
> > > > > > md
> > > > > > >
> > > > > > > While running this I found the following bug.
> > > > > > >
> > > > > > > https://github.com/apache/accumulo/issues/1231
> > > > > > >
> > > > > > > I ran the new garbage collection simulation on a 6 node cluster
> > > with
> > > > > > > 15 generators and 1 collector for ~18hrs. I ran agitation for this
> > > > > > > test. I need to document how I ran this using docker swarm.
> > > > > > >
> > > > > > >
> > > https://github.com/apache/accumulo-testing/blob/master/docs/gcs.md
> > > > > > >
> > > > > > > I tried building Fluo against this RC, but was unable to because of
> > > > > > > the following bug.
> > > > > > >
> > > > > > > https://github.com/apache/accumulo-maven-plugin/issues/4
> > > > > > >
> > > > > > > I found the following minor issues while testing.
> > > > > > >
> > > > > > > https://github.com/apache/accumulo-testing/issues/84
> > > > > > > https://github.com/apache/accumulo/issues/1230
> > > > > > > https://github.com/apache/accumulo/issues/1223
> > > > > > >
> > > > > > > Thanks for putting this RC together Christopher.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jun 20, 2019 at 11:42 PM Christopher <ct...@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Accumulo Developers,
> > > > > > > >
> > > > > > > > Well, this is exciting, and a long time coming!
> > > > > > > > Please consider the following candidate for Apache Accumulo
> > > 2.0.0.
> > > > > > > >
> > > > > > > > Git Commit:
> > > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98
> > > > > > > > Branch:
> > > > > > > > 2.0.0-rc1
> > > > > > > >
> > > > > > > > If this vote passes, a gpg-signed tag will be created using:
> > > > > > > > git tag -f -m 'Apache Accumulo 2.0.0' -s rel/2.0.0 \
> > > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98
> > > > > > > >
> > > > > > > > Staging repo:
> > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > > 080
> > > > > > > > Source (official release artifact):
> > > > > > > >
> > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-src.tar.gz
> > > > > > > > Binary:
> > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-bin.tar.gz
> > > > > > > >
> > > > > > > > Append ".asc" to download the cryptographic signature for a given
> > > > > > > artifact.
> > > > > > > > (You can also append ".sha1" or ".md5" instead in order to verify
> > > > > > > > the
> > > > > > > checksums
> > > > > > > > generated by Maven to verify the integrity of the Nexus
> > > repository
> > > > > > > > staging area.)
> > > > > > > >
> > > > > > > > Signing keys are available at
> > > > > > https://www.apache.org/dist/accumulo/KEYS
> > > > > > > > (Expected fingerprint: 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D)
> > > > > > > >
> > > > > > > > In addition to the tarballs and their signatures, the following
> > > > > > checksum
> > > > > > > > files will be added to the dist/release SVN area after release:
> > > > > > > > accumulo-2.0.0-src.tar.gz.sha512 will contain:
> > > > > > > > SHA512 (accumulo-2.0.0-src.tar.gz) =
> > > > > > > >
> > > > > > >
> > > > > >
> > > 92b7a0910b2fd9c28f47af5ce66b52b515c5affb16ada0da7f20499fd552f647e0841e
> > > > > > 2c61e67b609016bb3702888e42646176742082cc9ead33f782a83d021d
> > > > > > > > accumulo-2.0.0-bin.tar.gz.sha512 will contain:
> > > > > > > > SHA512 (accumulo-2.0.0-bin.tar.gz) =
> > > > > > > >
> > > > > > >
> > > > > >
> > > f85e60c38d883045bd797e75288900408d85aee16ae53eabc76d0bb3f0daea6f51b9e3
> > > > > > 5396c7a6c8ff5e19a4bcd668793f08772b42e6cf41e29b3d70476b95c1
> > > > > > > >
> > > > > > > > Release notes (in progress) can be found at:
> > > > > > > > https://accumulo.apache.org/release/accumulo-2.0.0/
> > > > > > > >
> > > > > > > > Release testing instructions:
> > > > > > > > https://accumulo.apache.org/contributor/verifying-release
> > > > > > > >
> > > > > > > > Please vote one of:
> > > > > > > > [ ] +1 - I have verified and accept...
> > > > > > > > [ ] +0 - I have reservations, but not strong enough to vote
> > > against...
> > > > > > > > [ ] -1 - Because..., I do not accept...
> > > > > > > > ... these artifacts as the 2.0.0 release of Apache Accumulo.
> > > > > > > >
> > > > > > > > This vote will remain open until at least Mon 24 Jun 2019
> > > 04:00:00
> > > > > > > > AM
> > > > > > > UTC.
> > > > > > > > (Mon 24 Jun 2019 12:00:00 AM EDT / Sun 23 Jun 2019 09:00:00 PM
> > > > > > > > PDT) Voting can continue after this deadline until the release
> > > > > > > > manager sends an email ending the vote.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > P.S. Hint: download the whole staging repo with
> > > > > > > > wget -erobots=off -r -l inf -np -nH \
> > > > > > > >
> > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1
> > > > > > 080/
> > > > > > > > # note the trailing slash is needed
> > > > > > >
> > > > > >
> > >