You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Konstantin Boudnik <co...@apache.org> on 2015/09/01 21:44:36 UTC
Re: CentOS Package Checksums Don't Match RPMs
I think this is the consequences of me fighting with the package signing... ;(
A couple of days ago I have re-ran 'createrepo' for all the RPM-based distros
and uploaded new repo files to the release. Not sure why the checksums differ
now...
I will take a look into this again tonight.
Cos
On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> I can second it:
>
> I added to /etc/yum.repo.d/meins.repo
>
> [meins]
> name=Bigtop epo
> baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> enabled=1
> gpgcheck=0
> priority=1
>
> and got
> ............
> Downloading packages:
> hbase-0.98.12-1.el7.centos.noa FAILED =============================================-] 849 kB/s | 62 MB 00:00:00 ETA
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=meins clean metadata
> Trying other mirror.
> .............
>
> Olaf
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Indeee we do! Thanks for tireless testing and catching these mistakes! ;)
On September 8, 2015 5:35:07 AM PDT, Evans Ye <ev...@apache.org> wrote:
>Hi Cos,
>
>I randomly install fedora packages from the S3 repo and it's all good.
>Thank you so much for your effort and the quick fix.
>Finally. we now have our bigtop repos all set!
>
>Evans
>
>2015-09-08 13:36 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
>> Just to make sure this doesn't fall through - I have flushed out and
>> synced F20 repo.
>>
>> Everything should be in order now. Thansk!
>> Cos
>>
>> On Mon, Sep 07, 2015 at 03:59PM, Konstantin Boudnik wrote:
>> > On Tue, Sep 08, 2015 at 02:21AM, Evans Ye wrote:
>> > > I ran deployment test on those 3 yum repos.
>> > > centos6, 7 are good, however fedora is still failing.
>> > > It looks like the exactly same issue is still in Fedora repo
>which is
>> some
>> > > of the rpm are still the old one signed by old key.
>> > >
>> > > # old package
>> > > $ rpm --checksig hadoop-2.6.0-1.fc20.x86_64.rpm
>> > > hadoop-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
>> (MISSING
>> > > KEYS: (MD5) PGP#d0c3824f)
>> > >
>> > > # new package
>> > > $ rpm --checksig bigtop-utils-1.0.0-1.fc20.noarch.rpm
>> > > bigtop-utils-1.0.0-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
>> > >
>> > > Cos could you please wipe up and resync Fedora repo on S3?
>Thanks!
>> >
>> > Thanks for validating and being so patient - I will do this by the
>end
>> of the day today.
>> >
>> > Cos
>> >
>> > >
>> > > Evans
>> > >
>> > > 2015-09-06 9:39 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>> > >
>> > > > And it is done now - fresh copy of all centos packages signed
>by the
>> > > > correct
>> > > > key (as validated locally). Thanks for keep finding those,
>Evans!
>> > > >
>> > > > Cos
>> > > >
>> > > > On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
>> > > > > It is pretty nuts because the packages I have locally are all
>> signed
>> > > > with the
>> > > > > correct key. But when I download the package in question I
>see
>> that the
>> > > > local
>> > > > > one is different from the downloaded. And the latter is
>signed
>> with the
>> > > > wrong
>> > > > > key indeed.
>> > > > >
>> > > > > Considering that I had synced everything after re-signing,
>> there're only
>> > > > two
>> > > > > possibilities that I see:
>> > > > > - S3 eventual consistency bites us in the rear. Which might
>be
>> possible
>> > > > for
>> > > > > in the short run, but I don't see how it couldn't be
>updated
>> after
>> > > > all that
>> > > > > time
>> > > > > - s3cmd has screwed up and didn't updated some of the
>packages. I
>> am
>> > > > going to
>> > > > > wipe out _all_ rpm distros and resync it right away. This
>might
>> cause
>> > > > a
>> > > > > short interruption in the packages availability, but at
>least
>> we'll
>> > > > have
>> > > > > all correct stuff up there.
>> > > > >
>> > > > > Should be done in about 30 minutes. Stay tuned
>> > > > > Cos
>> > > > >
>> > > > > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
>> > > > > > Sorry guys. I'm back with the issue again. ;)
>> > > > > >
>> > > > > > Turns out that some of the rpms are good, some are not.
>Look at
>> my
>> > > > tests
>> > > > > > below:
>> > > > > >
>> > > > > >
>> > > > > > ### Centos 6 repo ###
>> > > > > >
>> > > > > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
>> > > > > >
>> > > > > > $ wget
>> > > > > >
>> > > >
>>
>https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
>> > > > > > -O /etc/yum.repos.d/bigtop.repo
>> > > > > >
>> > > > > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc
>> bigtop-tomcat
>> > > > > > zookeeper # Successfully installed
>> > > > > >
>> > > > > > $ yum -y install hadoop hadoop-hdfs
>> > > > > >
>> > > > > > ...
>> > > > > >
>> > > > > > Error Downloading Packages:
>> > > > > >
>> > > > > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
>> > > > > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from
>bigtop:
>> [Errno
>> > > > 256]
>> > > > > > No more mirrors to try.
>> > > > > >
>> > > > > > hadoop-2.6.0-1.el6.x86_64: failure:
>> > > > > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop:
>[Errno
>> 256] No
>> > > > > > more mirrors to try.
>> > > > > >
>> > > > > >
>> > > > > > I find the same set of packages(groovy, utils, jsvc,
>tomcat,
>> > > > zookeeper) can
>> > > > > > be successfully installed across centos 6, 7 and fedora
>repos
>> and the
>> > > > other
>> > > > > > same set of packages failed to install across the
>platforms.
>> > > > Therefore, I
>> > > > > > think there might be an issue happening during some sort of
>> automation
>> > > > > > steps.
>> > > > > >
>> > > > > > In addition, I suspect that those packages failed to
>install are
>> still
>> > > > > > signed by old key, hence the subkey issue found by Cos
>blocks the
>> > > > packages
>> > > > > > to be installed.
>> > > > > >
>> > > > > >
>> > > > > > [root@34696969ce7d /]# rpm --checksig
>> > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
>> > > > > >
>> > > > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP)
>md5
>> NOT OK
>> > > > > > (MISSING KEYS: (MD5) PGP#d0c3824f)
>> > > > > >
>> > > > > > [root@34696969ce7d /]# rpm --checksig
>> > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm
>> > > > > >
>> > > > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp
>md5 OK
>> > > > > >
>> > > > > >
>> > > > > > Cos can you first check that the hadoop* packages has been
>> successfully
>> > > > > > resigned by your newly generated code signing key? Thanks!
>> > > > > >
>> > > > > >
>> > > > > > Evans
>> > > > > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
>> > > > > >
>> > > > > > > Appreciate the sentiment guys and thanks for kind words!
>> > > > > > > The irony here is that I don't even like this type of
>> packaging and
>> > > > not
>> > > > > > > using
>> > > > > > > it if I can help it ;) Oh well...
>> > > > > > >
>> > > > > > > To close this thread - I will try to put together a blog
>about
>> 1.0
>> > > > later
>> > > > > > > today. Thanks everyone for the testing, patience, and -
>kudos
>> to
>> > > > Evans -
>> > > > > > > detailed instructions on how to reproduce the issue!
>> > > > > > >
>> > > > > > > Cos
>> > > > > > >
>> > > > > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
>> > > > > > > > Yes thanks cos for getting this centos stuff figured
>out.!
>> > > > > > > >
>> > > > > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <
>> apurtell@apache.org
>> > > > >
>> > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > Thanks for sticking with it Cos. That's an annoying
>bug.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <
>> > > > cos@apache.org>
>> > > > > > > wrote:
>> > > > > > > > >>
>> > > > > > > > >> Ok, as I suspected there's a long standing (at least
>from
>> 2006)
>> > > > bug
>> > > > > > > in RPM
>> > > > > > > > >> that doesn't allow to validate RPM signature if a
>subkey
>> has
>> > > > been
>> > > > > > > used for
>> > > > > > > > >> signing.
>> > > > > > > > >>
>> > > > > > > > >> I ended up generating a new key pair (just for this
>> purpose) and
>> > > > > > > resigning
>> > > > > > > > >> all
>> > > > > > > > >> binaries with it; then resyncing everything with s3.
>I
>> also have
>> > > > > > > updated
>> > > > > > > > >> KEYS
>> > > > > > > > >> file with the new one. I have quickly ran a test on
>> centos7 by
>> > > > > > > installing
>> > > > > > > > >> bigtop-utils on an empty container and everything
>worked,
>> > > > including
>> > > > > > > > >> automatic
>> > > > > > > > >> import of the keys and the validation/installation
>of the
>> > > > package.
>> > > > > > > Looks
>> > > > > > > > >> like
>> > > > > > > > >> we are in the clear.
>> > > > > > > > >>
>> > > > > > > > >> Please shout if you see otherwise. Thanks everyone
>for
>> your
>> > > > patience!
>> > > > > > > > >> Cos
>> > > > > > > > >>
>> > > > > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik
>> wrote:
>> > > > > > > > >>> I think there's a difference between how you've
>signed
>> the
>> > > > pkgs and
>> > > > > > > how
>> > > > > > > > >> I did
>> > > > > > > > >>> it. I signed with sub-key (as I mentioned before)
>and yum
>> > > > doesn't
>> > > > > > > > >> recognize
>> > > > > > > > >>> it. Seemingly, it expects that the master key was
>used
>> for
>> > > > signing.
>> > > > > > > > >>>
>> > > > > > > > >>> Also, in your repo file below
>> > > > > > > > >>>
>gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>> > > > > > > > >>> points to the old keys. The location should be
>> > > > > > > > >>> gpgkey=
>> > > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
>> > > > > > > > >>>
>> > > > > > > > >>> I am pretty sure I have exported my key with
>--armor
>> option
>> > > > back in
>> > > > > > > the
>> > > > > > > > >> day.
>> > > > > > > > >>> But I will repeat it and see if I can fix the
>situation,
>> which
>> > > > I also
>> > > > > > > > >> observer
>> > > > > > > > >>> following your steps. If that's the only issue I
>will
>> update
>> > > > the KEYS
>> > > > > > > > >> and we
>> > > > > > > > >>> should be completed by tonight ;)
>> > > > > > > > >>>
>> > > > > > > > >>> Thanks for your help!
>> > > > > > > > >>> Cos
>> > > > > > > > >>>
>> > > > > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
>> > > > > > > > >>>> This is the same issue we're trying to solve in
>the
>> mailing
>> > > > thread
>> > > > > > > > >>>> "convenience artifacts are signed and uploaded".
>I've
>> built a
>> > > > sample
>> > > > > > > > >> repo
>> > > > > > > > >>>> which works properly by using my own key "Evans
>Ye" to
>> sign
>> > > > and to
>> > > > > > > > >> export
>> > > > > > > > >>>> GPG KEY. So I believe the following steps should
>be the
>> right
>> > > > way to
>> > > > > > > > >> sign
>> > > > > > > > >>>> packages and export the gpgkey:
>> > > > > > > > >>>>
>> > > > > > > > >>>> $ find -name *.rpm | xargs rpm
>--define="%_gpg_name
>> Evans Ye"
>> > > > > > > --addsign
>> > > > > > > > >>>>
>> > > > > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
>> > > > > > > > >>>> I've verified that the hash is matched now in our
>> official
>> > > > repo.
>> > > > > > > > >>>> So I guess the main issue left is using
>non-armored gpg
>> key,
>> > > > if we
>> > > > > > > > >> manually
>> > > > > > > > >>>> import the gpgkey in the repo file:
>> > > > > > > > >>>>
>> > > > > > > > >>>> [bigtop]
>> > > > > > > > >>>> name=Bigtop
>> > > > > > > > >>>> enabled=1
>> > > > > > > > >>>> gpgcheck=1
>> > > > > > > > >>>> type=NONE
>> > > > > > > > >>>> baseurl=
>> > > > > > >
>http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
>> > > > > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>> > > > > > > > >>>>
>> > > > > > > > >>>> [root@48723d98dc1b ~]# rpm --import
>> > > > > > > > >>>>
>https://dist.apache.org/repos/dist/release/bigtop/KEYS
>> > > > > > > > >>>> error:
>> https://dist.apache.org/repos/dist/release/bigtop/KEYS:
>> > > > key
>> > > > > > > 2
>> > > > > > > > >> not an
>> > > > > > > > >>>> armored public key.
>> > > > > > > > >>>>
>> > > > > > > > >>>> It gets error.
>> > > > > > > > >>>> However, my own exported armored key can be
>imported
>> without
>> > > > an
>> > > > > > > error.
>> > > > > > > > >>>> That's the different.
>> > > > > > > > >>>>
>> > > > > > > > >>>> Can you confirm that the gpgkey(
>> > > > > > > > >> http://archive.apache.org/dist/bigtop/KEYS)
>> > > > > > > > >>>> is exported with --armor flag?
>> > > > > > > > >>>>
>> > > > > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <
>> cos@apache.org
>> > > > >:
>> > > > > > > > >>>>
>> > > > > > > > >>>>> Looks like I have figured out what's wrong with
>my
>> key. And
>> > > > it is
>> > > > > > > > >>>>> _nothing_.
>> > > > > > > > >>>>> However, it seems that I can not sign RPMs with
>subkey
>> as
>> > > > YUM can
>> > > > > > > > >> not find
>> > > > > > > > >>>>> the
>> > > > > > > > >>>>> key while importing. Can anyone confirm or
>disprove my
>> train
>> > > > of
>> > > > > > > > >> thoughts?
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> Thanks!
>> > > > > > > > >>>>> Cos
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin
>Boudnik
>> wrote:
>> > > > > > > > >>>>>> I've resynced the repodata once again and I
>don't see
>> this
>> > > > issue
>> > > > > > > > >> on the
>> > > > > > > > >>>>>> centos7 anymore. However, yum still complains
>about
>> the key
>> > > > being
>> > > > > > > > >> no
>> > > > > > > > >>>>>> available, but there's a workaround by setting
>> gpgcheck=0
>> > > > And I am
>> > > > > > > > >> going
>> > > > > > > > >>>>> to
>> > > > > > > > >>>>>> figure out what to do with it and why my key
>isn't
>> working
>> > > > as
>> > > > > > > > >> expected.
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> I also have discovered that the gpgkey file URL
>is
>> using
>> > > > the old
>> > > > > > > > >>>>> incubation
>> > > > > > > > >>>>>> KEYS. Fixed that as well.
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> Please let me know if you still see the issue
>with
>> checksums
>> > > > > > > > >> mismatch.
>> > > > > > > > >>>>>> Thanks,
>> > > > > > > > >>>>>> Cos
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin
>Boudnik
>> wrote:
>> > > > > > > > >>>>>>> I think this is the consequences of me fighting
>with
>> the
>> > > > package
>> > > > > > > > >>>>> signing... ;(
>> > > > > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo'
>for
>> all the
>> > > > > > > > >> RPM-based
>> > > > > > > > >>>>> distros
>> > > > > > > > >>>>>>> and uploaded new repo files to the release. Not
>sure
>> why
>> > > > the
>> > > > > > > > >> checksums
>> > > > > > > > >>>>> differ
>> > > > > > > > >>>>>>> now...
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> I will take a look into this again tonight.
>> > > > > > > > >>>>>>> Cos
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe
>wrote:
>> > > > > > > > >>>>>>>> I can second it:
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> [meins]
>> > > > > > > > >>>>>>>> name=Bigtop epo
>> > > > > > > > >>>>>>>> baseurl=
>> > > > > > > > >>>>>
>> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
>> > > > > > > > >>>>>>>> enabled=1
>> > > > > > > > >>>>>>>> gpgcheck=0
>> > > > > > > > >>>>>>>> priority=1
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> and got
>> > > > > > > > >>>>>>>> ............
>> > > > > > > > >>>>>>>> Downloading packages:
>> > > > > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
>> > > > > > > > >>>>>
>> =============================================-] 849
>> > > > kB/s
>> > > > > > > > >> | 62
>> > > > > > > > >>>>> MB 00:00:00 ETA
>> > > > > > > > >>
>> > > > > > >
>> > > >
>>
>http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
>> > > > > > > > >> :
>> > > > > > > > >>>>> [Errno -1] Package does not match intended
>download.
>> > > > Suggestion:
>> > > > > > > run
>> > > > > > > > >> yum
>> > > > > > > > >>>>> --enablerepo=meins clean metadata
>> > > > > > > > >>>>>>>> Trying other mirror.
>> > > > > > > > >>>>>>>> .............
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> Olaf
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Best regards,
>> > > > > > > > >
>> > > > > > > > > - Andy
>> > > > > > > > >
>> > > > > > > > > Problems worthy of attack prove their worth by
>hitting
>> back. -
>> > > > Piet
>> > > > > > > Hein
>> > > > > > > > > (via Tom White)
>> > > > > > >
>> > > >
>> > > >
>> > > >
>>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Evans Ye <ev...@apache.org>.
Hi Cos,
I randomly install fedora packages from the S3 repo and it's all good.
Thank you so much for your effort and the quick fix.
Finally. we now have our bigtop repos all set!
Evans
2015-09-08 13:36 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> Just to make sure this doesn't fall through - I have flushed out and
> synced F20 repo.
>
> Everything should be in order now. Thansk!
> Cos
>
> On Mon, Sep 07, 2015 at 03:59PM, Konstantin Boudnik wrote:
> > On Tue, Sep 08, 2015 at 02:21AM, Evans Ye wrote:
> > > I ran deployment test on those 3 yum repos.
> > > centos6, 7 are good, however fedora is still failing.
> > > It looks like the exactly same issue is still in Fedora repo which is
> some
> > > of the rpm are still the old one signed by old key.
> > >
> > > # old package
> > > $ rpm --checksig hadoop-2.6.0-1.fc20.x86_64.rpm
> > > hadoop-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> (MISSING
> > > KEYS: (MD5) PGP#d0c3824f)
> > >
> > > # new package
> > > $ rpm --checksig bigtop-utils-1.0.0-1.fc20.noarch.rpm
> > > bigtop-utils-1.0.0-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> > >
> > > Cos could you please wipe up and resync Fedora repo on S3? Thanks!
> >
> > Thanks for validating and being so patient - I will do this by the end
> of the day today.
> >
> > Cos
> >
> > >
> > > Evans
> > >
> > > 2015-09-06 9:39 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > >
> > > > And it is done now - fresh copy of all centos packages signed by the
> > > > correct
> > > > key (as validated locally). Thanks for keep finding those, Evans!
> > > >
> > > > Cos
> > > >
> > > > On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
> > > > > It is pretty nuts because the packages I have locally are all
> signed
> > > > with the
> > > > > correct key. But when I download the package in question I see
> that the
> > > > local
> > > > > one is different from the downloaded. And the latter is signed
> with the
> > > > wrong
> > > > > key indeed.
> > > > >
> > > > > Considering that I had synced everything after re-signing,
> there're only
> > > > two
> > > > > possibilities that I see:
> > > > > - S3 eventual consistency bites us in the rear. Which might be
> possible
> > > > for
> > > > > in the short run, but I don't see how it couldn't be updated
> after
> > > > all that
> > > > > time
> > > > > - s3cmd has screwed up and didn't updated some of the packages. I
> am
> > > > going to
> > > > > wipe out _all_ rpm distros and resync it right away. This might
> cause
> > > > a
> > > > > short interruption in the packages availability, but at least
> we'll
> > > > have
> > > > > all correct stuff up there.
> > > > >
> > > > > Should be done in about 30 minutes. Stay tuned
> > > > > Cos
> > > > >
> > > > > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> > > > > > Sorry guys. I'm back with the issue again. ;)
> > > > > >
> > > > > > Turns out that some of the rpms are good, some are not. Look at
> my
> > > > tests
> > > > > > below:
> > > > > >
> > > > > >
> > > > > > ### Centos 6 repo ###
> > > > > >
> > > > > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
> > > > > >
> > > > > > $ wget
> > > > > >
> > > >
> https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> > > > > > -O /etc/yum.repos.d/bigtop.repo
> > > > > >
> > > > > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc
> bigtop-tomcat
> > > > > > zookeeper # Successfully installed
> > > > > >
> > > > > > $ yum -y install hadoop hadoop-hdfs
> > > > > >
> > > > > > ...
> > > > > >
> > > > > > Error Downloading Packages:
> > > > > >
> > > > > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> > > > > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop:
> [Errno
> > > > 256]
> > > > > > No more mirrors to try.
> > > > > >
> > > > > > hadoop-2.6.0-1.el6.x86_64: failure:
> > > > > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno
> 256] No
> > > > > > more mirrors to try.
> > > > > >
> > > > > >
> > > > > > I find the same set of packages(groovy, utils, jsvc, tomcat,
> > > > zookeeper) can
> > > > > > be successfully installed across centos 6, 7 and fedora repos
> and the
> > > > other
> > > > > > same set of packages failed to install across the platforms.
> > > > Therefore, I
> > > > > > think there might be an issue happening during some sort of
> automation
> > > > > > steps.
> > > > > >
> > > > > > In addition, I suspect that those packages failed to install are
> still
> > > > > > signed by old key, hence the subkey issue found by Cos blocks the
> > > > packages
> > > > > > to be installed.
> > > > > >
> > > > > >
> > > > > > [root@34696969ce7d /]# rpm --checksig
> > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
> > > > > >
> > > > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5
> NOT OK
> > > > > > (MISSING KEYS: (MD5) PGP#d0c3824f)
> > > > > >
> > > > > > [root@34696969ce7d /]# rpm --checksig
> > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm
> > > > > >
> > > > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> > > > > >
> > > > > >
> > > > > > Cos can you first check that the hadoop* packages has been
> successfully
> > > > > > resigned by your newly generated code signing key? Thanks!
> > > > > >
> > > > > >
> > > > > > Evans
> > > > > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> > > > > >
> > > > > > > Appreciate the sentiment guys and thanks for kind words!
> > > > > > > The irony here is that I don't even like this type of
> packaging and
> > > > not
> > > > > > > using
> > > > > > > it if I can help it ;) Oh well...
> > > > > > >
> > > > > > > To close this thread - I will try to put together a blog about
> 1.0
> > > > later
> > > > > > > today. Thanks everyone for the testing, patience, and - kudos
> to
> > > > Evans -
> > > > > > > detailed instructions on how to reproduce the issue!
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > > > > > > Yes thanks cos for getting this centos stuff figured out.!
> > > > > > > >
> > > > > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <
> apurtell@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <
> > > > cos@apache.org>
> > > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >> Ok, as I suspected there's a long standing (at least from
> 2006)
> > > > bug
> > > > > > > in RPM
> > > > > > > > >> that doesn't allow to validate RPM signature if a subkey
> has
> > > > been
> > > > > > > used for
> > > > > > > > >> signing.
> > > > > > > > >>
> > > > > > > > >> I ended up generating a new key pair (just for this
> purpose) and
> > > > > > > resigning
> > > > > > > > >> all
> > > > > > > > >> binaries with it; then resyncing everything with s3. I
> also have
> > > > > > > updated
> > > > > > > > >> KEYS
> > > > > > > > >> file with the new one. I have quickly ran a test on
> centos7 by
> > > > > > > installing
> > > > > > > > >> bigtop-utils on an empty container and everything worked,
> > > > including
> > > > > > > > >> automatic
> > > > > > > > >> import of the keys and the validation/installation of the
> > > > package.
> > > > > > > Looks
> > > > > > > > >> like
> > > > > > > > >> we are in the clear.
> > > > > > > > >>
> > > > > > > > >> Please shout if you see otherwise. Thanks everyone for
> your
> > > > patience!
> > > > > > > > >> Cos
> > > > > > > > >>
> > > > > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik
> wrote:
> > > > > > > > >>> I think there's a difference between how you've signed
> the
> > > > pkgs and
> > > > > > > how
> > > > > > > > >> I did
> > > > > > > > >>> it. I signed with sub-key (as I mentioned before) and yum
> > > > doesn't
> > > > > > > > >> recognize
> > > > > > > > >>> it. Seemingly, it expects that the master key was used
> for
> > > > signing.
> > > > > > > > >>>
> > > > > > > > >>> Also, in your repo file below
> > > > > > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > > > >>> points to the old keys. The location should be
> > > > > > > > >>> gpgkey=
> > > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > > > >>>
> > > > > > > > >>> I am pretty sure I have exported my key with --armor
> option
> > > > back in
> > > > > > > the
> > > > > > > > >> day.
> > > > > > > > >>> But I will repeat it and see if I can fix the situation,
> which
> > > > I also
> > > > > > > > >> observer
> > > > > > > > >>> following your steps. If that's the only issue I will
> update
> > > > the KEYS
> > > > > > > > >> and we
> > > > > > > > >>> should be completed by tonight ;)
> > > > > > > > >>>
> > > > > > > > >>> Thanks for your help!
> > > > > > > > >>> Cos
> > > > > > > > >>>
> > > > > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > > > > > >>>> This is the same issue we're trying to solve in the
> mailing
> > > > thread
> > > > > > > > >>>> "convenience artifacts are signed and uploaded". I've
> built a
> > > > sample
> > > > > > > > >> repo
> > > > > > > > >>>> which works properly by using my own key "Evans Ye" to
> sign
> > > > and to
> > > > > > > > >> export
> > > > > > > > >>>> GPG KEY. So I believe the following steps should be the
> right
> > > > way to
> > > > > > > > >> sign
> > > > > > > > >>>> packages and export the gpgkey:
> > > > > > > > >>>>
> > > > > > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name
> Evans Ye"
> > > > > > > --addsign
> > > > > > > > >>>>
> > > > > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > > > > > >>>> I've verified that the hash is matched now in our
> official
> > > > repo.
> > > > > > > > >>>> So I guess the main issue left is using non-armored gpg
> key,
> > > > if we
> > > > > > > > >> manually
> > > > > > > > >>>> import the gpgkey in the repo file:
> > > > > > > > >>>>
> > > > > > > > >>>> [bigtop]
> > > > > > > > >>>> name=Bigtop
> > > > > > > > >>>> enabled=1
> > > > > > > > >>>> gpgcheck=1
> > > > > > > > >>>> type=NONE
> > > > > > > > >>>> baseurl=
> > > > > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > > > >>>>
> > > > > > > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > > > > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > > > >>>> error:
> https://dist.apache.org/repos/dist/release/bigtop/KEYS:
> > > > key
> > > > > > > 2
> > > > > > > > >> not an
> > > > > > > > >>>> armored public key.
> > > > > > > > >>>>
> > > > > > > > >>>> It gets error.
> > > > > > > > >>>> However, my own exported armored key can be imported
> without
> > > > an
> > > > > > > error.
> > > > > > > > >>>> That's the different.
> > > > > > > > >>>>
> > > > > > > > >>>> Can you confirm that the gpgkey(
> > > > > > > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > > > > > > >>>> is exported with --armor flag?
> > > > > > > > >>>>
> > > > > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <
> cos@apache.org
> > > > >:
> > > > > > > > >>>>
> > > > > > > > >>>>> Looks like I have figured out what's wrong with my
> key. And
> > > > it is
> > > > > > > > >>>>> _nothing_.
> > > > > > > > >>>>> However, it seems that I can not sign RPMs with subkey
> as
> > > > YUM can
> > > > > > > > >> not find
> > > > > > > > >>>>> the
> > > > > > > > >>>>> key while importing. Can anyone confirm or disprove my
> train
> > > > of
> > > > > > > > >> thoughts?
> > > > > > > > >>>>>
> > > > > > > > >>>>> Thanks!
> > > > > > > > >>>>> Cos
> > > > > > > > >>>>>
> > > > > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik
> wrote:
> > > > > > > > >>>>>> I've resynced the repodata once again and I don't see
> this
> > > > issue
> > > > > > > > >> on the
> > > > > > > > >>>>>> centos7 anymore. However, yum still complains about
> the key
> > > > being
> > > > > > > > >> no
> > > > > > > > >>>>>> available, but there's a workaround by setting
> gpgcheck=0
> > > > And I am
> > > > > > > > >> going
> > > > > > > > >>>>> to
> > > > > > > > >>>>>> figure out what to do with it and why my key isn't
> working
> > > > as
> > > > > > > > >> expected.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> I also have discovered that the gpgkey file URL is
> using
> > > > the old
> > > > > > > > >>>>> incubation
> > > > > > > > >>>>>> KEYS. Fixed that as well.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Please let me know if you still see the issue with
> checksums
> > > > > > > > >> mismatch.
> > > > > > > > >>>>>> Thanks,
> > > > > > > > >>>>>> Cos
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik
> wrote:
> > > > > > > > >>>>>>> I think this is the consequences of me fighting with
> the
> > > > package
> > > > > > > > >>>>> signing... ;(
> > > > > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for
> all the
> > > > > > > > >> RPM-based
> > > > > > > > >>>>> distros
> > > > > > > > >>>>>>> and uploaded new repo files to the release. Not sure
> why
> > > > the
> > > > > > > > >> checksums
> > > > > > > > >>>>> differ
> > > > > > > > >>>>>>> now...
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> I will take a look into this again tonight.
> > > > > > > > >>>>>>> Cos
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > > > >>>>>>>> I can second it:
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> [meins]
> > > > > > > > >>>>>>>> name=Bigtop epo
> > > > > > > > >>>>>>>> baseurl=
> > > > > > > > >>>>>
> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > > > >>>>>>>> enabled=1
> > > > > > > > >>>>>>>> gpgcheck=0
> > > > > > > > >>>>>>>> priority=1
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> and got
> > > > > > > > >>>>>>>> ............
> > > > > > > > >>>>>>>> Downloading packages:
> > > > > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > > > > > > >>>>>
> =============================================-] 849
> > > > kB/s
> > > > > > > > >> | 62
> > > > > > > > >>>>> MB 00:00:00 ETA
> > > > > > > > >>
> > > > > > >
> > > >
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > > > > > > >> :
> > > > > > > > >>>>> [Errno -1] Package does not match intended download.
> > > > Suggestion:
> > > > > > > run
> > > > > > > > >> yum
> > > > > > > > >>>>> --enablerepo=meins clean metadata
> > > > > > > > >>>>>>>> Trying other mirror.
> > > > > > > > >>>>>>>> .............
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> Olaf
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > >
> > > > > > > > > - Andy
> > > > > > > > >
> > > > > > > > > Problems worthy of attack prove their worth by hitting
> back. -
> > > > Piet
> > > > > > > Hein
> > > > > > > > > (via Tom White)
> > > > > > >
> > > >
> > > >
> > > >
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Just to make sure this doesn't fall through - I have flushed out and synced F20 repo.
Everything should be in order now. Thansk!
Cos
On Mon, Sep 07, 2015 at 03:59PM, Konstantin Boudnik wrote:
> On Tue, Sep 08, 2015 at 02:21AM, Evans Ye wrote:
> > I ran deployment test on those 3 yum repos.
> > centos6, 7 are good, however fedora is still failing.
> > It looks like the exactly same issue is still in Fedora repo which is some
> > of the rpm are still the old one signed by old key.
> >
> > # old package
> > $ rpm --checksig hadoop-2.6.0-1.fc20.x86_64.rpm
> > hadoop-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING
> > KEYS: (MD5) PGP#d0c3824f)
> >
> > # new package
> > $ rpm --checksig bigtop-utils-1.0.0-1.fc20.noarch.rpm
> > bigtop-utils-1.0.0-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> >
> > Cos could you please wipe up and resync Fedora repo on S3? Thanks!
>
> Thanks for validating and being so patient - I will do this by the end of the day today.
>
> Cos
>
> >
> > Evans
> >
> > 2015-09-06 9:39 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> >
> > > And it is done now - fresh copy of all centos packages signed by the
> > > correct
> > > key (as validated locally). Thanks for keep finding those, Evans!
> > >
> > > Cos
> > >
> > > On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
> > > > It is pretty nuts because the packages I have locally are all signed
> > > with the
> > > > correct key. But when I download the package in question I see that the
> > > local
> > > > one is different from the downloaded. And the latter is signed with the
> > > wrong
> > > > key indeed.
> > > >
> > > > Considering that I had synced everything after re-signing, there're only
> > > two
> > > > possibilities that I see:
> > > > - S3 eventual consistency bites us in the rear. Which might be possible
> > > for
> > > > in the short run, but I don't see how it couldn't be updated after
> > > all that
> > > > time
> > > > - s3cmd has screwed up and didn't updated some of the packages. I am
> > > going to
> > > > wipe out _all_ rpm distros and resync it right away. This might cause
> > > a
> > > > short interruption in the packages availability, but at least we'll
> > > have
> > > > all correct stuff up there.
> > > >
> > > > Should be done in about 30 minutes. Stay tuned
> > > > Cos
> > > >
> > > > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> > > > > Sorry guys. I'm back with the issue again. ;)
> > > > >
> > > > > Turns out that some of the rpms are good, some are not. Look at my
> > > tests
> > > > > below:
> > > > >
> > > > >
> > > > > ### Centos 6 repo ###
> > > > >
> > > > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
> > > > >
> > > > > $ wget
> > > > >
> > > https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> > > > > -O /etc/yum.repos.d/bigtop.repo
> > > > >
> > > > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
> > > > > zookeeper # Successfully installed
> > > > >
> > > > > $ yum -y install hadoop hadoop-hdfs
> > > > >
> > > > > ...
> > > > >
> > > > > Error Downloading Packages:
> > > > >
> > > > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> > > > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno
> > > 256]
> > > > > No more mirrors to try.
> > > > >
> > > > > hadoop-2.6.0-1.el6.x86_64: failure:
> > > > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
> > > > > more mirrors to try.
> > > > >
> > > > >
> > > > > I find the same set of packages(groovy, utils, jsvc, tomcat,
> > > zookeeper) can
> > > > > be successfully installed across centos 6, 7 and fedora repos and the
> > > other
> > > > > same set of packages failed to install across the platforms.
> > > Therefore, I
> > > > > think there might be an issue happening during some sort of automation
> > > > > steps.
> > > > >
> > > > > In addition, I suspect that those packages failed to install are still
> > > > > signed by old key, hence the subkey issue found by Cos blocks the
> > > packages
> > > > > to be installed.
> > > > >
> > > > >
> > > > > [root@34696969ce7d /]# rpm --checksig
> > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
> > > > >
> > > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> > > > > (MISSING KEYS: (MD5) PGP#d0c3824f)
> > > > >
> > > > > [root@34696969ce7d /]# rpm --checksig
> > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm
> > > > >
> > > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> > > > >
> > > > >
> > > > > Cos can you first check that the hadoop* packages has been successfully
> > > > > resigned by your newly generated code signing key? Thanks!
> > > > >
> > > > >
> > > > > Evans
> > > > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> > > > >
> > > > > > Appreciate the sentiment guys and thanks for kind words!
> > > > > > The irony here is that I don't even like this type of packaging and
> > > not
> > > > > > using
> > > > > > it if I can help it ;) Oh well...
> > > > > >
> > > > > > To close this thread - I will try to put together a blog about 1.0
> > > later
> > > > > > today. Thanks everyone for the testing, patience, and - kudos to
> > > Evans -
> > > > > > detailed instructions on how to reproduce the issue!
> > > > > >
> > > > > > Cos
> > > > > >
> > > > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > > > > > Yes thanks cos for getting this centos stuff figured out.!
> > > > > > >
> > > > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <apurtell@apache.org
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > > > > > >
> > > > > > > >
> > > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <
> > > cos@apache.org>
> > > > > > wrote:
> > > > > > > >>
> > > > > > > >> Ok, as I suspected there's a long standing (at least from 2006)
> > > bug
> > > > > > in RPM
> > > > > > > >> that doesn't allow to validate RPM signature if a subkey has
> > > been
> > > > > > used for
> > > > > > > >> signing.
> > > > > > > >>
> > > > > > > >> I ended up generating a new key pair (just for this purpose) and
> > > > > > resigning
> > > > > > > >> all
> > > > > > > >> binaries with it; then resyncing everything with s3. I also have
> > > > > > updated
> > > > > > > >> KEYS
> > > > > > > >> file with the new one. I have quickly ran a test on centos7 by
> > > > > > installing
> > > > > > > >> bigtop-utils on an empty container and everything worked,
> > > including
> > > > > > > >> automatic
> > > > > > > >> import of the keys and the validation/installation of the
> > > package.
> > > > > > Looks
> > > > > > > >> like
> > > > > > > >> we are in the clear.
> > > > > > > >>
> > > > > > > >> Please shout if you see otherwise. Thanks everyone for your
> > > patience!
> > > > > > > >> Cos
> > > > > > > >>
> > > > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > > > > > >>> I think there's a difference between how you've signed the
> > > pkgs and
> > > > > > how
> > > > > > > >> I did
> > > > > > > >>> it. I signed with sub-key (as I mentioned before) and yum
> > > doesn't
> > > > > > > >> recognize
> > > > > > > >>> it. Seemingly, it expects that the master key was used for
> > > signing.
> > > > > > > >>>
> > > > > > > >>> Also, in your repo file below
> > > > > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > > >>> points to the old keys. The location should be
> > > > > > > >>> gpgkey=
> > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > > >>>
> > > > > > > >>> I am pretty sure I have exported my key with --armor option
> > > back in
> > > > > > the
> > > > > > > >> day.
> > > > > > > >>> But I will repeat it and see if I can fix the situation, which
> > > I also
> > > > > > > >> observer
> > > > > > > >>> following your steps. If that's the only issue I will update
> > > the KEYS
> > > > > > > >> and we
> > > > > > > >>> should be completed by tonight ;)
> > > > > > > >>>
> > > > > > > >>> Thanks for your help!
> > > > > > > >>> Cos
> > > > > > > >>>
> > > > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > > > > >>>> This is the same issue we're trying to solve in the mailing
> > > thread
> > > > > > > >>>> "convenience artifacts are signed and uploaded". I've built a
> > > sample
> > > > > > > >> repo
> > > > > > > >>>> which works properly by using my own key "Evans Ye" to sign
> > > and to
> > > > > > > >> export
> > > > > > > >>>> GPG KEY. So I believe the following steps should be the right
> > > way to
> > > > > > > >> sign
> > > > > > > >>>> packages and export the gpgkey:
> > > > > > > >>>>
> > > > > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> > > > > > --addsign
> > > > > > > >>>>
> > > > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > > > > >>>> I've verified that the hash is matched now in our official
> > > repo.
> > > > > > > >>>> So I guess the main issue left is using non-armored gpg key,
> > > if we
> > > > > > > >> manually
> > > > > > > >>>> import the gpgkey in the repo file:
> > > > > > > >>>>
> > > > > > > >>>> [bigtop]
> > > > > > > >>>> name=Bigtop
> > > > > > > >>>> enabled=1
> > > > > > > >>>> gpgcheck=1
> > > > > > > >>>> type=NONE
> > > > > > > >>>> baseurl=
> > > > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > > >>>>
> > > > > > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > > > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS:
> > > key
> > > > > > 2
> > > > > > > >> not an
> > > > > > > >>>> armored public key.
> > > > > > > >>>>
> > > > > > > >>>> It gets error.
> > > > > > > >>>> However, my own exported armored key can be imported without
> > > an
> > > > > > error.
> > > > > > > >>>> That's the different.
> > > > > > > >>>>
> > > > > > > >>>> Can you confirm that the gpgkey(
> > > > > > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > > > > > >>>> is exported with --armor flag?
> > > > > > > >>>>
> > > > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <cos@apache.org
> > > >:
> > > > > > > >>>>
> > > > > > > >>>>> Looks like I have figured out what's wrong with my key. And
> > > it is
> > > > > > > >>>>> _nothing_.
> > > > > > > >>>>> However, it seems that I can not sign RPMs with subkey as
> > > YUM can
> > > > > > > >> not find
> > > > > > > >>>>> the
> > > > > > > >>>>> key while importing. Can anyone confirm or disprove my train
> > > of
> > > > > > > >> thoughts?
> > > > > > > >>>>>
> > > > > > > >>>>> Thanks!
> > > > > > > >>>>> Cos
> > > > > > > >>>>>
> > > > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > > > >>>>>> I've resynced the repodata once again and I don't see this
> > > issue
> > > > > > > >> on the
> > > > > > > >>>>>> centos7 anymore. However, yum still complains about the key
> > > being
> > > > > > > >> no
> > > > > > > >>>>>> available, but there's a workaround by setting gpgcheck=0
> > > And I am
> > > > > > > >> going
> > > > > > > >>>>> to
> > > > > > > >>>>>> figure out what to do with it and why my key isn't working
> > > as
> > > > > > > >> expected.
> > > > > > > >>>>>>
> > > > > > > >>>>>> I also have discovered that the gpgkey file URL is using
> > > the old
> > > > > > > >>>>> incubation
> > > > > > > >>>>>> KEYS. Fixed that as well.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Please let me know if you still see the issue with checksums
> > > > > > > >> mismatch.
> > > > > > > >>>>>> Thanks,
> > > > > > > >>>>>> Cos
> > > > > > > >>>>>>
> > > > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > > > >>>>>>> I think this is the consequences of me fighting with the
> > > package
> > > > > > > >>>>> signing... ;(
> > > > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > > > > > > >> RPM-based
> > > > > > > >>>>> distros
> > > > > > > >>>>>>> and uploaded new repo files to the release. Not sure why
> > > the
> > > > > > > >> checksums
> > > > > > > >>>>> differ
> > > > > > > >>>>>>> now...
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> I will take a look into this again tonight.
> > > > > > > >>>>>>> Cos
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > > >>>>>>>> I can second it:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> [meins]
> > > > > > > >>>>>>>> name=Bigtop epo
> > > > > > > >>>>>>>> baseurl=
> > > > > > > >>>>>
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > > >>>>>>>> enabled=1
> > > > > > > >>>>>>>> gpgcheck=0
> > > > > > > >>>>>>>> priority=1
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> and got
> > > > > > > >>>>>>>> ............
> > > > > > > >>>>>>>> Downloading packages:
> > > > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > > > > > >>>>> =============================================-] 849
> > > kB/s
> > > > > > > >> | 62
> > > > > > > >>>>> MB 00:00:00 ETA
> > > > > > > >>
> > > > > >
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > > > > > >> :
> > > > > > > >>>>> [Errno -1] Package does not match intended download.
> > > Suggestion:
> > > > > > run
> > > > > > > >> yum
> > > > > > > >>>>> --enablerepo=meins clean metadata
> > > > > > > >>>>>>>> Trying other mirror.
> > > > > > > >>>>>>>> .............
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Olaf
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > >
> > > > > > > > - Andy
> > > > > > > >
> > > > > > > > Problems worthy of attack prove their worth by hitting back. -
> > > Piet
> > > > > > Hein
> > > > > > > > (via Tom White)
> > > > > >
> > >
> > >
> > >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
On Tue, Sep 08, 2015 at 02:21AM, Evans Ye wrote:
> I ran deployment test on those 3 yum repos.
> centos6, 7 are good, however fedora is still failing.
> It looks like the exactly same issue is still in Fedora repo which is some
> of the rpm are still the old one signed by old key.
>
> # old package
> $ rpm --checksig hadoop-2.6.0-1.fc20.x86_64.rpm
> hadoop-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING
> KEYS: (MD5) PGP#d0c3824f)
>
> # new package
> $ rpm --checksig bigtop-utils-1.0.0-1.fc20.noarch.rpm
> bigtop-utils-1.0.0-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
>
> Cos could you please wipe up and resync Fedora repo on S3? Thanks!
Thanks for validating and being so patient - I will do this by the end of the day today.
Cos
>
> Evans
>
> 2015-09-06 9:39 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
> > And it is done now - fresh copy of all centos packages signed by the
> > correct
> > key (as validated locally). Thanks for keep finding those, Evans!
> >
> > Cos
> >
> > On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
> > > It is pretty nuts because the packages I have locally are all signed
> > with the
> > > correct key. But when I download the package in question I see that the
> > local
> > > one is different from the downloaded. And the latter is signed with the
> > wrong
> > > key indeed.
> > >
> > > Considering that I had synced everything after re-signing, there're only
> > two
> > > possibilities that I see:
> > > - S3 eventual consistency bites us in the rear. Which might be possible
> > for
> > > in the short run, but I don't see how it couldn't be updated after
> > all that
> > > time
> > > - s3cmd has screwed up and didn't updated some of the packages. I am
> > going to
> > > wipe out _all_ rpm distros and resync it right away. This might cause
> > a
> > > short interruption in the packages availability, but at least we'll
> > have
> > > all correct stuff up there.
> > >
> > > Should be done in about 30 minutes. Stay tuned
> > > Cos
> > >
> > > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> > > > Sorry guys. I'm back with the issue again. ;)
> > > >
> > > > Turns out that some of the rpms are good, some are not. Look at my
> > tests
> > > > below:
> > > >
> > > >
> > > > ### Centos 6 repo ###
> > > >
> > > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
> > > >
> > > > $ wget
> > > >
> > https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> > > > -O /etc/yum.repos.d/bigtop.repo
> > > >
> > > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
> > > > zookeeper # Successfully installed
> > > >
> > > > $ yum -y install hadoop hadoop-hdfs
> > > >
> > > > ...
> > > >
> > > > Error Downloading Packages:
> > > >
> > > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> > > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno
> > 256]
> > > > No more mirrors to try.
> > > >
> > > > hadoop-2.6.0-1.el6.x86_64: failure:
> > > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
> > > > more mirrors to try.
> > > >
> > > >
> > > > I find the same set of packages(groovy, utils, jsvc, tomcat,
> > zookeeper) can
> > > > be successfully installed across centos 6, 7 and fedora repos and the
> > other
> > > > same set of packages failed to install across the platforms.
> > Therefore, I
> > > > think there might be an issue happening during some sort of automation
> > > > steps.
> > > >
> > > > In addition, I suspect that those packages failed to install are still
> > > > signed by old key, hence the subkey issue found by Cos blocks the
> > packages
> > > > to be installed.
> > > >
> > > >
> > > > [root@34696969ce7d /]# rpm --checksig
> > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
> > > >
> > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> > > > (MISSING KEYS: (MD5) PGP#d0c3824f)
> > > >
> > > > [root@34696969ce7d /]# rpm --checksig
> > bigtop-groovy-2.3.8-1.fc20.noarch.rpm
> > > >
> > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> > > >
> > > >
> > > > Cos can you first check that the hadoop* packages has been successfully
> > > > resigned by your newly generated code signing key? Thanks!
> > > >
> > > >
> > > > Evans
> > > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> > > >
> > > > > Appreciate the sentiment guys and thanks for kind words!
> > > > > The irony here is that I don't even like this type of packaging and
> > not
> > > > > using
> > > > > it if I can help it ;) Oh well...
> > > > >
> > > > > To close this thread - I will try to put together a blog about 1.0
> > later
> > > > > today. Thanks everyone for the testing, patience, and - kudos to
> > Evans -
> > > > > detailed instructions on how to reproduce the issue!
> > > > >
> > > > > Cos
> > > > >
> > > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > > > > Yes thanks cos for getting this centos stuff figured out.!
> > > > > >
> > > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <apurtell@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > > > > >
> > > > > > >
> > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <
> > cos@apache.org>
> > > > > wrote:
> > > > > > >>
> > > > > > >> Ok, as I suspected there's a long standing (at least from 2006)
> > bug
> > > > > in RPM
> > > > > > >> that doesn't allow to validate RPM signature if a subkey has
> > been
> > > > > used for
> > > > > > >> signing.
> > > > > > >>
> > > > > > >> I ended up generating a new key pair (just for this purpose) and
> > > > > resigning
> > > > > > >> all
> > > > > > >> binaries with it; then resyncing everything with s3. I also have
> > > > > updated
> > > > > > >> KEYS
> > > > > > >> file with the new one. I have quickly ran a test on centos7 by
> > > > > installing
> > > > > > >> bigtop-utils on an empty container and everything worked,
> > including
> > > > > > >> automatic
> > > > > > >> import of the keys and the validation/installation of the
> > package.
> > > > > Looks
> > > > > > >> like
> > > > > > >> we are in the clear.
> > > > > > >>
> > > > > > >> Please shout if you see otherwise. Thanks everyone for your
> > patience!
> > > > > > >> Cos
> > > > > > >>
> > > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > > > > >>> I think there's a difference between how you've signed the
> > pkgs and
> > > > > how
> > > > > > >> I did
> > > > > > >>> it. I signed with sub-key (as I mentioned before) and yum
> > doesn't
> > > > > > >> recognize
> > > > > > >>> it. Seemingly, it expects that the master key was used for
> > signing.
> > > > > > >>>
> > > > > > >>> Also, in your repo file below
> > > > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > >>> points to the old keys. The location should be
> > > > > > >>> gpgkey=
> > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > >>>
> > > > > > >>> I am pretty sure I have exported my key with --armor option
> > back in
> > > > > the
> > > > > > >> day.
> > > > > > >>> But I will repeat it and see if I can fix the situation, which
> > I also
> > > > > > >> observer
> > > > > > >>> following your steps. If that's the only issue I will update
> > the KEYS
> > > > > > >> and we
> > > > > > >>> should be completed by tonight ;)
> > > > > > >>>
> > > > > > >>> Thanks for your help!
> > > > > > >>> Cos
> > > > > > >>>
> > > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > > > >>>> This is the same issue we're trying to solve in the mailing
> > thread
> > > > > > >>>> "convenience artifacts are signed and uploaded". I've built a
> > sample
> > > > > > >> repo
> > > > > > >>>> which works properly by using my own key "Evans Ye" to sign
> > and to
> > > > > > >> export
> > > > > > >>>> GPG KEY. So I believe the following steps should be the right
> > way to
> > > > > > >> sign
> > > > > > >>>> packages and export the gpgkey:
> > > > > > >>>>
> > > > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> > > > > --addsign
> > > > > > >>>>
> > > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > > > >>>> I've verified that the hash is matched now in our official
> > repo.
> > > > > > >>>> So I guess the main issue left is using non-armored gpg key,
> > if we
> > > > > > >> manually
> > > > > > >>>> import the gpgkey in the repo file:
> > > > > > >>>>
> > > > > > >>>> [bigtop]
> > > > > > >>>> name=Bigtop
> > > > > > >>>> enabled=1
> > > > > > >>>> gpgcheck=1
> > > > > > >>>> type=NONE
> > > > > > >>>> baseurl=
> > > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > > >>>>
> > > > > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS:
> > key
> > > > > 2
> > > > > > >> not an
> > > > > > >>>> armored public key.
> > > > > > >>>>
> > > > > > >>>> It gets error.
> > > > > > >>>> However, my own exported armored key can be imported without
> > an
> > > > > error.
> > > > > > >>>> That's the different.
> > > > > > >>>>
> > > > > > >>>> Can you confirm that the gpgkey(
> > > > > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > > > > >>>> is exported with --armor flag?
> > > > > > >>>>
> > > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <cos@apache.org
> > >:
> > > > > > >>>>
> > > > > > >>>>> Looks like I have figured out what's wrong with my key. And
> > it is
> > > > > > >>>>> _nothing_.
> > > > > > >>>>> However, it seems that I can not sign RPMs with subkey as
> > YUM can
> > > > > > >> not find
> > > > > > >>>>> the
> > > > > > >>>>> key while importing. Can anyone confirm or disprove my train
> > of
> > > > > > >> thoughts?
> > > > > > >>>>>
> > > > > > >>>>> Thanks!
> > > > > > >>>>> Cos
> > > > > > >>>>>
> > > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > > >>>>>> I've resynced the repodata once again and I don't see this
> > issue
> > > > > > >> on the
> > > > > > >>>>>> centos7 anymore. However, yum still complains about the key
> > being
> > > > > > >> no
> > > > > > >>>>>> available, but there's a workaround by setting gpgcheck=0
> > And I am
> > > > > > >> going
> > > > > > >>>>> to
> > > > > > >>>>>> figure out what to do with it and why my key isn't working
> > as
> > > > > > >> expected.
> > > > > > >>>>>>
> > > > > > >>>>>> I also have discovered that the gpgkey file URL is using
> > the old
> > > > > > >>>>> incubation
> > > > > > >>>>>> KEYS. Fixed that as well.
> > > > > > >>>>>>
> > > > > > >>>>>> Please let me know if you still see the issue with checksums
> > > > > > >> mismatch.
> > > > > > >>>>>> Thanks,
> > > > > > >>>>>> Cos
> > > > > > >>>>>>
> > > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > > >>>>>>> I think this is the consequences of me fighting with the
> > package
> > > > > > >>>>> signing... ;(
> > > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > > > > > >> RPM-based
> > > > > > >>>>> distros
> > > > > > >>>>>>> and uploaded new repo files to the release. Not sure why
> > the
> > > > > > >> checksums
> > > > > > >>>>> differ
> > > > > > >>>>>>> now...
> > > > > > >>>>>>>
> > > > > > >>>>>>> I will take a look into this again tonight.
> > > > > > >>>>>>> Cos
> > > > > > >>>>>>>
> > > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > >>>>>>>> I can second it:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> [meins]
> > > > > > >>>>>>>> name=Bigtop epo
> > > > > > >>>>>>>> baseurl=
> > > > > > >>>>>
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > >>>>>>>> enabled=1
> > > > > > >>>>>>>> gpgcheck=0
> > > > > > >>>>>>>> priority=1
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> and got
> > > > > > >>>>>>>> ............
> > > > > > >>>>>>>> Downloading packages:
> > > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > > > > >>>>> =============================================-] 849
> > kB/s
> > > > > > >> | 62
> > > > > > >>>>> MB 00:00:00 ETA
> > > > > > >>
> > > > >
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > > > > >> :
> > > > > > >>>>> [Errno -1] Package does not match intended download.
> > Suggestion:
> > > > > run
> > > > > > >> yum
> > > > > > >>>>> --enablerepo=meins clean metadata
> > > > > > >>>>>>>> Trying other mirror.
> > > > > > >>>>>>>> .............
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Olaf
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > >
> > > > > > > - Andy
> > > > > > >
> > > > > > > Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > > > Hein
> > > > > > > (via Tom White)
> > > > >
> >
> >
> >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Evans Ye <ev...@apache.org>.
I ran deployment test on those 3 yum repos.
centos6, 7 are good, however fedora is still failing.
It looks like the exactly same issue is still in Fedora repo which is some
of the rpm are still the old one signed by old key.
# old package
$ rpm --checksig hadoop-2.6.0-1.fc20.x86_64.rpm
hadoop-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING
KEYS: (MD5) PGP#d0c3824f)
# new package
$ rpm --checksig bigtop-utils-1.0.0-1.fc20.noarch.rpm
bigtop-utils-1.0.0-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
Cos could you please wipe up and resync Fedora repo on S3? Thanks!
Evans
2015-09-06 9:39 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> And it is done now - fresh copy of all centos packages signed by the
> correct
> key (as validated locally). Thanks for keep finding those, Evans!
>
> Cos
>
> On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
> > It is pretty nuts because the packages I have locally are all signed
> with the
> > correct key. But when I download the package in question I see that the
> local
> > one is different from the downloaded. And the latter is signed with the
> wrong
> > key indeed.
> >
> > Considering that I had synced everything after re-signing, there're only
> two
> > possibilities that I see:
> > - S3 eventual consistency bites us in the rear. Which might be possible
> for
> > in the short run, but I don't see how it couldn't be updated after
> all that
> > time
> > - s3cmd has screwed up and didn't updated some of the packages. I am
> going to
> > wipe out _all_ rpm distros and resync it right away. This might cause
> a
> > short interruption in the packages availability, but at least we'll
> have
> > all correct stuff up there.
> >
> > Should be done in about 30 minutes. Stay tuned
> > Cos
> >
> > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> > > Sorry guys. I'm back with the issue again. ;)
> > >
> > > Turns out that some of the rpms are good, some are not. Look at my
> tests
> > > below:
> > >
> > >
> > > ### Centos 6 repo ###
> > >
> > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
> > >
> > > $ wget
> > >
> https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> > > -O /etc/yum.repos.d/bigtop.repo
> > >
> > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
> > > zookeeper # Successfully installed
> > >
> > > $ yum -y install hadoop hadoop-hdfs
> > >
> > > ...
> > >
> > > Error Downloading Packages:
> > >
> > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno
> 256]
> > > No more mirrors to try.
> > >
> > > hadoop-2.6.0-1.el6.x86_64: failure:
> > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
> > > more mirrors to try.
> > >
> > >
> > > I find the same set of packages(groovy, utils, jsvc, tomcat,
> zookeeper) can
> > > be successfully installed across centos 6, 7 and fedora repos and the
> other
> > > same set of packages failed to install across the platforms.
> Therefore, I
> > > think there might be an issue happening during some sort of automation
> > > steps.
> > >
> > > In addition, I suspect that those packages failed to install are still
> > > signed by old key, hence the subkey issue found by Cos blocks the
> packages
> > > to be installed.
> > >
> > >
> > > [root@34696969ce7d /]# rpm --checksig
> hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
> > >
> > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> > > (MISSING KEYS: (MD5) PGP#d0c3824f)
> > >
> > > [root@34696969ce7d /]# rpm --checksig
> bigtop-groovy-2.3.8-1.fc20.noarch.rpm
> > >
> > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> > >
> > >
> > > Cos can you first check that the hadoop* packages has been successfully
> > > resigned by your newly generated code signing key? Thanks!
> > >
> > >
> > > Evans
> > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> > >
> > > > Appreciate the sentiment guys and thanks for kind words!
> > > > The irony here is that I don't even like this type of packaging and
> not
> > > > using
> > > > it if I can help it ;) Oh well...
> > > >
> > > > To close this thread - I will try to put together a blog about 1.0
> later
> > > > today. Thanks everyone for the testing, patience, and - kudos to
> Evans -
> > > > detailed instructions on how to reproduce the issue!
> > > >
> > > > Cos
> > > >
> > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > > > Yes thanks cos for getting this centos stuff figured out.!
> > > > >
> > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <apurtell@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > > > >
> > > > > >
> > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <
> cos@apache.org>
> > > > wrote:
> > > > > >>
> > > > > >> Ok, as I suspected there's a long standing (at least from 2006)
> bug
> > > > in RPM
> > > > > >> that doesn't allow to validate RPM signature if a subkey has
> been
> > > > used for
> > > > > >> signing.
> > > > > >>
> > > > > >> I ended up generating a new key pair (just for this purpose) and
> > > > resigning
> > > > > >> all
> > > > > >> binaries with it; then resyncing everything with s3. I also have
> > > > updated
> > > > > >> KEYS
> > > > > >> file with the new one. I have quickly ran a test on centos7 by
> > > > installing
> > > > > >> bigtop-utils on an empty container and everything worked,
> including
> > > > > >> automatic
> > > > > >> import of the keys and the validation/installation of the
> package.
> > > > Looks
> > > > > >> like
> > > > > >> we are in the clear.
> > > > > >>
> > > > > >> Please shout if you see otherwise. Thanks everyone for your
> patience!
> > > > > >> Cos
> > > > > >>
> > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > > > >>> I think there's a difference between how you've signed the
> pkgs and
> > > > how
> > > > > >> I did
> > > > > >>> it. I signed with sub-key (as I mentioned before) and yum
> doesn't
> > > > > >> recognize
> > > > > >>> it. Seemingly, it expects that the master key was used for
> signing.
> > > > > >>>
> > > > > >>> Also, in your repo file below
> > > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > >>> points to the old keys. The location should be
> > > > > >>> gpgkey=
> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > >>>
> > > > > >>> I am pretty sure I have exported my key with --armor option
> back in
> > > > the
> > > > > >> day.
> > > > > >>> But I will repeat it and see if I can fix the situation, which
> I also
> > > > > >> observer
> > > > > >>> following your steps. If that's the only issue I will update
> the KEYS
> > > > > >> and we
> > > > > >>> should be completed by tonight ;)
> > > > > >>>
> > > > > >>> Thanks for your help!
> > > > > >>> Cos
> > > > > >>>
> > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > > >>>> This is the same issue we're trying to solve in the mailing
> thread
> > > > > >>>> "convenience artifacts are signed and uploaded". I've built a
> sample
> > > > > >> repo
> > > > > >>>> which works properly by using my own key "Evans Ye" to sign
> and to
> > > > > >> export
> > > > > >>>> GPG KEY. So I believe the following steps should be the right
> way to
> > > > > >> sign
> > > > > >>>> packages and export the gpgkey:
> > > > > >>>>
> > > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> > > > --addsign
> > > > > >>>>
> > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > > >>>> I've verified that the hash is matched now in our official
> repo.
> > > > > >>>> So I guess the main issue left is using non-armored gpg key,
> if we
> > > > > >> manually
> > > > > >>>> import the gpgkey in the repo file:
> > > > > >>>>
> > > > > >>>> [bigtop]
> > > > > >>>> name=Bigtop
> > > > > >>>> enabled=1
> > > > > >>>> gpgcheck=1
> > > > > >>>> type=NONE
> > > > > >>>> baseurl=
> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > > >>>>
> > > > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS:
> key
> > > > 2
> > > > > >> not an
> > > > > >>>> armored public key.
> > > > > >>>>
> > > > > >>>> It gets error.
> > > > > >>>> However, my own exported armored key can be imported without
> an
> > > > error.
> > > > > >>>> That's the different.
> > > > > >>>>
> > > > > >>>> Can you confirm that the gpgkey(
> > > > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > > > >>>> is exported with --armor flag?
> > > > > >>>>
> > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <cos@apache.org
> >:
> > > > > >>>>
> > > > > >>>>> Looks like I have figured out what's wrong with my key. And
> it is
> > > > > >>>>> _nothing_.
> > > > > >>>>> However, it seems that I can not sign RPMs with subkey as
> YUM can
> > > > > >> not find
> > > > > >>>>> the
> > > > > >>>>> key while importing. Can anyone confirm or disprove my train
> of
> > > > > >> thoughts?
> > > > > >>>>>
> > > > > >>>>> Thanks!
> > > > > >>>>> Cos
> > > > > >>>>>
> > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > >>>>>> I've resynced the repodata once again and I don't see this
> issue
> > > > > >> on the
> > > > > >>>>>> centos7 anymore. However, yum still complains about the key
> being
> > > > > >> no
> > > > > >>>>>> available, but there's a workaround by setting gpgcheck=0
> And I am
> > > > > >> going
> > > > > >>>>> to
> > > > > >>>>>> figure out what to do with it and why my key isn't working
> as
> > > > > >> expected.
> > > > > >>>>>>
> > > > > >>>>>> I also have discovered that the gpgkey file URL is using
> the old
> > > > > >>>>> incubation
> > > > > >>>>>> KEYS. Fixed that as well.
> > > > > >>>>>>
> > > > > >>>>>> Please let me know if you still see the issue with checksums
> > > > > >> mismatch.
> > > > > >>>>>> Thanks,
> > > > > >>>>>> Cos
> > > > > >>>>>>
> > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > >>>>>>> I think this is the consequences of me fighting with the
> package
> > > > > >>>>> signing... ;(
> > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > > > > >> RPM-based
> > > > > >>>>> distros
> > > > > >>>>>>> and uploaded new repo files to the release. Not sure why
> the
> > > > > >> checksums
> > > > > >>>>> differ
> > > > > >>>>>>> now...
> > > > > >>>>>>>
> > > > > >>>>>>> I will take a look into this again tonight.
> > > > > >>>>>>> Cos
> > > > > >>>>>>>
> > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > >>>>>>>> I can second it:
> > > > > >>>>>>>>
> > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > > > >>>>>>>>
> > > > > >>>>>>>> [meins]
> > > > > >>>>>>>> name=Bigtop epo
> > > > > >>>>>>>> baseurl=
> > > > > >>>>>
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > >>>>>>>> enabled=1
> > > > > >>>>>>>> gpgcheck=0
> > > > > >>>>>>>> priority=1
> > > > > >>>>>>>>
> > > > > >>>>>>>> and got
> > > > > >>>>>>>> ............
> > > > > >>>>>>>> Downloading packages:
> > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > > > >>>>> =============================================-] 849
> kB/s
> > > > > >> | 62
> > > > > >>>>> MB 00:00:00 ETA
> > > > > >>
> > > >
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > > > >> :
> > > > > >>>>> [Errno -1] Package does not match intended download.
> Suggestion:
> > > > run
> > > > > >> yum
> > > > > >>>>> --enablerepo=meins clean metadata
> > > > > >>>>>>>> Trying other mirror.
> > > > > >>>>>>>> .............
> > > > > >>>>>>>>
> > > > > >>>>>>>> Olaf
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > > - Andy
> > > > > >
> > > > > > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > > Hein
> > > > > > (via Tom White)
> > > >
>
>
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
And it is done now - fresh copy of all centos packages signed by the correct
key (as validated locally). Thanks for keep finding those, Evans!
Cos
On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote:
> It is pretty nuts because the packages I have locally are all signed with the
> correct key. But when I download the package in question I see that the local
> one is different from the downloaded. And the latter is signed with the wrong
> key indeed.
>
> Considering that I had synced everything after re-signing, there're only two
> possibilities that I see:
> - S3 eventual consistency bites us in the rear. Which might be possible for
> in the short run, but I don't see how it couldn't be updated after all that
> time
> - s3cmd has screwed up and didn't updated some of the packages. I am going to
> wipe out _all_ rpm distros and resync it right away. This might cause a
> short interruption in the packages availability, but at least we'll have
> all correct stuff up there.
>
> Should be done in about 30 minutes. Stay tuned
> Cos
>
> On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> > Sorry guys. I'm back with the issue again. ;)
> >
> > Turns out that some of the rpms are good, some are not. Look at my tests
> > below:
> >
> >
> > ### Centos 6 repo ###
> >
> > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
> >
> > $ wget
> > https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> > -O /etc/yum.repos.d/bigtop.repo
> >
> > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
> > zookeeper # Successfully installed
> >
> > $ yum -y install hadoop hadoop-hdfs
> >
> > ...
> >
> > Error Downloading Packages:
> >
> > hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256]
> > No more mirrors to try.
> >
> > hadoop-2.6.0-1.el6.x86_64: failure:
> > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
> > more mirrors to try.
> >
> >
> > I find the same set of packages(groovy, utils, jsvc, tomcat, zookeeper) can
> > be successfully installed across centos 6, 7 and fedora repos and the other
> > same set of packages failed to install across the platforms. Therefore, I
> > think there might be an issue happening during some sort of automation
> > steps.
> >
> > In addition, I suspect that those packages failed to install are still
> > signed by old key, hence the subkey issue found by Cos blocks the packages
> > to be installed.
> >
> >
> > [root@34696969ce7d /]# rpm --checksig hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
> >
> > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> > (MISSING KEYS: (MD5) PGP#d0c3824f)
> >
> > [root@34696969ce7d /]# rpm --checksig bigtop-groovy-2.3.8-1.fc20.noarch.rpm
> >
> > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
> >
> >
> > Cos can you first check that the hadoop* packages has been successfully
> > resigned by your newly generated code signing key? Thanks!
> >
> >
> > Evans
> > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> >
> > > Appreciate the sentiment guys and thanks for kind words!
> > > The irony here is that I don't even like this type of packaging and not
> > > using
> > > it if I can help it ;) Oh well...
> > >
> > > To close this thread - I will try to put together a blog about 1.0 later
> > > today. Thanks everyone for the testing, patience, and - kudos to Evans -
> > > detailed instructions on how to reproduce the issue!
> > >
> > > Cos
> > >
> > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > > Yes thanks cos for getting this centos stuff figured out.!
> > > >
> > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > > > >
> > > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > > >
> > > > >
> > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > > >>
> > > > >> Ok, as I suspected there's a long standing (at least from 2006) bug
> > > in RPM
> > > > >> that doesn't allow to validate RPM signature if a subkey has been
> > > used for
> > > > >> signing.
> > > > >>
> > > > >> I ended up generating a new key pair (just for this purpose) and
> > > resigning
> > > > >> all
> > > > >> binaries with it; then resyncing everything with s3. I also have
> > > updated
> > > > >> KEYS
> > > > >> file with the new one. I have quickly ran a test on centos7 by
> > > installing
> > > > >> bigtop-utils on an empty container and everything worked, including
> > > > >> automatic
> > > > >> import of the keys and the validation/installation of the package.
> > > Looks
> > > > >> like
> > > > >> we are in the clear.
> > > > >>
> > > > >> Please shout if you see otherwise. Thanks everyone for your patience!
> > > > >> Cos
> > > > >>
> > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > > >>> I think there's a difference between how you've signed the pkgs and
> > > how
> > > > >> I did
> > > > >>> it. I signed with sub-key (as I mentioned before) and yum doesn't
> > > > >> recognize
> > > > >>> it. Seemingly, it expects that the master key was used for signing.
> > > > >>>
> > > > >>> Also, in your repo file below
> > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > >>> points to the old keys. The location should be
> > > > >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > >>>
> > > > >>> I am pretty sure I have exported my key with --armor option back in
> > > the
> > > > >> day.
> > > > >>> But I will repeat it and see if I can fix the situation, which I also
> > > > >> observer
> > > > >>> following your steps. If that's the only issue I will update the KEYS
> > > > >> and we
> > > > >>> should be completed by tonight ;)
> > > > >>>
> > > > >>> Thanks for your help!
> > > > >>> Cos
> > > > >>>
> > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > >>>> This is the same issue we're trying to solve in the mailing thread
> > > > >>>> "convenience artifacts are signed and uploaded". I've built a sample
> > > > >> repo
> > > > >>>> which works properly by using my own key "Evans Ye" to sign and to
> > > > >> export
> > > > >>>> GPG KEY. So I believe the following steps should be the right way to
> > > > >> sign
> > > > >>>> packages and export the gpgkey:
> > > > >>>>
> > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> > > --addsign
> > > > >>>>
> > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > >>>> I've verified that the hash is matched now in our official repo.
> > > > >>>> So I guess the main issue left is using non-armored gpg key, if we
> > > > >> manually
> > > > >>>> import the gpgkey in the repo file:
> > > > >>>>
> > > > >>>> [bigtop]
> > > > >>>> name=Bigtop
> > > > >>>> enabled=1
> > > > >>>> gpgcheck=1
> > > > >>>> type=NONE
> > > > >>>> baseurl=
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > > >>>>
> > > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key
> > > 2
> > > > >> not an
> > > > >>>> armored public key.
> > > > >>>>
> > > > >>>> It gets error.
> > > > >>>> However, my own exported armored key can be imported without an
> > > error.
> > > > >>>> That's the different.
> > > > >>>>
> > > > >>>> Can you confirm that the gpgkey(
> > > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > > >>>> is exported with --armor flag?
> > > > >>>>
> > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > > > >>>>
> > > > >>>>> Looks like I have figured out what's wrong with my key. And it is
> > > > >>>>> _nothing_.
> > > > >>>>> However, it seems that I can not sign RPMs with subkey as YUM can
> > > > >> not find
> > > > >>>>> the
> > > > >>>>> key while importing. Can anyone confirm or disprove my train of
> > > > >> thoughts?
> > > > >>>>>
> > > > >>>>> Thanks!
> > > > >>>>> Cos
> > > > >>>>>
> > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > >>>>>> I've resynced the repodata once again and I don't see this issue
> > > > >> on the
> > > > >>>>>> centos7 anymore. However, yum still complains about the key being
> > > > >> no
> > > > >>>>>> available, but there's a workaround by setting gpgcheck=0 And I am
> > > > >> going
> > > > >>>>> to
> > > > >>>>>> figure out what to do with it and why my key isn't working as
> > > > >> expected.
> > > > >>>>>>
> > > > >>>>>> I also have discovered that the gpgkey file URL is using the old
> > > > >>>>> incubation
> > > > >>>>>> KEYS. Fixed that as well.
> > > > >>>>>>
> > > > >>>>>> Please let me know if you still see the issue with checksums
> > > > >> mismatch.
> > > > >>>>>> Thanks,
> > > > >>>>>> Cos
> > > > >>>>>>
> > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > >>>>>>> I think this is the consequences of me fighting with the package
> > > > >>>>> signing... ;(
> > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > > > >> RPM-based
> > > > >>>>> distros
> > > > >>>>>>> and uploaded new repo files to the release. Not sure why the
> > > > >> checksums
> > > > >>>>> differ
> > > > >>>>>>> now...
> > > > >>>>>>>
> > > > >>>>>>> I will take a look into this again tonight.
> > > > >>>>>>> Cos
> > > > >>>>>>>
> > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > >>>>>>>> I can second it:
> > > > >>>>>>>>
> > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > > >>>>>>>>
> > > > >>>>>>>> [meins]
> > > > >>>>>>>> name=Bigtop epo
> > > > >>>>>>>> baseurl=
> > > > >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > >>>>>>>> enabled=1
> > > > >>>>>>>> gpgcheck=0
> > > > >>>>>>>> priority=1
> > > > >>>>>>>>
> > > > >>>>>>>> and got
> > > > >>>>>>>> ............
> > > > >>>>>>>> Downloading packages:
> > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > > >>>>> =============================================-] 849 kB/s
> > > > >> | 62
> > > > >>>>> MB 00:00:00 ETA
> > > > >>
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > > >> :
> > > > >>>>> [Errno -1] Package does not match intended download. Suggestion:
> > > run
> > > > >> yum
> > > > >>>>> --enablerepo=meins clean metadata
> > > > >>>>>>>> Trying other mirror.
> > > > >>>>>>>> .............
> > > > >>>>>>>>
> > > > >>>>>>>> Olaf
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
It is pretty nuts because the packages I have locally are all signed with the
correct key. But when I download the package in question I see that the local
one is different from the downloaded. And the latter is signed with the wrong
key indeed.
Considering that I had synced everything after re-signing, there're only two
possibilities that I see:
- S3 eventual consistency bites us in the rear. Which might be possible for
in the short run, but I don't see how it couldn't be updated after all that
time
- s3cmd has screwed up and didn't updated some of the packages. I am going to
wipe out _all_ rpm distros and resync it right away. This might cause a
short interruption in the packages availability, but at least we'll have
all correct stuff up there.
Should be done in about 30 minutes. Stay tuned
Cos
On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote:
> Sorry guys. I'm back with the issue again. ;)
>
> Turns out that some of the rpms are good, some are not. Look at my tests
> below:
>
>
> ### Centos 6 repo ###
>
> $ docker run -ti --rm bigtop/puppet:centos-6 bash -l
>
> $ wget
> https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
> -O /etc/yum.repos.d/bigtop.repo
>
> $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
> zookeeper # Successfully installed
>
> $ yum -y install hadoop hadoop-hdfs
>
> ...
>
> Error Downloading Packages:
>
> hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
> hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256]
> No more mirrors to try.
>
> hadoop-2.6.0-1.el6.x86_64: failure:
> hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
> more mirrors to try.
>
>
> I find the same set of packages(groovy, utils, jsvc, tomcat, zookeeper) can
> be successfully installed across centos 6, 7 and fedora repos and the other
> same set of packages failed to install across the platforms. Therefore, I
> think there might be an issue happening during some sort of automation
> steps.
>
> In addition, I suspect that those packages failed to install are still
> signed by old key, hence the subkey issue found by Cos blocks the packages
> to be installed.
>
>
> [root@34696969ce7d /]# rpm --checksig hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
>
> hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
> (MISSING KEYS: (MD5) PGP#d0c3824f)
>
> [root@34696969ce7d /]# rpm --checksig bigtop-groovy-2.3.8-1.fc20.noarch.rpm
>
> bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
>
>
> Cos can you first check that the hadoop* packages has been successfully
> resigned by your newly generated code signing key? Thanks!
>
>
> Evans
> 2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
>
> > Appreciate the sentiment guys and thanks for kind words!
> > The irony here is that I don't even like this type of packaging and not
> > using
> > it if I can help it ;) Oh well...
> >
> > To close this thread - I will try to put together a blog about 1.0 later
> > today. Thanks everyone for the testing, patience, and - kudos to Evans -
> > detailed instructions on how to reproduce the issue!
> >
> > Cos
> >
> > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > > Yes thanks cos for getting this centos stuff figured out.!
> > >
> > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> > > >
> > > > Thanks for sticking with it Cos. That's an annoying bug.
> > > >
> > > >
> > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org>
> > wrote:
> > > >>
> > > >> Ok, as I suspected there's a long standing (at least from 2006) bug
> > in RPM
> > > >> that doesn't allow to validate RPM signature if a subkey has been
> > used for
> > > >> signing.
> > > >>
> > > >> I ended up generating a new key pair (just for this purpose) and
> > resigning
> > > >> all
> > > >> binaries with it; then resyncing everything with s3. I also have
> > updated
> > > >> KEYS
> > > >> file with the new one. I have quickly ran a test on centos7 by
> > installing
> > > >> bigtop-utils on an empty container and everything worked, including
> > > >> automatic
> > > >> import of the keys and the validation/installation of the package.
> > Looks
> > > >> like
> > > >> we are in the clear.
> > > >>
> > > >> Please shout if you see otherwise. Thanks everyone for your patience!
> > > >> Cos
> > > >>
> > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > >>> I think there's a difference between how you've signed the pkgs and
> > how
> > > >> I did
> > > >>> it. I signed with sub-key (as I mentioned before) and yum doesn't
> > > >> recognize
> > > >>> it. Seemingly, it expects that the master key was used for signing.
> > > >>>
> > > >>> Also, in your repo file below
> > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > >>> points to the old keys. The location should be
> > > >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > >>>
> > > >>> I am pretty sure I have exported my key with --armor option back in
> > the
> > > >> day.
> > > >>> But I will repeat it and see if I can fix the situation, which I also
> > > >> observer
> > > >>> following your steps. If that's the only issue I will update the KEYS
> > > >> and we
> > > >>> should be completed by tonight ;)
> > > >>>
> > > >>> Thanks for your help!
> > > >>> Cos
> > > >>>
> > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > >>>> This is the same issue we're trying to solve in the mailing thread
> > > >>>> "convenience artifacts are signed and uploaded". I've built a sample
> > > >> repo
> > > >>>> which works properly by using my own key "Evans Ye" to sign and to
> > > >> export
> > > >>>> GPG KEY. So I believe the following steps should be the right way to
> > > >> sign
> > > >>>> packages and export the gpgkey:
> > > >>>>
> > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> > --addsign
> > > >>>>
> > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > > >>>> I've verified that the hash is matched now in our official repo.
> > > >>>> So I guess the main issue left is using non-armored gpg key, if we
> > > >> manually
> > > >>>> import the gpgkey in the repo file:
> > > >>>>
> > > >>>> [bigtop]
> > > >>>> name=Bigtop
> > > >>>> enabled=1
> > > >>>> gpgcheck=1
> > > >>>> type=NONE
> > > >>>> baseurl=
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > >>>>
> > > >>>> [root@48723d98dc1b ~]# rpm --import
> > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key
> > 2
> > > >> not an
> > > >>>> armored public key.
> > > >>>>
> > > >>>> It gets error.
> > > >>>> However, my own exported armored key can be imported without an
> > error.
> > > >>>> That's the different.
> > > >>>>
> > > >>>> Can you confirm that the gpgkey(
> > > >> http://archive.apache.org/dist/bigtop/KEYS)
> > > >>>> is exported with --armor flag?
> > > >>>>
> > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > > >>>>
> > > >>>>> Looks like I have figured out what's wrong with my key. And it is
> > > >>>>> _nothing_.
> > > >>>>> However, it seems that I can not sign RPMs with subkey as YUM can
> > > >> not find
> > > >>>>> the
> > > >>>>> key while importing. Can anyone confirm or disprove my train of
> > > >> thoughts?
> > > >>>>>
> > > >>>>> Thanks!
> > > >>>>> Cos
> > > >>>>>
> > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > >>>>>> I've resynced the repodata once again and I don't see this issue
> > > >> on the
> > > >>>>>> centos7 anymore. However, yum still complains about the key being
> > > >> no
> > > >>>>>> available, but there's a workaround by setting gpgcheck=0 And I am
> > > >> going
> > > >>>>> to
> > > >>>>>> figure out what to do with it and why my key isn't working as
> > > >> expected.
> > > >>>>>>
> > > >>>>>> I also have discovered that the gpgkey file URL is using the old
> > > >>>>> incubation
> > > >>>>>> KEYS. Fixed that as well.
> > > >>>>>>
> > > >>>>>> Please let me know if you still see the issue with checksums
> > > >> mismatch.
> > > >>>>>> Thanks,
> > > >>>>>> Cos
> > > >>>>>>
> > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > >>>>>>> I think this is the consequences of me fighting with the package
> > > >>>>> signing... ;(
> > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > > >> RPM-based
> > > >>>>> distros
> > > >>>>>>> and uploaded new repo files to the release. Not sure why the
> > > >> checksums
> > > >>>>> differ
> > > >>>>>>> now...
> > > >>>>>>>
> > > >>>>>>> I will take a look into this again tonight.
> > > >>>>>>> Cos
> > > >>>>>>>
> > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > >>>>>>>> I can second it:
> > > >>>>>>>>
> > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > > >>>>>>>>
> > > >>>>>>>> [meins]
> > > >>>>>>>> name=Bigtop epo
> > > >>>>>>>> baseurl=
> > > >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > >>>>>>>> enabled=1
> > > >>>>>>>> gpgcheck=0
> > > >>>>>>>> priority=1
> > > >>>>>>>>
> > > >>>>>>>> and got
> > > >>>>>>>> ............
> > > >>>>>>>> Downloading packages:
> > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > > >>>>> =============================================-] 849 kB/s
> > > >> | 62
> > > >>>>> MB 00:00:00 ETA
> > > >>
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > > >> :
> > > >>>>> [Errno -1] Package does not match intended download. Suggestion:
> > run
> > > >> yum
> > > >>>>> --enablerepo=meins clean metadata
> > > >>>>>>>> Trying other mirror.
> > > >>>>>>>> .............
> > > >>>>>>>>
> > > >>>>>>>> Olaf
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Evans Ye <ev...@apache.org>.
Sorry guys. I'm back with the issue again. ;)
Turns out that some of the rpms are good, some are not. Look at my tests
below:
### Centos 6 repo ###
$ docker run -ti --rm bigtop/puppet:centos-6 bash -l
$ wget
https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo
-O /etc/yum.repos.d/bigtop.repo
$ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat
zookeeper # Successfully installed
$ yum -y install hadoop hadoop-hdfs
...
Error Downloading Packages:
hadoop-hdfs-2.6.0-1.el6.x86_64: failure:
hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256]
No more mirrors to try.
hadoop-2.6.0-1.el6.x86_64: failure:
hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No
more mirrors to try.
I find the same set of packages(groovy, utils, jsvc, tomcat, zookeeper) can
be successfully installed across centos 6, 7 and fedora repos and the other
same set of packages failed to install across the platforms. Therefore, I
think there might be an issue happening during some sort of automation
steps.
In addition, I suspect that those packages failed to install are still
signed by old key, hence the subkey issue found by Cos blocks the packages
to be installed.
[root@34696969ce7d /]# rpm --checksig hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm
hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK
(MISSING KEYS: (MD5) PGP#d0c3824f)
[root@34696969ce7d /]# rpm --checksig bigtop-groovy-2.3.8-1.fc20.noarch.rpm
bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK
Cos can you first check that the hadoop* packages has been successfully
resigned by your newly generated code signing key? Thanks!
Evans
2015年9月4日 上午2:23於 "Konstantin Boudnik" <co...@apache.org>寫道：
> Appreciate the sentiment guys and thanks for kind words!
> The irony here is that I don't even like this type of packaging and not
> using
> it if I can help it ;) Oh well...
>
> To close this thread - I will try to put together a blog about 1.0 later
> today. Thanks everyone for the testing, patience, and - kudos to Evans -
> detailed instructions on how to reproduce the issue!
>
> Cos
>
> On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> > Yes thanks cos for getting this centos stuff figured out.!
> >
> > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > >
> > > Thanks for sticking with it Cos. That's an annoying bug.
> > >
> > >
> > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org>
> wrote:
> > >>
> > >> Ok, as I suspected there's a long standing (at least from 2006) bug
> in RPM
> > >> that doesn't allow to validate RPM signature if a subkey has been
> used for
> > >> signing.
> > >>
> > >> I ended up generating a new key pair (just for this purpose) and
> resigning
> > >> all
> > >> binaries with it; then resyncing everything with s3. I also have
> updated
> > >> KEYS
> > >> file with the new one. I have quickly ran a test on centos7 by
> installing
> > >> bigtop-utils on an empty container and everything worked, including
> > >> automatic
> > >> import of the keys and the validation/installation of the package.
> Looks
> > >> like
> > >> we are in the clear.
> > >>
> > >> Please shout if you see otherwise. Thanks everyone for your patience!
> > >> Cos
> > >>
> > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > >>> I think there's a difference between how you've signed the pkgs and
> how
> > >> I did
> > >>> it. I signed with sub-key (as I mentioned before) and yum doesn't
> > >> recognize
> > >>> it. Seemingly, it expects that the master key was used for signing.
> > >>>
> > >>> Also, in your repo file below
> > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > >>> points to the old keys. The location should be
> > >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > >>>
> > >>> I am pretty sure I have exported my key with --armor option back in
> the
> > >> day.
> > >>> But I will repeat it and see if I can fix the situation, which I also
> > >> observer
> > >>> following your steps. If that's the only issue I will update the KEYS
> > >> and we
> > >>> should be completed by tonight ;)
> > >>>
> > >>> Thanks for your help!
> > >>> Cos
> > >>>
> > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > >>>> This is the same issue we're trying to solve in the mailing thread
> > >>>> "convenience artifacts are signed and uploaded". I've built a sample
> > >> repo
> > >>>> which works properly by using my own key "Evans Ye" to sign and to
> > >> export
> > >>>> GPG KEY. So I believe the following steps should be the right way to
> > >> sign
> > >>>> packages and export the gpgkey:
> > >>>>
> > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> --addsign
> > >>>>
> > >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> > >>>> I've verified that the hash is matched now in our official repo.
> > >>>> So I guess the main issue left is using non-armored gpg key, if we
> > >> manually
> > >>>> import the gpgkey in the repo file:
> > >>>>
> > >>>> [bigtop]
> > >>>> name=Bigtop
> > >>>> enabled=1
> > >>>> gpgcheck=1
> > >>>> type=NONE
> > >>>> baseurl=
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > >>>>
> > >>>> [root@48723d98dc1b ~]# rpm --import
> > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key
> 2
> > >> not an
> > >>>> armored public key.
> > >>>>
> > >>>> It gets error.
> > >>>> However, my own exported armored key can be imported without an
> error.
> > >>>> That's the different.
> > >>>>
> > >>>> Can you confirm that the gpgkey(
> > >> http://archive.apache.org/dist/bigtop/KEYS)
> > >>>> is exported with --armor flag?
> > >>>>
> > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > >>>>
> > >>>>> Looks like I have figured out what's wrong with my key. And it is
> > >>>>> _nothing_.
> > >>>>> However, it seems that I can not sign RPMs with subkey as YUM can
> > >> not find
> > >>>>> the
> > >>>>> key while importing. Can anyone confirm or disprove my train of
> > >> thoughts?
> > >>>>>
> > >>>>> Thanks!
> > >>>>> Cos
> > >>>>>
> > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > >>>>>> I've resynced the repodata once again and I don't see this issue
> > >> on the
> > >>>>>> centos7 anymore. However, yum still complains about the key being
> > >> no
> > >>>>>> available, but there's a workaround by setting gpgcheck=0 And I am
> > >> going
> > >>>>> to
> > >>>>>> figure out what to do with it and why my key isn't working as
> > >> expected.
> > >>>>>>
> > >>>>>> I also have discovered that the gpgkey file URL is using the old
> > >>>>> incubation
> > >>>>>> KEYS. Fixed that as well.
> > >>>>>>
> > >>>>>> Please let me know if you still see the issue with checksums
> > >> mismatch.
> > >>>>>> Thanks,
> > >>>>>> Cos
> > >>>>>>
> > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > >>>>>>> I think this is the consequences of me fighting with the package
> > >>>>> signing... ;(
> > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> > >> RPM-based
> > >>>>> distros
> > >>>>>>> and uploaded new repo files to the release. Not sure why the
> > >> checksums
> > >>>>> differ
> > >>>>>>> now...
> > >>>>>>>
> > >>>>>>> I will take a look into this again tonight.
> > >>>>>>> Cos
> > >>>>>>>
> > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > >>>>>>>> I can second it:
> > >>>>>>>>
> > >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> > >>>>>>>>
> > >>>>>>>> [meins]
> > >>>>>>>> name=Bigtop epo
> > >>>>>>>> baseurl=
> > >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > >>>>>>>> enabled=1
> > >>>>>>>> gpgcheck=0
> > >>>>>>>> priority=1
> > >>>>>>>>
> > >>>>>>>> and got
> > >>>>>>>> ............
> > >>>>>>>> Downloading packages:
> > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> > >>>>> =============================================-] 849 kB/s
> > >> | 62
> > >>>>> MB 00:00:00 ETA
> > >>
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> > >> :
> > >>>>> [Errno -1] Package does not match intended download. Suggestion:
> run
> > >> yum
> > >>>>> --enablerepo=meins clean metadata
> > >>>>>>>> Trying other mirror.
> > >>>>>>>> .............
> > >>>>>>>>
> > >>>>>>>> Olaf
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Appreciate the sentiment guys and thanks for kind words!
The irony here is that I don't even like this type of packaging and not using
it if I can help it ;) Oh well...
To close this thread - I will try to put together a blog about 1.0 later
today. Thanks everyone for the testing, patience, and - kudos to Evans -
detailed instructions on how to reproduce the issue!
Cos
On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote:
> Yes thanks cos for getting this centos stuff figured out.!
>
> > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <ap...@apache.org> wrote:
> >
> > Thanks for sticking with it Cos. That's an annoying bug.
> >
> >
> >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org> wrote:
> >>
> >> Ok, as I suspected there's a long standing (at least from 2006) bug in RPM
> >> that doesn't allow to validate RPM signature if a subkey has been used for
> >> signing.
> >>
> >> I ended up generating a new key pair (just for this purpose) and resigning
> >> all
> >> binaries with it; then resyncing everything with s3. I also have updated
> >> KEYS
> >> file with the new one. I have quickly ran a test on centos7 by installing
> >> bigtop-utils on an empty container and everything worked, including
> >> automatic
> >> import of the keys and the validation/installation of the package. Looks
> >> like
> >> we are in the clear.
> >>
> >> Please shout if you see otherwise. Thanks everyone for your patience!
> >> Cos
> >>
> >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> >>> I think there's a difference between how you've signed the pkgs and how
> >> I did
> >>> it. I signed with sub-key (as I mentioned before) and yum doesn't
> >> recognize
> >>> it. Seemingly, it expects that the master key was used for signing.
> >>>
> >>> Also, in your repo file below
> >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> >>> points to the old keys. The location should be
> >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> >>>
> >>> I am pretty sure I have exported my key with --armor option back in the
> >> day.
> >>> But I will repeat it and see if I can fix the situation, which I also
> >> observer
> >>> following your steps. If that's the only issue I will update the KEYS
> >> and we
> >>> should be completed by tonight ;)
> >>>
> >>> Thanks for your help!
> >>> Cos
> >>>
> >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> >>>> This is the same issue we're trying to solve in the mailing thread
> >>>> "convenience artifacts are signed and uploaded". I've built a sample
> >> repo
> >>>> which works properly by using my own key "Evans Ye" to sign and to
> >> export
> >>>> GPG KEY. So I believe the following steps should be the right way to
> >> sign
> >>>> packages and export the gpgkey:
> >>>>
> >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
> >>>>
> >>>> $ gpg --armor --output KEYS --export 'Evans Ye'
> >>>> I've verified that the hash is matched now in our official repo.
> >>>> So I guess the main issue left is using non-armored gpg key, if we
> >> manually
> >>>> import the gpgkey in the repo file:
> >>>>
> >>>> [bigtop]
> >>>> name=Bigtop
> >>>> enabled=1
> >>>> gpgcheck=1
> >>>> type=NONE
> >>>> baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> >>>>
> >>>> [root@48723d98dc1b ~]# rpm --import
> >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2
> >> not an
> >>>> armored public key.
> >>>>
> >>>> It gets error.
> >>>> However, my own exported armored key can be imported without an error.
> >>>> That's the different.
> >>>>
> >>>> Can you confirm that the gpgkey(
> >> http://archive.apache.org/dist/bigtop/KEYS)
> >>>> is exported with --armor flag?
> >>>>
> >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> >>>>
> >>>>> Looks like I have figured out what's wrong with my key. And it is
> >>>>> _nothing_.
> >>>>> However, it seems that I can not sign RPMs with subkey as YUM can
> >> not find
> >>>>> the
> >>>>> key while importing. Can anyone confirm or disprove my train of
> >> thoughts?
> >>>>>
> >>>>> Thanks!
> >>>>> Cos
> >>>>>
> >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> >>>>>> I've resynced the repodata once again and I don't see this issue
> >> on the
> >>>>>> centos7 anymore. However, yum still complains about the key being
> >> no
> >>>>>> available, but there's a workaround by setting gpgcheck=0 And I am
> >> going
> >>>>> to
> >>>>>> figure out what to do with it and why my key isn't working as
> >> expected.
> >>>>>>
> >>>>>> I also have discovered that the gpgkey file URL is using the old
> >>>>> incubation
> >>>>>> KEYS. Fixed that as well.
> >>>>>>
> >>>>>> Please let me know if you still see the issue with checksums
> >> mismatch.
> >>>>>> Thanks,
> >>>>>> Cos
> >>>>>>
> >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> >>>>>>> I think this is the consequences of me fighting with the package
> >>>>> signing... ;(
> >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
> >> RPM-based
> >>>>> distros
> >>>>>>> and uploaded new repo files to the release. Not sure why the
> >> checksums
> >>>>> differ
> >>>>>>> now...
> >>>>>>>
> >>>>>>> I will take a look into this again tonight.
> >>>>>>> Cos
> >>>>>>>
> >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> >>>>>>>> I can second it:
> >>>>>>>>
> >>>>>>>> I added to /etc/yum.repo.d/meins.repo
> >>>>>>>>
> >>>>>>>> [meins]
> >>>>>>>> name=Bigtop epo
> >>>>>>>> baseurl=
> >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> >>>>>>>> enabled=1
> >>>>>>>> gpgcheck=0
> >>>>>>>> priority=1
> >>>>>>>>
> >>>>>>>> and got
> >>>>>>>> ............
> >>>>>>>> Downloading packages:
> >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
> >>>>> =============================================-] 849 kB/s
> >> | 62
> >>>>> MB 00:00:00 ETA
> >> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> >> :
> >>>>> [Errno -1] Package does not match intended download. Suggestion: run
> >> yum
> >>>>> --enablerepo=meins clean metadata
> >>>>>>>> Trying other mirror.
> >>>>>>>> .............
> >>>>>>>>
> >>>>>>>> Olaf
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
Re: CentOS Package Checksums Don't Match RPMs
Posted by Jay Vyas <ja...@gmail.com>.
Yes thanks cos for getting this centos stuff figured out.!
> On Sep 3, 2015, at 12:35 PM, Andrew Purtell <ap...@apache.org> wrote:
>
> Thanks for sticking with it Cos. That's an annoying bug.
>
>
>> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>> Ok, as I suspected there's a long standing (at least from 2006) bug in RPM
>> that doesn't allow to validate RPM signature if a subkey has been used for
>> signing.
>>
>> I ended up generating a new key pair (just for this purpose) and resigning
>> all
>> binaries with it; then resyncing everything with s3. I also have updated
>> KEYS
>> file with the new one. I have quickly ran a test on centos7 by installing
>> bigtop-utils on an empty container and everything worked, including
>> automatic
>> import of the keys and the validation/installation of the package. Looks
>> like
>> we are in the clear.
>>
>> Please shout if you see otherwise. Thanks everyone for your patience!
>> Cos
>>
>>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
>>> I think there's a difference between how you've signed the pkgs and how
>> I did
>>> it. I signed with sub-key (as I mentioned before) and yum doesn't
>> recognize
>>> it. Seemingly, it expects that the master key was used for signing.
>>>
>>> Also, in your repo file below
>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>>> points to the old keys. The location should be
>>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
>>>
>>> I am pretty sure I have exported my key with --armor option back in the
>> day.
>>> But I will repeat it and see if I can fix the situation, which I also
>> observer
>>> following your steps. If that's the only issue I will update the KEYS
>> and we
>>> should be completed by tonight ;)
>>>
>>> Thanks for your help!
>>> Cos
>>>
>>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
>>>> This is the same issue we're trying to solve in the mailing thread
>>>> "convenience artifacts are signed and uploaded". I've built a sample
>> repo
>>>> which works properly by using my own key "Evans Ye" to sign and to
>> export
>>>> GPG KEY. So I believe the following steps should be the right way to
>> sign
>>>> packages and export the gpgkey:
>>>>
>>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
>>>>
>>>> $ gpg --armor --output KEYS --export 'Evans Ye'
>>>> I've verified that the hash is matched now in our official repo.
>>>> So I guess the main issue left is using non-armored gpg key, if we
>> manually
>>>> import the gpgkey in the repo file:
>>>>
>>>> [bigtop]
>>>> name=Bigtop
>>>> enabled=1
>>>> gpgcheck=1
>>>> type=NONE
>>>> baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
>>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>>>>
>>>> [root@48723d98dc1b ~]# rpm --import
>>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
>>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2
>> not an
>>>> armored public key.
>>>>
>>>> It gets error.
>>>> However, my own exported armored key can be imported without an error.
>>>> That's the different.
>>>>
>>>> Can you confirm that the gpgkey(
>> http://archive.apache.org/dist/bigtop/KEYS)
>>>> is exported with --armor flag?
>>>>
>>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>>>>
>>>>> Looks like I have figured out what's wrong with my key. And it is
>>>>> _nothing_.
>>>>> However, it seems that I can not sign RPMs with subkey as YUM can
>> not find
>>>>> the
>>>>> key while importing. Can anyone confirm or disprove my train of
>> thoughts?
>>>>>
>>>>> Thanks!
>>>>> Cos
>>>>>
>>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
>>>>>> I've resynced the repodata once again and I don't see this issue
>> on the
>>>>>> centos7 anymore. However, yum still complains about the key being
>> no
>>>>>> available, but there's a workaround by setting gpgcheck=0 And I am
>> going
>>>>> to
>>>>>> figure out what to do with it and why my key isn't working as
>> expected.
>>>>>>
>>>>>> I also have discovered that the gpgkey file URL is using the old
>>>>> incubation
>>>>>> KEYS. Fixed that as well.
>>>>>>
>>>>>> Please let me know if you still see the issue with checksums
>> mismatch.
>>>>>> Thanks,
>>>>>> Cos
>>>>>>
>>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
>>>>>>> I think this is the consequences of me fighting with the package
>>>>> signing... ;(
>>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
>> RPM-based
>>>>> distros
>>>>>>> and uploaded new repo files to the release. Not sure why the
>> checksums
>>>>> differ
>>>>>>> now...
>>>>>>>
>>>>>>> I will take a look into this again tonight.
>>>>>>> Cos
>>>>>>>
>>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
>>>>>>>> I can second it:
>>>>>>>>
>>>>>>>> I added to /etc/yum.repo.d/meins.repo
>>>>>>>>
>>>>>>>> [meins]
>>>>>>>> name=Bigtop epo
>>>>>>>> baseurl=
>>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
>>>>>>>> enabled=1
>>>>>>>> gpgcheck=0
>>>>>>>> priority=1
>>>>>>>>
>>>>>>>> and got
>>>>>>>> ............
>>>>>>>> Downloading packages:
>>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
>>>>> =============================================-] 849 kB/s
>> | 62
>>>>> MB 00:00:00 ETA
>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
>> :
>>>>> [Errno -1] Package does not match intended download. Suggestion: run
>> yum
>>>>> --enablerepo=meins clean metadata
>>>>>>>> Trying other mirror.
>>>>>>>> .............
>>>>>>>>
>>>>>>>> Olaf
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
Re: CentOS Package Checksums Don't Match RPMs
Posted by Andrew Purtell <ap...@apache.org>.
Thanks for sticking with it Cos. That's an annoying bug.
On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Ok, as I suspected there's a long standing (at least from 2006) bug in RPM
> that doesn't allow to validate RPM signature if a subkey has been used for
> signing.
>
> I ended up generating a new key pair (just for this purpose) and resigning
> all
> binaries with it; then resyncing everything with s3. I also have updated
> KEYS
> file with the new one. I have quickly ran a test on centos7 by installing
> bigtop-utils on an empty container and everything worked, including
> automatic
> import of the keys and the validation/installation of the package. Looks
> like
> we are in the clear.
>
> Please shout if you see otherwise. Thanks everyone for your patience!
> Cos
>
> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > I think there's a difference between how you've signed the pkgs and how
> I did
> > it. I signed with sub-key (as I mentioned before) and yum doesn't
> recognize
> > it. Seemingly, it expects that the master key was used for signing.
> >
> > Also, in your repo file below
> > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > points to the old keys. The location should be
> > gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> >
> > I am pretty sure I have exported my key with --armor option back in the
> day.
> > But I will repeat it and see if I can fix the situation, which I also
> observer
> > following your steps. If that's the only issue I will update the KEYS
> and we
> > should be completed by tonight ;)
> >
> > Thanks for your help!
> > Cos
> >
> > On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > This is the same issue we're trying to solve in the mailing thread
> > > "convenience artifacts are signed and uploaded". I've built a sample
> repo
> > > which works properly by using my own key "Evans Ye" to sign and to
> export
> > > GPG KEY. So I believe the following steps should be the right way to
> sign
> > > packages and export the gpgkey:
> > >
> > > $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
> > >
> > > $ gpg --armor --output KEYS --export 'Evans Ye'
> > > I've verified that the hash is matched now in our official repo.
> > > So I guess the main issue left is using non-armored gpg key, if we
> manually
> > > import the gpgkey in the repo file:
> > >
> > > [bigtop]
> > > name=Bigtop
> > > enabled=1
> > > gpgcheck=1
> > > type=NONE
> > > baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > >
> > > [root@48723d98dc1b ~]# rpm --import
> > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2
> not an
> > > armored public key.
> > >
> > > It gets error.
> > > However, my own exported armored key can be imported without an error.
> > > That's the different.
> > >
> > > Can you confirm that the gpgkey(
> http://archive.apache.org/dist/bigtop/KEYS)
> > > is exported with --armor flag?
> > >
> > > 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > >
> > > > Looks like I have figured out what's wrong with my key. And it is
> > > > _nothing_.
> > > > However, it seems that I can not sign RPMs with subkey as YUM can
> not find
> > > > the
> > > > key while importing. Can anyone confirm or disprove my train of
> thoughts?
> > > >
> > > > Thanks!
> > > > Cos
> > > >
> > > > On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > I've resynced the repodata once again and I don't see this issue
> on the
> > > > > centos7 anymore. However, yum still complains about the key being
> no
> > > > > available, but there's a workaround by setting gpgcheck=0 And I am
> going
> > > > to
> > > > > figure out what to do with it and why my key isn't working as
> expected.
> > > > >
> > > > > I also have discovered that the gpgkey file URL is using the old
> > > > incubation
> > > > > KEYS. Fixed that as well.
> > > > >
> > > > > Please let me know if you still see the issue with checksums
> mismatch.
> > > > > Thanks,
> > > > > Cos
> > > > >
> > > > > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > > I think this is the consequences of me fighting with the package
> > > > signing... ;(
> > > > > > A couple of days ago I have re-ran 'createrepo' for all the
> RPM-based
> > > > distros
> > > > > > and uploaded new repo files to the release. Not sure why the
> checksums
> > > > differ
> > > > > > now...
> > > > > >
> > > > > > I will take a look into this again tonight.
> > > > > > Cos
> > > > > >
> > > > > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > > I can second it:
> > > > > > >
> > > > > > > I added to /etc/yum.repo.d/meins.repo
> > > > > > >
> > > > > > > [meins]
> > > > > > > name=Bigtop epo
> > > > > > > baseurl=
> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > > enabled=1
> > > > > > > gpgcheck=0
> > > > > > > priority=1
> > > > > > >
> > > > > > > and got
> > > > > > > ............
> > > > > > > Downloading packages:
> > > > > > > hbase-0.98.12-1.el7.centos.noa FAILED
> > > > =============================================-] 849 kB/s
> | 62
> > > > MB 00:00:00 ETA
> > > > > > >
> > > >
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> :
> > > > [Errno -1] Package does not match intended download. Suggestion: run
> yum
> > > > --enablerepo=meins clean metadata
> > > > > > > Trying other mirror.
> > > > > > > .............
> > > > > > >
> > > > > > > Olaf
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: CentOS Package Checksums Don't Match RPMs
Posted by RJ Nowling <rn...@gmail.com>.
Way to go, Cos!
> On Sep 3, 2015, at 2:00 AM, Evans Ye <ev...@apache.org> wrote:
>
> You rock, Cos!
> I confirm that the centos 6 and fedora repo are also working properly now.
> Thanks for taking time fixing up the repos!
>
> 2015-09-03 12:40 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
>> Oh, the new key Id is 0xFA08B173 in case somebody needs to look it up. It
>> is
>> signed with my primary key that is the part of Apache WOT.
>>
>> Cos
>>
>>> On Wed, Sep 02, 2015 at 09:31PM, Konstantin Boudnik wrote:
>>> Ok, as I suspected there's a long standing (at least from 2006) bug in
>> RPM
>>> that doesn't allow to validate RPM signature if a subkey has been used
>> for
>>> signing.
>>>
>>> I ended up generating a new key pair (just for this purpose) and
>> resigning all
>>> binaries with it; then resyncing everything with s3. I also have updated
>> KEYS
>>> file with the new one. I have quickly ran a test on centos7 by installing
>>> bigtop-utils on an empty container and everything worked, including
>> automatic
>>> import of the keys and the validation/installation of the package. Looks
>> like
>>> we are in the clear.
>>>
>>> Please shout if you see otherwise. Thanks everyone for your patience!
>>> Cos
>>>
>>>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
>>>> I think there's a difference between how you've signed the pkgs and
>> how I did
>>>> it. I signed with sub-key (as I mentioned before) and yum doesn't
>> recognize
>>>> it. Seemingly, it expects that the master key was used for signing.
>>>>
>>>> Also, in your repo file below
>>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>>>> points to the old keys. The location should be
>>>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
>>>>
>>>> I am pretty sure I have exported my key with --armor option back in
>> the day.
>>>> But I will repeat it and see if I can fix the situation, which I also
>> observer
>>>> following your steps. If that's the only issue I will update the KEYS
>> and we
>>>> should be completed by tonight ;)
>>>>
>>>> Thanks for your help!
>>>> Cos
>>>>
>>>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
>>>>> This is the same issue we're trying to solve in the mailing thread
>>>>> "convenience artifacts are signed and uploaded". I've built a sample
>> repo
>>>>> which works properly by using my own key "Evans Ye" to sign and to
>> export
>>>>> GPG KEY. So I believe the following steps should be the right way to
>> sign
>>>>> packages and export the gpgkey:
>>>>>
>>>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
>> --addsign
>>>>>
>>>>> $ gpg --armor --output KEYS --export 'Evans Ye'
>>>>> I've verified that the hash is matched now in our official repo.
>>>>> So I guess the main issue left is using non-armored gpg key, if we
>> manually
>>>>> import the gpgkey in the repo file:
>>>>>
>>>>> [bigtop]
>>>>> name=Bigtop
>>>>> enabled=1
>>>>> gpgcheck=1
>>>>> type=NONE
>>>>> baseurl=
>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
>>>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>>>>>
>>>>> [root@48723d98dc1b ~]# rpm --import
>>>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS
>>>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key
>> 2 not an
>>>>> armored public key.
>>>>>
>>>>> It gets error.
>>>>> However, my own exported armored key can be imported without an
>> error.
>>>>> That's the different.
>>>>>
>>>>> Can you confirm that the gpgkey(
>> http://archive.apache.org/dist/bigtop/KEYS)
>>>>> is exported with --armor flag?
>>>>>
>>>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>>>>>
>>>>>> Looks like I have figured out what's wrong with my key. And it is
>>>>>> _nothing_.
>>>>>> However, it seems that I can not sign RPMs with subkey as YUM can
>> not find
>>>>>> the
>>>>>> key while importing. Can anyone confirm or disprove my train of
>> thoughts?
>>>>>>
>>>>>> Thanks!
>>>>>> Cos
>>>>>>
>>>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
>>>>>>> I've resynced the repodata once again and I don't see this issue
>> on the
>>>>>>> centos7 anymore. However, yum still complains about the key
>> being no
>>>>>>> available, but there's a workaround by setting gpgcheck=0 And I
>> am going
>>>>>> to
>>>>>>> figure out what to do with it and why my key isn't working as
>> expected.
>>>>>>>
>>>>>>> I also have discovered that the gpgkey file URL is using the old
>>>>>> incubation
>>>>>>> KEYS. Fixed that as well.
>>>>>>>
>>>>>>> Please let me know if you still see the issue with checksums
>> mismatch.
>>>>>>> Thanks,
>>>>>>> Cos
>>>>>>>
>>>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
>>>>>>>> I think this is the consequences of me fighting with the
>> package
>>>>>> signing... ;(
>>>>>>>> A couple of days ago I have re-ran 'createrepo' for all the
>> RPM-based
>>>>>> distros
>>>>>>>> and uploaded new repo files to the release. Not sure why the
>> checksums
>>>>>> differ
>>>>>>>> now...
>>>>>>>>
>>>>>>>> I will take a look into this again tonight.
>>>>>>>> Cos
>>>>>>>>
>>>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
>>>>>>>>> I can second it:
>>>>>>>>>
>>>>>>>>> I added to /etc/yum.repo.d/meins.repo
>>>>>>>>>
>>>>>>>>> [meins]
>>>>>>>>> name=Bigtop epo
>>>>>>>>> baseurl=
>>>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
>>>>>>>>> enabled=1
>>>>>>>>> gpgcheck=0
>>>>>>>>> priority=1
>>>>>>>>>
>>>>>>>>> and got
>>>>>>>>> ............
>>>>>>>>> Downloading packages:
>>>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED
>>>>>> =============================================-] 849 kB/s
>> | 62
>>>>>> MB 00:00:00 ETA
>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
>> :
>>>>>> [Errno -1] Package does not match intended download. Suggestion:
>> run yum
>>>>>> --enablerepo=meins clean metadata
>>>>>>>>> Trying other mirror.
>>>>>>>>> .............
>>>>>>>>>
>>>>>>>>> Olaf
>>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Evans Ye <ev...@apache.org>.
You rock, Cos!
I confirm that the centos 6 and fedora repo are also working properly now.
Thanks for taking time fixing up the repos!
2015-09-03 12:40 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> Oh, the new key Id is 0xFA08B173 in case somebody needs to look it up. It
> is
> signed with my primary key that is the part of Apache WOT.
>
> Cos
>
> On Wed, Sep 02, 2015 at 09:31PM, Konstantin Boudnik wrote:
> > Ok, as I suspected there's a long standing (at least from 2006) bug in
> RPM
> > that doesn't allow to validate RPM signature if a subkey has been used
> for
> > signing.
> >
> > I ended up generating a new key pair (just for this purpose) and
> resigning all
> > binaries with it; then resyncing everything with s3. I also have updated
> KEYS
> > file with the new one. I have quickly ran a test on centos7 by installing
> > bigtop-utils on an empty container and everything worked, including
> automatic
> > import of the keys and the validation/installation of the package. Looks
> like
> > we are in the clear.
> >
> > Please shout if you see otherwise. Thanks everyone for your patience!
> > Cos
> >
> > On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > > I think there's a difference between how you've signed the pkgs and
> how I did
> > > it. I signed with sub-key (as I mentioned before) and yum doesn't
> recognize
> > > it. Seemingly, it expects that the master key was used for signing.
> > >
> > > Also, in your repo file below
> > > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > points to the old keys. The location should be
> > > gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > >
> > > I am pretty sure I have exported my key with --armor option back in
> the day.
> > > But I will repeat it and see if I can fix the situation, which I also
> observer
> > > following your steps. If that's the only issue I will update the KEYS
> and we
> > > should be completed by tonight ;)
> > >
> > > Thanks for your help!
> > > Cos
> > >
> > > On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > > This is the same issue we're trying to solve in the mailing thread
> > > > "convenience artifacts are signed and uploaded". I've built a sample
> repo
> > > > which works properly by using my own key "Evans Ye" to sign and to
> export
> > > > GPG KEY. So I believe the following steps should be the right way to
> sign
> > > > packages and export the gpgkey:
> > > >
> > > > $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye"
> --addsign
> > > >
> > > > $ gpg --armor --output KEYS --export 'Evans Ye'
> > > > I've verified that the hash is matched now in our official repo.
> > > > So I guess the main issue left is using non-armored gpg key, if we
> manually
> > > > import the gpgkey in the repo file:
> > > >
> > > > [bigtop]
> > > > name=Bigtop
> > > > enabled=1
> > > > gpgcheck=1
> > > > type=NONE
> > > > baseurl=
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > > >
> > > > [root@48723d98dc1b ~]# rpm --import
> > > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > > error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key
> 2 not an
> > > > armored public key.
> > > >
> > > > It gets error.
> > > > However, my own exported armored key can be imported without an
> error.
> > > > That's the different.
> > > >
> > > > Can you confirm that the gpgkey(
> http://archive.apache.org/dist/bigtop/KEYS)
> > > > is exported with --armor flag?
> > > >
> > > > 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > > >
> > > > > Looks like I have figured out what's wrong with my key. And it is
> > > > > _nothing_.
> > > > > However, it seems that I can not sign RPMs with subkey as YUM can
> not find
> > > > > the
> > > > > key while importing. Can anyone confirm or disprove my train of
> thoughts?
> > > > >
> > > > > Thanks!
> > > > > Cos
> > > > >
> > > > > On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > > I've resynced the repodata once again and I don't see this issue
> on the
> > > > > > centos7 anymore. However, yum still complains about the key
> being no
> > > > > > available, but there's a workaround by setting gpgcheck=0 And I
> am going
> > > > > to
> > > > > > figure out what to do with it and why my key isn't working as
> expected.
> > > > > >
> > > > > > I also have discovered that the gpgkey file URL is using the old
> > > > > incubation
> > > > > > KEYS. Fixed that as well.
> > > > > >
> > > > > > Please let me know if you still see the issue with checksums
> mismatch.
> > > > > > Thanks,
> > > > > > Cos
> > > > > >
> > > > > > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > > > I think this is the consequences of me fighting with the
> package
> > > > > signing... ;(
> > > > > > > A couple of days ago I have re-ran 'createrepo' for all the
> RPM-based
> > > > > distros
> > > > > > > and uploaded new repo files to the release. Not sure why the
> checksums
> > > > > differ
> > > > > > > now...
> > > > > > >
> > > > > > > I will take a look into this again tonight.
> > > > > > > Cos
> > > > > > >
> > > > > > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > > > I can second it:
> > > > > > > >
> > > > > > > > I added to /etc/yum.repo.d/meins.repo
> > > > > > > >
> > > > > > > > [meins]
> > > > > > > > name=Bigtop epo
> > > > > > > > baseurl=
> > > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > > > enabled=1
> > > > > > > > gpgcheck=0
> > > > > > > > priority=1
> > > > > > > >
> > > > > > > > and got
> > > > > > > > ............
> > > > > > > > Downloading packages:
> > > > > > > > hbase-0.98.12-1.el7.centos.noa FAILED
> > > > > =============================================-] 849 kB/s
> | 62
> > > > > MB 00:00:00 ETA
> > > > > > > >
> > > > >
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm
> :
> > > > > [Errno -1] Package does not match intended download. Suggestion:
> run yum
> > > > > --enablerepo=meins clean metadata
> > > > > > > > Trying other mirror.
> > > > > > > > .............
> > > > > > > >
> > > > > > > > Olaf
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Oh, the new key Id is 0xFA08B173 in case somebody needs to look it up. It is
signed with my primary key that is the part of Apache WOT.
Cos
On Wed, Sep 02, 2015 at 09:31PM, Konstantin Boudnik wrote:
> Ok, as I suspected there's a long standing (at least from 2006) bug in RPM
> that doesn't allow to validate RPM signature if a subkey has been used for
> signing.
>
> I ended up generating a new key pair (just for this purpose) and resigning all
> binaries with it; then resyncing everything with s3. I also have updated KEYS
> file with the new one. I have quickly ran a test on centos7 by installing
> bigtop-utils on an empty container and everything worked, including automatic
> import of the keys and the validation/installation of the package. Looks like
> we are in the clear.
>
> Please shout if you see otherwise. Thanks everyone for your patience!
> Cos
>
> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> > I think there's a difference between how you've signed the pkgs and how I did
> > it. I signed with sub-key (as I mentioned before) and yum doesn't recognize
> > it. Seemingly, it expects that the master key was used for signing.
> >
> > Also, in your repo file below
> > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > points to the old keys. The location should be
> > gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
> >
> > I am pretty sure I have exported my key with --armor option back in the day.
> > But I will repeat it and see if I can fix the situation, which I also observer
> > following your steps. If that's the only issue I will update the KEYS and we
> > should be completed by tonight ;)
> >
> > Thanks for your help!
> > Cos
> >
> > On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > > This is the same issue we're trying to solve in the mailing thread
> > > "convenience artifacts are signed and uploaded". I've built a sample repo
> > > which works properly by using my own key "Evans Ye" to sign and to export
> > > GPG KEY. So I believe the following steps should be the right way to sign
> > > packages and export the gpgkey:
> > >
> > > $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
> > >
> > > $ gpg --armor --output KEYS --export 'Evans Ye'
> > > I've verified that the hash is matched now in our official repo.
> > > So I guess the main issue left is using non-armored gpg key, if we manually
> > > import the gpgkey in the repo file:
> > >
> > > [bigtop]
> > > name=Bigtop
> > > enabled=1
> > > gpgcheck=1
> > > type=NONE
> > > baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> > >
> > > [root@48723d98dc1b ~]# rpm --import
> > > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > > error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2 not an
> > > armored public key.
> > >
> > > It gets error.
> > > However, my own exported armored key can be imported without an error.
> > > That's the different.
> > >
> > > Can you confirm that the gpgkey(http://archive.apache.org/dist/bigtop/KEYS)
> > > is exported with --armor flag?
> > >
> > > 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> > >
> > > > Looks like I have figured out what's wrong with my key. And it is
> > > > _nothing_.
> > > > However, it seems that I can not sign RPMs with subkey as YUM can not find
> > > > the
> > > > key while importing. Can anyone confirm or disprove my train of thoughts?
> > > >
> > > > Thanks!
> > > > Cos
> > > >
> > > > On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > > I've resynced the repodata once again and I don't see this issue on the
> > > > > centos7 anymore. However, yum still complains about the key being no
> > > > > available, but there's a workaround by setting gpgcheck=0 And I am going
> > > > to
> > > > > figure out what to do with it and why my key isn't working as expected.
> > > > >
> > > > > I also have discovered that the gpgkey file URL is using the old
> > > > incubation
> > > > > KEYS. Fixed that as well.
> > > > >
> > > > > Please let me know if you still see the issue with checksums mismatch.
> > > > > Thanks,
> > > > > Cos
> > > > >
> > > > > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > > I think this is the consequences of me fighting with the package
> > > > signing... ;(
> > > > > > A couple of days ago I have re-ran 'createrepo' for all the RPM-based
> > > > distros
> > > > > > and uploaded new repo files to the release. Not sure why the checksums
> > > > differ
> > > > > > now...
> > > > > >
> > > > > > I will take a look into this again tonight.
> > > > > > Cos
> > > > > >
> > > > > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > > I can second it:
> > > > > > >
> > > > > > > I added to /etc/yum.repo.d/meins.repo
> > > > > > >
> > > > > > > [meins]
> > > > > > > name=Bigtop epo
> > > > > > > baseurl=
> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > > enabled=1
> > > > > > > gpgcheck=0
> > > > > > > priority=1
> > > > > > >
> > > > > > > and got
> > > > > > > ............
> > > > > > > Downloading packages:
> > > > > > > hbase-0.98.12-1.el7.centos.noa FAILED
> > > > =============================================-] 849 kB/s | 62
> > > > MB 00:00:00 ETA
> > > > > > >
> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm:
> > > > [Errno -1] Package does not match intended download. Suggestion: run yum
> > > > --enablerepo=meins clean metadata
> > > > > > > Trying other mirror.
> > > > > > > .............
> > > > > > >
> > > > > > > Olaf
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Ok, as I suspected there's a long standing (at least from 2006) bug in RPM
that doesn't allow to validate RPM signature if a subkey has been used for
signing.
I ended up generating a new key pair (just for this purpose) and resigning all
binaries with it; then resyncing everything with s3. I also have updated KEYS
file with the new one. I have quickly ran a test on centos7 by installing
bigtop-utils on an empty container and everything worked, including automatic
import of the keys and the validation/installation of the package. Looks like
we are in the clear.
Please shout if you see otherwise. Thanks everyone for your patience!
Cos
On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote:
> I think there's a difference between how you've signed the pkgs and how I did
> it. I signed with sub-key (as I mentioned before) and yum doesn't recognize
> it. Seemingly, it expects that the master key was used for signing.
>
> Also, in your repo file below
> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> points to the old keys. The location should be
> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
>
> I am pretty sure I have exported my key with --armor option back in the day.
> But I will repeat it and see if I can fix the situation, which I also observer
> following your steps. If that's the only issue I will update the KEYS and we
> should be completed by tonight ;)
>
> Thanks for your help!
> Cos
>
> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> > This is the same issue we're trying to solve in the mailing thread
> > "convenience artifacts are signed and uploaded". I've built a sample repo
> > which works properly by using my own key "Evans Ye" to sign and to export
> > GPG KEY. So I believe the following steps should be the right way to sign
> > packages and export the gpgkey:
> >
> > $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
> >
> > $ gpg --armor --output KEYS --export 'Evans Ye'
> > I've verified that the hash is matched now in our official repo.
> > So I guess the main issue left is using non-armored gpg key, if we manually
> > import the gpgkey in the repo file:
> >
> > [bigtop]
> > name=Bigtop
> > enabled=1
> > gpgcheck=1
> > type=NONE
> > baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> > gpgkey=http://archive.apache.org/dist/bigtop/KEYS
> >
> > [root@48723d98dc1b ~]# rpm --import
> > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> > error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2 not an
> > armored public key.
> >
> > It gets error.
> > However, my own exported armored key can be imported without an error.
> > That's the different.
> >
> > Can you confirm that the gpgkey(http://archive.apache.org/dist/bigtop/KEYS)
> > is exported with --armor flag?
> >
> > 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> >
> > > Looks like I have figured out what's wrong with my key. And it is
> > > _nothing_.
> > > However, it seems that I can not sign RPMs with subkey as YUM can not find
> > > the
> > > key while importing. Can anyone confirm or disprove my train of thoughts?
> > >
> > > Thanks!
> > > Cos
> > >
> > > On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > > I've resynced the repodata once again and I don't see this issue on the
> > > > centos7 anymore. However, yum still complains about the key being no
> > > > available, but there's a workaround by setting gpgcheck=0 And I am going
> > > to
> > > > figure out what to do with it and why my key isn't working as expected.
> > > >
> > > > I also have discovered that the gpgkey file URL is using the old
> > > incubation
> > > > KEYS. Fixed that as well.
> > > >
> > > > Please let me know if you still see the issue with checksums mismatch.
> > > > Thanks,
> > > > Cos
> > > >
> > > > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > > I think this is the consequences of me fighting with the package
> > > signing... ;(
> > > > > A couple of days ago I have re-ran 'createrepo' for all the RPM-based
> > > distros
> > > > > and uploaded new repo files to the release. Not sure why the checksums
> > > differ
> > > > > now...
> > > > >
> > > > > I will take a look into this again tonight.
> > > > > Cos
> > > > >
> > > > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > > I can second it:
> > > > > >
> > > > > > I added to /etc/yum.repo.d/meins.repo
> > > > > >
> > > > > > [meins]
> > > > > > name=Bigtop epo
> > > > > > baseurl=
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > > enabled=1
> > > > > > gpgcheck=0
> > > > > > priority=1
> > > > > >
> > > > > > and got
> > > > > > ............
> > > > > > Downloading packages:
> > > > > > hbase-0.98.12-1.el7.centos.noa FAILED
> > > =============================================-] 849 kB/s | 62
> > > MB 00:00:00 ETA
> > > > > >
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm:
> > > [Errno -1] Package does not match intended download. Suggestion: run yum
> > > --enablerepo=meins clean metadata
> > > > > > Trying other mirror.
> > > > > > .............
> > > > > >
> > > > > > Olaf
> > > > > >
> > > > >
> > > > >
> > >
> > >
> > >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
I think there's a difference between how you've signed the pkgs and how I did
it. I signed with sub-key (as I mentioned before) and yum doesn't recognize
it. Seemingly, it expects that the master key was used for signing.
Also, in your repo file below
gpgkey=http://archive.apache.org/dist/bigtop/KEYS
points to the old keys. The location should be
gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
I am pretty sure I have exported my key with --armor option back in the day.
But I will repeat it and see if I can fix the situation, which I also observer
following your steps. If that's the only issue I will update the KEYS and we
should be completed by tonight ;)
Thanks for your help!
Cos
On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote:
> This is the same issue we're trying to solve in the mailing thread
> "convenience artifacts are signed and uploaded". I've built a sample repo
> which works properly by using my own key "Evans Ye" to sign and to export
> GPG KEY. So I believe the following steps should be the right way to sign
> packages and export the gpgkey:
>
> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
>
> $ gpg --armor --output KEYS --export 'Evans Ye'
> I've verified that the hash is matched now in our official repo.
> So I guess the main issue left is using non-armored gpg key, if we manually
> import the gpgkey in the repo file:
>
> [bigtop]
> name=Bigtop
> enabled=1
> gpgcheck=1
> type=NONE
> baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
> gpgkey=http://archive.apache.org/dist/bigtop/KEYS
>
> [root@48723d98dc1b ~]# rpm --import
> https://dist.apache.org/repos/dist/release/bigtop/KEYS
> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2 not an
> armored public key.
>
> It gets error.
> However, my own exported armored key can be imported without an error.
> That's the different.
>
> Can you confirm that the gpgkey(http://archive.apache.org/dist/bigtop/KEYS)
> is exported with --armor flag?
>
> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
> > Looks like I have figured out what's wrong with my key. And it is
> > _nothing_.
> > However, it seems that I can not sign RPMs with subkey as YUM can not find
> > the
> > key while importing. Can anyone confirm or disprove my train of thoughts?
> >
> > Thanks!
> > Cos
> >
> > On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > > I've resynced the repodata once again and I don't see this issue on the
> > > centos7 anymore. However, yum still complains about the key being no
> > > available, but there's a workaround by setting gpgcheck=0 And I am going
> > to
> > > figure out what to do with it and why my key isn't working as expected.
> > >
> > > I also have discovered that the gpgkey file URL is using the old
> > incubation
> > > KEYS. Fixed that as well.
> > >
> > > Please let me know if you still see the issue with checksums mismatch.
> > > Thanks,
> > > Cos
> > >
> > > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > > I think this is the consequences of me fighting with the package
> > signing... ;(
> > > > A couple of days ago I have re-ran 'createrepo' for all the RPM-based
> > distros
> > > > and uploaded new repo files to the release. Not sure why the checksums
> > differ
> > > > now...
> > > >
> > > > I will take a look into this again tonight.
> > > > Cos
> > > >
> > > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > > I can second it:
> > > > >
> > > > > I added to /etc/yum.repo.d/meins.repo
> > > > >
> > > > > [meins]
> > > > > name=Bigtop epo
> > > > > baseurl=
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > > enabled=1
> > > > > gpgcheck=0
> > > > > priority=1
> > > > >
> > > > > and got
> > > > > ............
> > > > > Downloading packages:
> > > > > hbase-0.98.12-1.el7.centos.noa FAILED
> > =============================================-] 849 kB/s | 62
> > MB 00:00:00 ETA
> > > > >
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm:
> > [Errno -1] Package does not match intended download. Suggestion: run yum
> > --enablerepo=meins clean metadata
> > > > > Trying other mirror.
> > > > > .............
> > > > >
> > > > > Olaf
> > > > >
> > > >
> > > >
> >
> >
> >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Evans Ye <ev...@apache.org>.
This is the same issue we're trying to solve in the mailing thread
"convenience artifacts are signed and uploaded". I've built a sample repo
which works properly by using my own key "Evans Ye" to sign and to export
GPG KEY. So I believe the following steps should be the right way to sign
packages and export the gpgkey:
$ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" --addsign
$ gpg --armor --output KEYS --export 'Evans Ye'
I've verified that the hash is matched now in our official repo.
So I guess the main issue left is using non-armored gpg key, if we manually
import the gpgkey in the repo file:
[bigtop]
name=Bigtop
enabled=1
gpgcheck=1
type=NONE
baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64
gpgkey=http://archive.apache.org/dist/bigtop/KEYS
[root@48723d98dc1b ~]# rpm --import
https://dist.apache.org/repos/dist/release/bigtop/KEYS
error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key 2 not an
armored public key.
It gets error.
However, my own exported armored key can be imported without an error.
That's the different.
Can you confirm that the gpgkey(http://archive.apache.org/dist/bigtop/KEYS)
is exported with --armor flag?
2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> Looks like I have figured out what's wrong with my key. And it is
> _nothing_.
> However, it seems that I can not sign RPMs with subkey as YUM can not find
> the
> key while importing. Can anyone confirm or disprove my train of thoughts?
>
> Thanks!
> Cos
>
> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> > I've resynced the repodata once again and I don't see this issue on the
> > centos7 anymore. However, yum still complains about the key being no
> > available, but there's a workaround by setting gpgcheck=0 And I am going
> to
> > figure out what to do with it and why my key isn't working as expected.
> >
> > I also have discovered that the gpgkey file URL is using the old
> incubation
> > KEYS. Fixed that as well.
> >
> > Please let me know if you still see the issue with checksums mismatch.
> > Thanks,
> > Cos
> >
> > On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > > I think this is the consequences of me fighting with the package
> signing... ;(
> > > A couple of days ago I have re-ran 'createrepo' for all the RPM-based
> distros
> > > and uploaded new repo files to the release. Not sure why the checksums
> differ
> > > now...
> > >
> > > I will take a look into this again tonight.
> > > Cos
> > >
> > > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > > I can second it:
> > > >
> > > > I added to /etc/yum.repo.d/meins.repo
> > > >
> > > > [meins]
> > > > name=Bigtop epo
> > > > baseurl=
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > > enabled=1
> > > > gpgcheck=0
> > > > priority=1
> > > >
> > > > and got
> > > > ............
> > > > Downloading packages:
> > > > hbase-0.98.12-1.el7.centos.noa FAILED
> =============================================-] 849 kB/s | 62
> MB 00:00:00 ETA
> > > >
> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm:
> [Errno -1] Package does not match intended download. Suggestion: run yum
> --enablerepo=meins clean metadata
> > > > Trying other mirror.
> > > > .............
> > > >
> > > > Olaf
> > > >
> > >
> > >
>
>
>
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
Looks like I have figured out what's wrong with my key. And it is _nothing_.
However, it seems that I can not sign RPMs with subkey as YUM can not find the
key while importing. Can anyone confirm or disprove my train of thoughts?
Thanks!
Cos
On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote:
> I've resynced the repodata once again and I don't see this issue on the
> centos7 anymore. However, yum still complains about the key being no
> available, but there's a workaround by setting gpgcheck=0 And I am going to
> figure out what to do with it and why my key isn't working as expected.
>
> I also have discovered that the gpgkey file URL is using the old incubation
> KEYS. Fixed that as well.
>
> Please let me know if you still see the issue with checksums mismatch.
> Thanks,
> Cos
>
> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> > I think this is the consequences of me fighting with the package signing... ;(
> > A couple of days ago I have re-ran 'createrepo' for all the RPM-based distros
> > and uploaded new repo files to the release. Not sure why the checksums differ
> > now...
> >
> > I will take a look into this again tonight.
> > Cos
> >
> > On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > > I can second it:
> > >
> > > I added to /etc/yum.repo.d/meins.repo
> > >
> > > [meins]
> > > name=Bigtop epo
> > > baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > > enabled=1
> > > gpgcheck=0
> > > priority=1
> > >
> > > and got
> > > ............
> > > Downloading packages:
> > > hbase-0.98.12-1.el7.centos.noa FAILED =============================================-] 849 kB/s | 62 MB 00:00:00 ETA
> > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=meins clean metadata
> > > Trying other mirror.
> > > .............
> > >
> > > Olaf
> > >
> >
> >
Re: CentOS Package Checksums Don't Match RPMs
Posted by Konstantin Boudnik <co...@apache.org>.
I've resynced the repodata once again and I don't see this issue on the
centos7 anymore. However, yum still complains about the key being no
available, but there's a workaround by setting gpgcheck=0 And I am going to
figure out what to do with it and why my key isn't working as expected.
I also have discovered that the gpgkey file URL is using the old incubation
KEYS. Fixed that as well.
Please let me know if you still see the issue with checksums mismatch.
Thanks,
Cos
On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote:
> I think this is the consequences of me fighting with the package signing... ;(
> A couple of days ago I have re-ran 'createrepo' for all the RPM-based distros
> and uploaded new repo files to the release. Not sure why the checksums differ
> now...
>
> I will take a look into this again tonight.
> Cos
>
> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote:
> > I can second it:
> >
> > I added to /etc/yum.repo.d/meins.repo
> >
> > [meins]
> > name=Bigtop epo
> > baseurl=http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/
> > enabled=1
> > gpgcheck=0
> > priority=1
> >
> > and got
> > ............
> > Downloading packages:
> > hbase-0.98.12-1.el7.centos.noa FAILED =============================================-] 849 kB/s | 62 MB 00:00:00 ETA
> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=meins clean metadata
> > Trying other mirror.
> > .............
> >
> > Olaf
> >
>
>