You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "andersk (via GitHub)" <gi...@apache.org> on 2024/05/15 20:39:41 UTC
[I] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
andersk opened a new issue, #41678:
URL: https://github.com/apache/arrow/issues/41678
### Describe the bug, including details regarding any error messages, version, and platform.
The [PGP signature](https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-apt-source-latest-bullseye.deb.asc) on the newly uploaded version of [apache-arrow-apt-source-latest-bullseye.deb](https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-apt-source-latest-bullseye.deb) (and possibly others) fails verification, because it’s signed by a key that’s not in [apache-arrow-keyring.gpg](https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-keyring.gpg).
I would appreciate if this can be fixed by resigning the .deb with a key that’s already in the keyring, rather than adding new keys to the keyring, because we need to cut a new release of the [Zulip server](https://github.com/zulip/zulip) every time we need to change its [included copy of the keyring](https://github.com/zulip/zulip/blob/8.x/scripts/setup/apt-repos/zulip/apache-arrow-keyring.gpg). (Downloading a new copies of the keyring every time it changes defeats the whole point of verifying PGP signatures, because an attacker who can tamper with the binary can also tamper with the keyring.)
```console
$ wget https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-apt-source-latest-bullseye.deb
$ wget https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-apt-source-latest-bullseye.deb.asc
$ wget https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-keyring.gpg
$ gpgv --keyring=./apache-arrow-keyring.gpg apache-arrow-apt-source-latest-bullseye.deb.asc apache-arrow-apt-source-latest-bullseye.deb
gpgv: Signature made Thu 09 May 2024 03:56:53 AM PDT
gpgv: using RSA key AF6AADA4C9835B75973FF5DA275C532289DD0F4A
gpgv: Can't check signature: No public key
$ gpg --no-default-keyring --keyring=./apache-arrow-keyring.gpg --list-keys
./apache-arrow-keyring.gpg
--------------------------
pub rsa2048 2013-04-10 [SCEA] [expired: 2017-04-10]
35CF82A165DDBBA29B307B7497D7E8647AE7E47B
uid [ expired] Julien Le Dem <ju...@ledem.net>
pub rsa4096 2016-09-19 [SC] [expired: 2020-09-19]
87C072B8B6405B5780D66A3D02DABFDF1679D194
uid [ expired] Julien Le Dem <ju...@ledem.net>
pub rsa4096 2016-11-06 [SC]
F2A765669021A3D3094C200B29D94E228CAAD602
uid [ unknown] Uwe L. Korn <uw...@apache.org>
sub rsa4096 2016-11-06 [E]
pub rsa4096 2017-05-01 [SC]
6D09E881160096717426C638F105883A1735623D
uid [ unknown] William Wesley McKinney (CODE SIGNING KEY) <we...@apache.org>
sub rsa4096 2017-05-01 [E]
pub rsa4096 2010-08-30 [SC]
08D3564B7C6A9CAFBFF6A66791D18FCF079F8007
uid [ unknown] Kouhei Sutou <ko...@cozmixng.org>
uid [ unknown] Kouhei Sutou <ko...@clear-code.com>
sub rsa4096 2010-08-30 [E]
pub rsa4096 2018-06-13 [SC]
E6E4AA55F38337A6EFC7A5549F453D0CC3E4F6BA
uid [ unknown] Charles Phillip Cloud <cp...@gmail.com>
sub rsa4096 2018-06-13 [E]
pub rsa4096 2014-09-04 [SCEA]
2CC23EF090A28BAC5DDC885F769BDC2129C2658C
uid [ unknown] keybase.io/pitrou <pi...@keybase.io>
pub rsa4096 2019-01-10 [SC]
265F80AB84FE03127E14F01125BCCA5220D84079
uid [ unknown] Krisztian Szucs (apache) <sz...@gmail.com>
sub rsa4096 2019-01-10 [E]
pub rsa4096 2019-09-24 [SC] [expired: 2020-09-23]
E47C810A90FE21FF448DA938755E743692EA1D85
uid [ expired] Micah Kornfield (Apache Arrow Code Signing Key) <em...@gmail.com>
```
### Component(s)
Packaging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Release][Packaging] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "thisisnic (via GitHub) via github" <gi...@arrow.apache.org>.
thisisnic commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-6871669850
@kou is this issue still valid given all the changes since it was last active?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2113701840
Supply chain attacks are of course a valid concern but we can't avoid changes to the KEYS file, Arrow is a large project and individuals sign our releases, sometimes someone new takes on the responsibility (or new sub-projects are added) so changes are needed.
For apache/arrow releases there was one addition made in the last 2+ years and we added the key 6+ months before first using it to allow for the change to propagate. So it can't really be said that the file changes frequently. (there were a small number (2-3?) sub project additions unrelated to release singing for apache/arrow).
I think something that could make sense in the future is to announce changes to the KEYS file on the user mailing list with a hash of the new file, signed with an existing key from the file. This would provide a second channel to confirm the validity of any changes to the file. What do you think @kou?
In addition there was a change to the ASF release policy that now allows automatic release signing with a key not bound to an individual. This would further lower the frequency of changes to KEYS likely towards 0. But our releases process is quite involved and there are strict requirements for automatic signing to be implemented so this is not a change we can promise for the near future.
When we implement automatic release signing it might also be possible to additionally make use of https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/ to add another source of trust but I haven't looked into the requirements or uses cases for this so far.
If you have ideas (other than never changing KEYS :wink:) on how to improve this for downstream projects/users, contributions are of course always appreciated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Release][Packaging] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "andersk (via GitHub) via github" <gi...@arrow.apache.org>.
andersk commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-6873083298
This is still valid. The specific signature from the original report no longer matters, but the underlying problem hasn’t changed. Many current artifacts are signed by keys that aren’t in https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-keyring.gpg, which still has the same 9 keys as in 2024.
- `apache-arrow-apt-source-latest-noble.deb.asc` (26.0.1, 2026-10-30) and the `noble`/`trixie` `InRelease` files are signed by `7BD1A9A50E4F701BCBFD90EF9E922B2D60E9FD1C`, a key created 2026-02-11.
- `bookworm/InRelease` (2026-07-24) is signed by `BE7EF45DBAD38E4EECED390E9CBA4EF977CA20B8`.
All of these verify against the current `KEYS`, but `KEYS` has grown from 16 to 25 keys since this issue was opened. So a keyring fetched at any point in the past still eventually stops working, and there is no secure way to fetch a new keyring without being exposed to the very threats that the keyring is supposed to defend against (server compromise, SSL vulnerability, etc.).
Please create a single stable key whose role is to sign the Arrow artifacts, not to represent the personal identity one of many rotating developers unverifiably associated with the project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "nikmolnar (via GitHub)" <gi...@apache.org>.
nikmolnar commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2116251261
This is also affecting `apache-arrow-apt-source-latest-jammy.deb`, and updating keys from https://downloads.apache.org/arrow/KEYS doesn't fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "andersk (via GitHub)" <gi...@apache.org>.
andersk commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2113519633
That doesn’t solve the issue. There are 16 distinct keys in that new keyring file, 5 of which are expired, so presumably it’s changing quite frequently. Like I said, downloading a new copies of the keyring every time it changes defeats the whole point of verifying PGP signatures, because an attacker who can tamper with the binary can also tamper with the keyring.
Zulip needs a stable root of trust for our external dependencies that can be baked into our release tarball. Ideally we would like it to be the one that’s already baked into our existing release tarball, so that our existing release tarball isn’t broken.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2113496926
Ah, https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-keyring.gpg is deprecated by `apache-arrow-apt-source-latest*.deb` and it's not updated.
You need to use https://dist.apache.org/repos/dist/release/arrow/KEYS instead:
```bash
curl https://downloads.apache.org/arrow/KEYS | gpg --no-default-keyring --keyring /tmp/apache-arrow-apt-source.gpg --import -
```
Similar issue: #40967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Release][Packaging] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2116687386
> I think something that could make sense in the future is to announce changes to the KEYS file on the user mailing list with a hash of the new file, signed with an existing key from the file. This would provide a second channel to confirm the validity of any changes to the file. What do you think @kou?
It makes sense. Let's do it next time. We will do it on our release announce too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Release][Packaging] apache-arrow-apt-source-latest-bullseye.deb.asc is signed by a key not in apache-arrow-keyring.gpg [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41678:
URL: https://github.com/apache/arrow/issues/41678#issuecomment-2116854924
We may not be able to use `apache-arrow-apt-source-lastest-XXX.deb{,.asc}` with old https://downloads.apache.org/arrow/KEYS because `apache-arrow-apt-source-lastest-XXX.deb.asc` is generated with the latest https://downloads.apache.org/arrow/KEYS . The latest `KEYS` may include more keys that aren't included in an old `KEYS`.
How about the following instructions with `KEYS` ( https://apache.jfrog.io/artifactory/arrow/debian/apache-arrow-keyring.gpg is based on `KEYS`) that was added by https://github.com/zulip/zulip/commit/5ccb408f194ea54c6137f650384d6daa3173ceec (2023-07-28)?
1. Download the followings:
* https://apache.jfrog.io/artifactory/arrow/debian/pool/bullseye/main/a/apache-arrow-apt-source/apache-arrow-apt-source_13.0.0-1_all.deb
* https://apache.jfrog.io/artifactory/arrow/debian/pool/bullseye/main/a/apache-arrow-apt-source/apache-arrow-apt-source_13.0.0-1_all.deb.asc
2. Verify the downloaded `.deb` with the added `KEYS`
3. Install the verified `.deb`
4. Download the followings:
* https://apache.jfrog.io/artifactory/arrow/debian/pool/bullseye/main/a/apache-arrow-apt-source/apache-arrow-apt-source_14.0.0-1_all.deb
* https://apache.jfrog.io/artifactory/arrow/debian/pool/bullseye/main/a/apache-arrow-apt-source/apache-arrow-apt-source_14.0.0-1_all.deb.asc
5. Verify the downloaded `.deb` with the `KEYS` included in the installed `apache-arrow-apt-source_13.0.0-1`
6. Install the verified `.deb`
7. Download `apache-arrow-apt-source_15.0.0-1_all.deb{,.asc}`
8. ...
FYI: We can detect available `apache-arrow-apt-source`s at https://apache.jfrog.io/ui/native/arrow/debian/pool/bullseye/main/a/apache-arrow-apt-source/ .
(We can update `apache-arrow-apt-source` by `sudo apt update && sudo apt upgrade` but if the old `KEYS` is too old, `sudo apt update && sudo apt upgrade` may not work.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org