You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/05 12:01:48 UTC
[GitHub] [airflow] noelmcloughlin opened a new issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
noelmcloughlin opened a new issue #12832:
URL: https://github.com/apache/airflow/issues/12832
**Description**
The sha256sum file for apache-airflow releases is in an unexpected format for python-based checksum modules.
**Current file format:**
apache-airflow-1.10.13rc1-bin.tar.gz: 36D641C0 F2AAEC4E BCE91BD2 66CE2BC6
AA2D995C 08C9B62A 0EA1CBEC 027E657B
8AF4B54E 6C3AD117 9634198D F6EA53F8
163711BA 95586B5B 7BCF7F4B 098A19E2
**Wanted formats**
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx amd64\apache-airflow-1.10.13-bin.tar.gz
**Or**
`xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
**Use case / motivation**
Ansible and salt python libraries to consume checksums do not understand the format...
```
ID: airflow-archive-install
Function: archive.extracted
Name: /opt/apache-airflow-1.10.13/bin/
Result: False
Comment: Attempt 1: Returned a result of "False", with the following comment: "Source hash https://github.com/apache/airflow/releases/download/1.10.13/ap
ache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid. The supported formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) eithe
r a path to a local file containing hashes, or a URI of a remote hash file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift
, s3. The hash may also not be of a valid length, the following are supported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (9
6), sha512 (128)."
Attempt 2: Returned a result of "False", with the following comment: "Source hash https://github.com/apache/airflow/releases/download/1.10.13/ap
ache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid. The supported formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) eithe
r a path to a local file containing hashes, or a URI of a remote hash file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift
, s3. The hash may also not be of a valid length, the following are supported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (9
6), sha512 (128)."
Source hash https://github.com/apache/airflow/releases/download/1.10.13/apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid. The supporte
d formats are: 1) a hash, 2) an expression in the format <hash_type>=<hash>, or 3) either a path to a local file containing hashes, or a URI of a remote hash
file. Supported protocols for remote hash files are: salt, file, http, https, ftp, swift, s3. The hash may also not be of a valid length, the following are su
pported hash types and lengths: md5 (32), sha1 (40), sha224 (56), sha256 (64), sha384 (96), sha512 (128).
Started: 11:39:44.082079
Duration: 123506.098 ms
```
**Related Issues**
No
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739502352
The file is valid, but not recognized by ansible. We are currently using GPG to generate the SHA files:
https://github.com/apache/airflow/blob/82dfa5f74bfa39bedf127763dde4d08678202445/dev/sign.sh#L33
And likewise we are using GPG to verify them:
https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#sha512-sum-check
```
echo "Checking $i"; gpg --print-md SHA512 `basename $i .sha512 ` | diff - $i
```
@noelmcloughlin I think as a workaround, you could use `skip_verify: true' and add a simple script verifying the signature in a similar way https://stackoverflow.com/questions/45287269/check-authenticity-of-file-in-ansible/45303473
@kaxil -> the format of the SHA files is indeed non-standard, so why don't we change it for 2.0? It's rather straightforward WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739439100
Maybe you can use Hash from PyPI:
https://pypi.org/project/apache-airflow/1.10.13/#copy-hash-modal-7de14478-7b22-42f1-b870-f0aa8bf1b3ce
```
bb637f95a2aef7b2f7d622ecda150d6a5794011bd9e8d610ab0e9a1f518325e9
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739502352
The file is valid, but not recognized by ansible. We are currently using GPG to generate the SHA files:
```
gpg --print-md SHA512 "${name}" > "${name}.sha512"` (See gpg --print-md SHA512 "${name}" > "${name}.sha512`
```
And likewise we are using GPG to verify them:
https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#sha512-sum-check
```
echo "Checking $i"; gpg --print-md SHA512 `basename $i .sha512 ` | diff - $i
```
@noelmcloughlin I think as a workaround, you could use `skip_verify: true' and add a simple script verifying the signature in a similar way https://stackoverflow.com/questions/45287269/check-authenticity-of-file-in-ansible/45303473
@kaxil -> the format of the SHA files is indeed non-standard, so why don't we change it for 2.0? It's rather straightforward WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739502352
The file is valid, but not recognized by ansible. We are currently using GPG to generate the SHA files:
https://github.com/apache/airflow/blob/82dfa5f74bfa39bedf127763dde4d08678202445/dev/sign.sh#L33
```
gpg --print-md SHA512 "${name}" > "${name}.sha512"
```
And likewise we are using GPG to verify them:
https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#sha512-sum-check
```
echo "Checking $i"; gpg --print-md SHA512 `basename $i .sha512 ` | diff - $i
```
@noelmcloughlin I think as a workaround, you could use `skip_verify: true' and add a simple script verifying the signature in a similar way https://stackoverflow.com/questions/45287269/check-authenticity-of-file-in-ansible/45303473
@kaxil -> the format of the SHA files is indeed non-standard, so why don't we change it for 2.0? It's rather straightforward WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739242039
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #12832:
URL: https://github.com/apache/airflow/issues/12832
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739502352
The file is valid, but not recognized by ansible. We are currently using GPG to generate the SHA files:
```
gpg --print-md SHA512 "${name}" > "${name}.sha512"` (See gpg --print-md SHA512 "${name}" > "${name}.sha512`
``
And likewise we are using GPG to verify them:
https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#sha512-sum-check
```
echo "Checking $i"; gpg --print-md SHA512 `basename $i .sha512 ` | diff - $i
```
@noelmcloughlin I think as a workaround, you could use `skip_verify: true' and add a simple script verifying the signature in a similar way https://stackoverflow.com/questions/45287269/check-authenticity-of-file-in-ansible/45303473
@kaxil -> the format of the SHA files is indeed non-standard, so why don't we change it for 2.0? It's rather straightforward WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739577148
Created a PR for it: https://github.com/apache/airflow/pull/12867
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] noelmcloughlin commented on issue #12832: Source hash apache-airflow-1.10.13-bin.tar.gz.sha512 format is invalid
Posted by GitBox <gi...@apache.org>.
noelmcloughlin commented on issue #12832:
URL: https://github.com/apache/airflow/issues/12832#issuecomment-739441830
The only workaround to install Airflow tarball with Salt is setting `skip_verify: true` to bypass the file format issue. I'm not sure what PyPI library they use in Salt, but it works for 99% of `sha256` files consumed from github.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org