You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2020/07/29 17:55:00 UTC
[GitHub] [incubator-heron] windhamwong opened a new pull request #3595: Switch to pigz to support multi-core tar command
windhamwong opened a new pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595
pigz is a multi-core version of tar, which makes the docker build process much faster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] Code0x58 commented on pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
Code0x58 commented on pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595#issuecomment-666023113
A couple of tests to show equivalence for consumers:
With gzip:
```
$ cat txt
oink
$ ls
txt
$ pigz txt
$ ls
txt.gz
$ gunzip txt.gz
$ ls
txt
$ cat txt
oink
```
With tar:
```
$ tar --remove-files --use-compress-program=pigz -cf out.tar.gz txt
$ ls
out.tar.gz
$ tar -zxf out.tar.gz
$ cat txt
oink
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis merged pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
nicknezis merged pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595#issuecomment-665910932
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] windhamwong commented on pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
windhamwong commented on pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595#issuecomment-666062598
https://zlib.net/pigz/
It is gzip format but with the parallel multi-core feature. The traditional gzip command uses only one core but modern CPU/vCPU has more than one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] windhamwong commented on pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
windhamwong commented on pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595#issuecomment-667061592
Docker build testing result:
ubuntu18.04 - success
ubuntu20.04 - success
ubuntu16.04 - success
debian9 - fail - python3.6 not supported (unrelated to pigz)
debian10 - fail - protobuf issue (unrelated to pigz)
centos7 - success
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] windhamwong commented on pull request #3595: Switch to pigz to support multi-core tar command
Posted by GitBox <gi...@apache.org>.
windhamwong commented on pull request #3595:
URL: https://github.com/apache/incubator-heron/pull/3595#issuecomment-666502369
let me try out all docker builds tomorrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org