You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/07/12 08:13:05 UTC
[GitHub] [pinot] gortiz opened a new pull request, #9044: Add a new workflow to check vulnerabilities using trivy
gortiz opened a new pull request, #9044:
URL: https://github.com/apache/pinot/pull/9044
This PR adds a new workflow that executes [Trivy](https://github.com/aquasecurity/trivy) in order to look for vulnerabilities. As I'm not a Pinot Committer, I'm not sure if this PR is going to work.
Trivy is an open source program with a huge vulnerability db that analyzes artifacts looking for vulnerabilities. Although Trivy can be used in different ways, the most common way to use it is to analyze a docker image. By doing that it can analyze the code dependencies (for example, jars) but also SO dependencies (like the zlib version that is used). Trivy can also be used to analyze infrastructure as code and other configs, but I don´t have experience doing so.
When programs like this are added to a software, it is expected to find a lot of vulnerabilities. It doesn't make sense to block on going PRs due to vulnerabilities that are already present in master branch, so I have relaxed the workflow to do not fail. We should change that once we fix all the vulnerabilities
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on a diff in pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
gortiz commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r921853388
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
Review Comment:
That is a very good idea. We also have to add some extra files like the package.json files or dockerfiles, but it totally makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on pull request #9044: Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
gortiz commented on PR #9044:
URL: https://github.com/apache/pinot/pull/9044#issuecomment-1186823553
I think this is ready to merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on a diff in pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
gortiz commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r921847874
##########
.github/workflows/scripts/docker/.pinot_docker_image_build.sh:
##########
@@ -0,0 +1,61 @@
+#!/bin/bash -x
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+if [ -z "${DOCKER_IMAGE_NAME}" ]; then
+ DOCKER_IMAGE_NAME="apachepinot/pinot"
+fi
+if [ -z "${PINOT_GIT_URL}" ]; then
+ PINOT_GIT_URL="https://github.com/apache/pinot.git"
+fi
+if [ -z "${PINOT_BRANCH}" ]; then
+ PINOT_BRANCH="master"
+fi
+if [ -z "${BUILD_PLATFORM}" ]; then
+ BUILD_PLATFORM="linux/arm64,linux/amd64"
+fi
+
+COMMIT_ID=`git rev-parse --short HEAD`
+DATE=`date +%Y%m%d`
+VERSION=`mvn help:evaluate -Dexpression=project.version -q -DforceStdout`
+tags=()
+if [ -z "${TAGS}" ]; then
+ tags=("${VERSION}-${COMMIT_ID}-${DATE}")
+ tags+=("latest")
+else
+ declare -a tags=($(echo ${TAGS} | tr "," " "))
+fi
+
+DOCKER_BUILD_TAGS=""
+for tag in "${tags[@]}"
+do
+ echo "Plan to build docker images for: ${DOCKER_IMAGE_NAME}:${tag}"
+ DOCKER_BUILD_TAGS+=" --tag ${DOCKER_IMAGE_NAME}:${tag} "
+done
Review Comment:
This has been copied from `.pinot_docker_image_build_and_push.sh`. In the workflow I always use a single tag (the commit sha)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 merged pull request #9044: Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
xiangfu0 merged PR #9044:
URL: https://github.com/apache/pinot/pull/9044
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9044: Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r923443744
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
+jobs:
+ verify-docker:
+ name: Verify Docker Image
+ runs-on: ubuntu-latest
+ steps:
+ - uses: docker/setup-qemu-action@v1
+ name: Set up QEMU
+ - uses: docker/setup-buildx-action@v1
+ name: Set up Docker Buildx
+ - uses: actions/checkout@v3
+ - name: Build the Docker image
+ env:
+ DOCKER_FILE_BASE_DIR: "docker/images/pinot"
+ DOCKER_IMAGE_NAME: "apachepinot/pinot"
+ BUILD_PLATFORM: "linux/amd64"
+ PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
+ PINOT_BRANCH: ${{ env.GITHUB_REF }}
+ TAGS: ${{ github.sha }}
+ run: .github/workflows/scripts/docker/.pinot_docker_image_build.sh
+
+ - name: Run Trivy vulnerability scanner (sarif)
+ uses: aquasecurity/trivy-action@master
+ with:
+ trivyignores: '.trivyignore'
+ image-ref: 'apachepinot/pinot:${{ github.sha }}'
+ format: 'sarif'
+ output: 'trivy-results.sarif'
+ vuln-type: 'os,library'
+ severity: 'CRITICAL,HIGH'
+ - name: Upload Trivy scan results to GitHub Security tab
+ uses: github/codeql-action/upload-sarif@v2
Review Comment:
i see. so if I understand correctly:
- the reason why this PR has so many vulns is b/c current master doesn't have a baseline vulnerability SARIF result.
- once merged, the first commit on master will have a large SARIF but the next one should be empty if it didn't modify any dependencies. correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on a diff in pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
gortiz commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r921847874
##########
.github/workflows/scripts/docker/.pinot_docker_image_build.sh:
##########
@@ -0,0 +1,61 @@
+#!/bin/bash -x
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+if [ -z "${DOCKER_IMAGE_NAME}" ]; then
+ DOCKER_IMAGE_NAME="apachepinot/pinot"
+fi
+if [ -z "${PINOT_GIT_URL}" ]; then
+ PINOT_GIT_URL="https://github.com/apache/pinot.git"
+fi
+if [ -z "${PINOT_BRANCH}" ]; then
+ PINOT_BRANCH="master"
+fi
+if [ -z "${BUILD_PLATFORM}" ]; then
+ BUILD_PLATFORM="linux/arm64,linux/amd64"
+fi
+
+COMMIT_ID=`git rev-parse --short HEAD`
+DATE=`date +%Y%m%d`
+VERSION=`mvn help:evaluate -Dexpression=project.version -q -DforceStdout`
+tags=()
+if [ -z "${TAGS}" ]; then
+ tags=("${VERSION}-${COMMIT_ID}-${DATE}")
+ tags+=("latest")
+else
+ declare -a tags=($(echo ${TAGS} | tr "," " "))
+fi
+
+DOCKER_BUILD_TAGS=""
+for tag in "${tags[@]}"
+do
+ echo "Plan to build docker images for: ${DOCKER_IMAGE_NAME}:${tag}"
+ DOCKER_BUILD_TAGS+=" --tag ${DOCKER_IMAGE_NAME}:${tag} "
+done
Review Comment:
This has been copied from `.pinot_docker_image_build_and_push.sh`. In the workflow we don't actually use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9044:
URL: https://github.com/apache/pinot/pull/9044#issuecomment-1183003523
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9044](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b73f4ca) into [master](https://codecov.io/gh/apache/pinot/commit/0c55422d8bb5ff0df3e7871c9b1a1e52be38aecc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0c55422) will **decrease** coverage by `26.14%`.
> The diff coverage is `43.83%`.
> :exclamation: Current head b73f4ca differs from pull request most recent head 684a681. Consider uploading reports for the commit 684a681 to get more accurate results
```diff
@@ Coverage Diff @@
## master #9044 +/- ##
=============================================
- Coverage 63.49% 37.35% -26.15%
+ Complexity 4714 193 -4521
=============================================
Files 1783 1831 +48
Lines 94134 96270 +2136
Branches 14140 14390 +250
=============================================
- Hits 59774 35964 -23810
- Misses 30102 57442 +27340
+ Partials 4258 2864 -1394
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `26.56% <23.28%> (?)` | |
| integration2 | `24.64% <21.91%> (?)` | |
| unittests1 | `?` | |
| unittests2 | `15.37% <19.17%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...re/common/evaluators/DefaultJsonPathEvaluator.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vZXZhbHVhdG9ycy9EZWZhdWx0SnNvblBhdGhFdmFsdWF0b3IuamF2YQ==) | `17.78% <0.00%> (-11.75%)` | :arrow_down: |
| [...e/operator/dociditerators/MVScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9NVlNjYW5Eb2NJZEl0ZXJhdG9yLmphdmE=) | `50.61% <0.00%> (-19.76%)` | :arrow_down: |
| [...e/operator/dociditerators/SVScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9TVlNjYW5Eb2NJZEl0ZXJhdG9yLmphdmE=) | `76.53% <0.00%> (+1.02%)` | :arrow_up: |
| [...local/indexsegment/mutable/MutableSegmentImpl.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9pbmRleHNlZ21lbnQvbXV0YWJsZS9NdXRhYmxlU2VnbWVudEltcGwuamF2YQ==) | `0.00% <0.00%> (-54.35%)` | :arrow_down: |
| [...e/impl/forward/FixedByteMVMutableForwardIndex.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9yZWFsdGltZS9pbXBsL2ZvcndhcmQvRml4ZWRCeXRlTVZNdXRhYmxlRm9yd2FyZEluZGV4LmphdmE=) | `0.00% <0.00%> (-70.79%)` | :arrow_down: |
| [...e/impl/forward/FixedByteSVMutableForwardIndex.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9yZWFsdGltZS9pbXBsL2ZvcndhcmQvRml4ZWRCeXRlU1ZNdXRhYmxlRm9yd2FyZEluZGV4LmphdmE=) | `0.00% <0.00%> (-93.88%)` | :arrow_down: |
| [...ime/impl/forward/VarByteSVMutableForwardIndex.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9yZWFsdGltZS9pbXBsL2ZvcndhcmQvVmFyQnl0ZVNWTXV0YWJsZUZvcndhcmRJbmRleC5qYXZh) | `0.00% <0.00%> (-66.67%)` | :arrow_down: |
| [...readers/constant/ConstantMVForwardIndexReader.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2luZGV4L3JlYWRlcnMvY29uc3RhbnQvQ29uc3RhbnRNVkZvcndhcmRJbmRleFJlYWRlci5qYXZh) | `0.00% <ø> (ø)` | |
| [...x/readers/forward/BaseChunkForwardIndexReader.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2luZGV4L3JlYWRlcnMvZm9yd2FyZC9CYXNlQ2h1bmtGb3J3YXJkSW5kZXhSZWFkZXIuamF2YQ==) | `0.00% <0.00%> (-46.50%)` | :arrow_down: |
| [.../readers/forward/FixedBitMVForwardIndexReader.java](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2luZGV4L3JlYWRlcnMvZm9yd2FyZC9GaXhlZEJpdE1WRm9yd2FyZEluZGV4UmVhZGVyLmphdmE=) | `0.00% <ø> (-73.50%)` | :arrow_down: |
| ... and [1285 more](https://codecov.io/gh/apache/pinot/pull/9044/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [0c55422...684a681](https://codecov.io/gh/apache/pinot/pull/9044?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
walterddr commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r921366801
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
+jobs:
+ verify-docker:
+ name: Verify Docker Image
+ runs-on: ubuntu-latest
+ steps:
+ - uses: docker/setup-qemu-action@v1
+ name: Set up QEMU
+ - uses: docker/setup-buildx-action@v1
+ name: Set up Docker Buildx
+ - uses: actions/checkout@v3
+ - name: Build the Docker image
+ env:
+ DOCKER_FILE_BASE_DIR: "docker/images/pinot"
+ DOCKER_IMAGE_NAME: "apachepinot/pinot"
+ BUILD_PLATFORM: "linux/amd64"
+ PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
+ PINOT_BRANCH: ${{ env.GITHUB_REF }}
+ TAGS: ${{ github.sha }}
+ run: .github/workflows/scripts/docker/.pinot_docker_image_build.sh
+
+ - name: Run Trivy vulnerability scanner (sarif)
+ uses: aquasecurity/trivy-action@master
+ with:
+ trivyignores: '.trivyignore'
+ image-ref: 'apachepinot/pinot:${{ github.sha }}'
+ format: 'sarif'
+ output: 'trivy-results.sarif'
+ vuln-type: 'os,library'
+ severity: 'CRITICAL,HIGH'
+ - name: Upload Trivy scan results to GitHub Security tab
+ uses: github/codeql-action/upload-sarif@v2
Review Comment:
is there an option to not create a failed GHA task but instead comment on the PR like the codecov bot?
##########
.github/workflows/scripts/docker/.pinot_docker_image_build.sh:
##########
@@ -0,0 +1,61 @@
+#!/bin/bash -x
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+if [ -z "${DOCKER_IMAGE_NAME}" ]; then
+ DOCKER_IMAGE_NAME="apachepinot/pinot"
+fi
+if [ -z "${PINOT_GIT_URL}" ]; then
+ PINOT_GIT_URL="https://github.com/apache/pinot.git"
+fi
+if [ -z "${PINOT_BRANCH}" ]; then
+ PINOT_BRANCH="master"
+fi
+if [ -z "${BUILD_PLATFORM}" ]; then
+ BUILD_PLATFORM="linux/arm64,linux/amd64"
+fi
+
+COMMIT_ID=`git rev-parse --short HEAD`
+DATE=`date +%Y%m%d`
+VERSION=`mvn help:evaluate -Dexpression=project.version -q -DforceStdout`
+tags=()
+if [ -z "${TAGS}" ]; then
+ tags=("${VERSION}-${COMMIT_ID}-${DATE}")
+ tags+=("latest")
+else
+ declare -a tags=($(echo ${TAGS} | tr "," " "))
+fi
+
+DOCKER_BUILD_TAGS=""
+for tag in "${tags[@]}"
+do
+ echo "Plan to build docker images for: ${DOCKER_IMAGE_NAME}:${tag}"
+ DOCKER_BUILD_TAGS+=" --tag ${DOCKER_IMAGE_NAME}:${tag} "
+done
Review Comment:
is there a reason we have to add multiple docker build tags?
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
Review Comment:
instead of ignore path, let's explicitly run this when `**/pom.xml` is changed.
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
+jobs:
+ verify-docker:
+ name: Verify Docker Image
+ runs-on: ubuntu-latest
+ steps:
+ - uses: docker/setup-qemu-action@v1
+ name: Set up QEMU
+ - uses: docker/setup-buildx-action@v1
+ name: Set up Docker Buildx
+ - uses: actions/checkout@v3
+ - name: Build the Docker image
+ env:
+ DOCKER_FILE_BASE_DIR: "docker/images/pinot"
+ DOCKER_IMAGE_NAME: "apachepinot/pinot"
+ BUILD_PLATFORM: "linux/amd64"
+ PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
+ PINOT_BRANCH: ${{ env.GITHUB_REF }}
+ TAGS: ${{ github.sha }}
+ run: .github/workflows/scripts/docker/.pinot_docker_image_build.sh
+
+ - name: Run Trivy vulnerability scanner (sarif)
+ uses: aquasecurity/trivy-action@master
+ with:
+ trivyignores: '.trivyignore'
+ image-ref: 'apachepinot/pinot:${{ github.sha }}'
+ format: 'sarif'
+ output: 'trivy-results.sarif'
+ vuln-type: 'os,library'
+ severity: 'CRITICAL,HIGH'
+ - name: Upload Trivy scan results to GitHub Security tab
+ uses: github/codeql-action/upload-sarif@v2
+ with:
+ sarif_file: 'trivy-results.sarif'
Review Comment:
EOF new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on a diff in pull request #9044: [DRAFT] Add a new workflow to check vulnerabilities using trivy
Posted by GitBox <gi...@apache.org>.
gortiz commented on code in PR #9044:
URL: https://github.com/apache/pinot/pull/9044#discussion_r921849859
##########
.github/workflows/pinot_vuln_check.yml:
##########
@@ -0,0 +1,66 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Pinot Dependencies
+on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
+ paths-ignore:
+ - "contrib/**"
+ - "docs/**"
+ - "licenses/**"
+ - "licenses-binary/**"
+ - "**.md"
+jobs:
+ verify-docker:
+ name: Verify Docker Image
+ runs-on: ubuntu-latest
+ steps:
+ - uses: docker/setup-qemu-action@v1
+ name: Set up QEMU
+ - uses: docker/setup-buildx-action@v1
+ name: Set up Docker Buildx
+ - uses: actions/checkout@v3
+ - name: Build the Docker image
+ env:
+ DOCKER_FILE_BASE_DIR: "docker/images/pinot"
+ DOCKER_IMAGE_NAME: "apachepinot/pinot"
+ BUILD_PLATFORM: "linux/amd64"
+ PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
+ PINOT_BRANCH: ${{ env.GITHUB_REF }}
+ TAGS: ${{ github.sha }}
+ run: .github/workflows/scripts/docker/.pinot_docker_image_build.sh
+
+ - name: Run Trivy vulnerability scanner (sarif)
+ uses: aquasecurity/trivy-action@master
+ with:
+ trivyignores: '.trivyignore'
+ image-ref: 'apachepinot/pinot:${{ github.sha }}'
+ format: 'sarif'
+ output: 'trivy-results.sarif'
+ vuln-type: 'os,library'
+ severity: 'CRITICAL,HIGH'
+ - name: Upload Trivy scan results to GitHub Security tab
+ uses: github/codeql-action/upload-sarif@v2
Review Comment:
I didn't find it. But the task compares the vulns in source and target branch. If no new vulnerability is added, it should be empty even if we have vulns in master. At least that is what I infer from the code scanning result.
I would recommend to merge the PR as it is to make it clear there are some vulnerabilities. We can add to .trivyignore the once we don't plan to fix soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org