You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "lordgamez (via GitHub)" <gi...@apache.org> on 2023/06/20 08:22:20 UTC
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1591: MINIFICPP-2130 Custom cache eviction strategy for GitHub Actions
lordgamez opened a new pull request, #1591:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1591
https://issues.apache.org/jira/browse/MINIFICPP-2130
-----------------
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically main)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1591: MINIFICPP-2130 Custom cache eviction strategy for GitHub Actions
Posted by "lordgamez (via GitHub)" <gi...@apache.org>.
lordgamez commented on code in PR #1591:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1591#discussion_r1247553731
##########
.github/workflows/ci.yml:
##########
@@ -142,6 +154,12 @@ jobs:
-DENABLE_AZURE=OFF -DENABLE_SPLUNK=OFF -DENABLE_GCP=OFF -DENABLE_PROCFS=OFF -DENABLE_BUSTACHE=ON -DENABLE_PCAP=ON -DENABLE_JNI=ON -DENABLE_SFTP=ON \
-DENABLE_LUA_SCRIPTING=OFF -DENABLE_PYTHON_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_ELASTICSEARCH=OFF -DENABLE_KUBERNETES=OFF -DENABLE_OPC=OFF ..
make -j$(nproc) VERBOSE=1
+ - name: cache save
+ uses: actions/cache/save@v3
+ if: always()
+ with:
+ path: ~/.ccache
+ key: ubuntu-20.04-ccache-${{github.ref}}-${{github.sha}}
Review Comment:
Before this change the cache save-restore was handled automatically in the action, by restoring the cache at the beginning of the job and saving the cache after the job was successful. The problem with this approach was that if any of the steps in the job failed, e.g. the tests were flaky or the linter failed, the build cache was not saved, so if we only wanted rerun the tests, or fixed the build in the next commit we had to run a clean build again. With this new approach after the build step is done it is saved immediately and can be reused even if any other steps fail (or even if the build fails somewhere).
##########
github_scripts/github_actions_cache_cleanup_tests.py:
##########
@@ -0,0 +1,80 @@
+#!/bin/python3
+
+import unittest
+from unittest.mock import MagicMock
+from github_actions_cache_cleanup import GithubActionsCacheCleaner
+
+
+class TestGithubActionsCacheCleaner(unittest.TestCase):
+ def create_mock_github_request_sender(self):
+ mock = MagicMock()
+ mock.list_open_pull_requests = MagicMock()
+ open_pull_requests = [
+ {
+ "number": "227",
+ "title": "MINIFICPP-13712 TEST1",
+ },
+ {
+ "number": "228",
+ "title": "MINIFICPP-9999 TEST2",
+ },
+ {
+ "number": "229",
+ "title": "MINIFICPP-123 TEST3",
+ }
+ ]
+ mock.list_open_pull_requests.return_value = open_pull_requests
+ caches = {
+ "actions_caches": [
+ {
+ "id": "999",
+ "key": "macos-xcode-ccache-refs/pull/226/merge-6c8d283f5bc894af8dfc295e5976a5f154753123",
+ },
+ {
+ "id": "11111",
+ "key": "ubuntu-20.04-ccache-refs/pull/227/merge-9d6d283f5bc894af8dfc295e5976a5f1b46649c4",
+ },
+ {
+ "id": "11112",
+ "key": "ubuntu-20.04-ccache-refs/pull/227/merge-1d6d283f5bc894af8dfc295e5976a5f154753487",
+ },
+ {
+ "id": "12345",
+ "key": "macos-xcode-ccache-refs/pull/227/merge-2d6d283f5bc894af8dfc295e5976a5f154753536",
+ },
+ {
+ "id": "22221",
+ "key": "macos-xcode-ccache-refs/heads/MINIFICPP-9999-9d5e183f5bc894af8dfc295e5976a5f1b4664456",
+ },
+ {
+ "id": "22222",
+ "key": "macos-xcode-ccache-refs/heads/MINIFICPP-9999-8f4d283f5bc894af8dfc295e5976a5f1b4664123",
+ },
+ {
+ "id": "44444",
+ "key": "ubuntu-20.04-all-clang-ccache-refs/heads/main-1d4d283f5bc894af8dfc295e5976a5f1b4664456",
+ },
+ {
+ "id": "55555",
+ "key": "ubuntu-20.04-all-clang-ccache-refs/heads/main-2f4d283f5bc894af8dfc295e5976a5f1b4664567",
+ }
Review Comment:
We check [which id is bigger,](https://github.com/apache/nifi-minifi-cpp/pull/1591/files#diff-2928f02afcafb093965cad50e466a01dc35c94e133f336a057c612bc530fb5d0R83) as I checked the cache ids are sequential in github actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1591: MINIFICPP-2130 Custom cache eviction strategy for GitHub Actions
Posted by "fgerlits (via GitHub)" <gi...@apache.org>.
fgerlits closed pull request #1591: MINIFICPP-2130 Custom cache eviction strategy for GitHub Actions
URL: https://github.com/apache/nifi-minifi-cpp/pull/1591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1591: MINIFICPP-2130 Custom cache eviction strategy for GitHub Actions
Posted by "szaszm (via GitHub)" <gi...@apache.org>.
szaszm commented on code in PR #1591:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1591#discussion_r1236873733
##########
.github/workflows/ci.yml:
##########
@@ -142,6 +154,12 @@ jobs:
-DENABLE_AZURE=OFF -DENABLE_SPLUNK=OFF -DENABLE_GCP=OFF -DENABLE_PROCFS=OFF -DENABLE_BUSTACHE=ON -DENABLE_PCAP=ON -DENABLE_JNI=ON -DENABLE_SFTP=ON \
-DENABLE_LUA_SCRIPTING=OFF -DENABLE_PYTHON_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_ELASTICSEARCH=OFF -DENABLE_KUBERNETES=OFF -DENABLE_OPC=OFF ..
make -j$(nproc) VERBOSE=1
+ - name: cache save
+ uses: actions/cache/save@v3
+ if: always()
+ with:
+ path: ~/.ccache
+ key: ubuntu-20.04-ccache-${{github.ref}}-${{github.sha}}
Review Comment:
How did this work before adding the explicit path declaration? Why is the change necessary?
##########
github_scripts/github_actions_cache_cleanup_tests.py:
##########
@@ -0,0 +1,80 @@
+#!/bin/python3
+
+import unittest
+from unittest.mock import MagicMock
+from github_actions_cache_cleanup import GithubActionsCacheCleaner
+
+
+class TestGithubActionsCacheCleaner(unittest.TestCase):
+ def create_mock_github_request_sender(self):
+ mock = MagicMock()
+ mock.list_open_pull_requests = MagicMock()
+ open_pull_requests = [
+ {
+ "number": "227",
+ "title": "MINIFICPP-13712 TEST1",
+ },
+ {
+ "number": "228",
+ "title": "MINIFICPP-9999 TEST2",
+ },
+ {
+ "number": "229",
+ "title": "MINIFICPP-123 TEST3",
+ }
+ ]
+ mock.list_open_pull_requests.return_value = open_pull_requests
+ caches = {
+ "actions_caches": [
+ {
+ "id": "999",
+ "key": "macos-xcode-ccache-refs/pull/226/merge-6c8d283f5bc894af8dfc295e5976a5f154753123",
+ },
+ {
+ "id": "11111",
+ "key": "ubuntu-20.04-ccache-refs/pull/227/merge-9d6d283f5bc894af8dfc295e5976a5f1b46649c4",
+ },
+ {
+ "id": "11112",
+ "key": "ubuntu-20.04-ccache-refs/pull/227/merge-1d6d283f5bc894af8dfc295e5976a5f154753487",
+ },
+ {
+ "id": "12345",
+ "key": "macos-xcode-ccache-refs/pull/227/merge-2d6d283f5bc894af8dfc295e5976a5f154753536",
+ },
+ {
+ "id": "22221",
+ "key": "macos-xcode-ccache-refs/heads/MINIFICPP-9999-9d5e183f5bc894af8dfc295e5976a5f1b4664456",
+ },
+ {
+ "id": "22222",
+ "key": "macos-xcode-ccache-refs/heads/MINIFICPP-9999-8f4d283f5bc894af8dfc295e5976a5f1b4664123",
+ },
+ {
+ "id": "44444",
+ "key": "ubuntu-20.04-all-clang-ccache-refs/heads/main-1d4d283f5bc894af8dfc295e5976a5f1b4664456",
+ },
+ {
+ "id": "55555",
+ "key": "ubuntu-20.04-all-clang-ccache-refs/heads/main-2f4d283f5bc894af8dfc295e5976a5f1b4664567",
+ }
Review Comment:
Out of these two, how do we determine which one is newer? We should keep the newest cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org