You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/02/19 14:02:57 UTC
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
lordgamez opened a new pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012
After checking the available solutions for caching the build objects on Windows with Visual Studio, clcache was chosen. From the available options unfortunately sccache did not work properly with msbuild, and clcache seemed to be a better option than cclash.
Runtimes measured in Github Actions:
Without caching:
vs2017 build: 1h 28m 37s
windows-vs2019: 56m 44s
With caching:
windows-vs2017: 51m 24s
windows-vs2019: 27m 33s
-----------------------------------------------------
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically main)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
szaszm commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579336757
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe"
- id: build
run: |
PATH %PATH%;C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86
PATH %PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Roslyn
+ set "CLCACHE_CL=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
win_build_vs.bat build /CI /S /A
shell: cmd
windows_VS2019:
name: "windows-vs2019"
runs-on: windows-2019
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2019-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2019-clcache-${{github.ref}}-
+ windows-vs2019-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe"
Review comment:
I suggest looking at `%VSToolsInstallDir%` which is defined to be `C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\` on my vs2017 installation and `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\` on vs2019
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
szaszm commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579331781
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
Review comment:
Did you try setting `CMAKE_CXX_COMPILER=sccache cl`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
arpadboda closed pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
adamdebreceni commented on pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#issuecomment-804726296
even though clcache is unmaintained, I am not against using it in the CI as the speed-up is substantial, at worst it could introduce inconsistencies between a fresh build a build using clcache (and in worst case false negatives), but as I understand clcache users voiced no such events/concerns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579324537
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe"
- id: build
run: |
PATH %PATH%;C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86
PATH %PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Roslyn
+ set "CLCACHE_CL=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
win_build_vs.bat build /CI /S /A
shell: cmd
windows_VS2019:
name: "windows-vs2019"
runs-on: windows-2019
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2019-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2019-clcache-${{github.ref}}-
+ windows-vs2019-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe"
Review comment:
You are right, I will find a solution for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
szaszm commented on pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#issuecomment-817732507
I wanted to clarify my stance on this issue. I maintain that I'm opposed to introducing CI caching at the expense of a dependency on an unmaintained project, but if there is a majority support for it, I'm not vetoing it.
Based on [this](https://www.apache.org/foundation/voting.html#expressing-votes-1-0-1-and-fractions), I would be a -0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez edited a comment on pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez edited a comment on pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#issuecomment-816602636
> I have tested this in my fork. It works very well on a 2nd CI run in the same branch: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/729202774 but it does not speed up a compile of the same code in a new branch: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/729538166. Is there a way to make the cache work across branches?
Currently all cached builds try to use 2 restore-keys for retrieving cache: the one for the current branch and the one for the main branch. It works the same way for the Windows builds. If a new branch is built and no cache is available for that yet, it will use the cache from the main branch. As the change is not yet submitted there is no available cache from the main branch that's why it is slow on the first compilation, but it should be available after the PR is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#issuecomment-816602636
> I have tested this in my fork. It works very well on a 2nd CI run in the same branch: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/729202774 but it does not speed up a compile of the same code in a new branch: https://github.com/fgerlits/nifi-minifi-cpp/actions/runs/729538166. Is there a way to make the cache work across branches?
Currently all cached builds try use 2 restore-keys for retrieving cache: the one for the current branch and the one for the main branch. It works the same way for the Windows builds. If a new branch is built and no cache is available for that yet, it will use the cache from the main branch. As the change is not yet submitted there is no available cache from the main branch that's why it is slow on the first compilation, but it should be available after the PR is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r580101740
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
Review comment:
Unfortunately I could not make sccache work with cmake and msbuild. I tried setting cmake parameters like the `CMAKE_CXX_COMPILER` and `CMAKE_CXX_COMPILER_LAUNCHER` but sccache was not used. I also tried multiple methods mentioned in https://github.com/mozilla/sccache/issues/107, like setting the msbuild configuration parameters and replacing the cl.exe, but unfortunately non of them seem to work. Replacing the cl.exe with sccache and adjusting the PATH was the only method where I could see the cache calls in the sccache statistics, but all of them were marked as `Non-cacheable calls`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579335111
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
Review comment:
I remember I tried `CMAKE_CXX_COMPILER_LAUNCHER` and it didn't work, maybe I tried this as well, but I'm not sure, I will give it a go.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
szaszm commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579298179
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe"
- id: build
run: |
PATH %PATH%;C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86
PATH %PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Roslyn
+ set "CLCACHE_CL=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
win_build_vs.bat build /CI /S /A
shell: cmd
windows_VS2019:
name: "windows-vs2019"
runs-on: windows-2019
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2019-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2019-clcache-${{github.ref}}-
+ windows-vs2019-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe"
Review comment:
Toolchain version numbers like those in the path change with every upgrade. I expect this to break every few months, when GitHub decides to upgrade its toolchain.
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
Review comment:
clcache is not maintained, so I'm not a fan of using it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r580343584
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl.exe"
- id: build
run: |
PATH %PATH%;C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86
PATH %PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Roslyn
+ set "CLCACHE_CL=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe"
win_build_vs.bat build /CI /S /A
shell: cmd
windows_VS2019:
name: "windows-vs2019"
runs-on: windows-2019
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2019-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2019-clcache-${{github.ref}}-
+ windows-vs2019-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
+ Expand-Archive -Force -Path clcache.zip -DestinationPath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64";
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe.config" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl_original.exe.config"
+ move "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\clcache.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe"
Review comment:
Removed the hardcoded versions in [b2f4a71](https://github.com/apache/nifi-minifi-cpp/pull/1012/commits/b2f4a712d7f84c3dec0e35d5146e162bf35def6d)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1012: MINIFICPP-1499 Add clcache to Windows builds
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1012:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1012#discussion_r579323553
##########
File path: .github/workflows/ci.yml
##########
@@ -64,27 +64,64 @@ jobs:
windows_VS2017:
name: "windows-vs2017"
runs-on: windows-2016
- timeout-minutes: 90
+ timeout-minutes: 120
+ env:
+ CLCACHE_DIR: ${{ GITHUB.WORKSPACE }}\clcache
+ CLCACHE_CL: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\cl_original.exe
steps:
- id: checkout
uses: actions/checkout@v2
+ - id: cache
+ uses: actions/cache@v2
+ with:
+ path: ${{ env.CLCACHE_DIR }}
+ key: windows-vs2017-clcache-${{github.ref}}-${{github.sha}}
+ restore-keys: |
+ windows-vs2017-clcache-${{github.ref}}-
+ windows-vs2017-clcache-refs/heads/main-
- name: Setup PATH
uses: microsoft/setup-msbuild@v1.0.2
+ - name: Setup clcache
+ run: |
+ (New-Object System.Net.WebClient).DownloadFile('https://github.com/frerich/clcache/releases/download/v4.2.0/clcache-4.2.0.zip', "$pwd\clcache.zip")
Review comment:
I will give another go with sccache and see if I can make it work. Unfortunately I only found 3 options of caching for MSVC (2 of which are not maintained), but still from what I've found clcache seem to be the preferred one. There are some forks with active development, the author of clcache advises people to merge new required features to those repositories. Although I think for our use case the last release version may suffice if sccache cannot be used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org