You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by dz...@apache.org on 2023/10/24 06:48:22 UTC
[drill] branch ci-mem-tweaks created (now 52eb8469a3)
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a change to branch ci-mem-tweaks
in repository https://gitbox.apache.org/repos/asf/drill.git
at 52eb8469a3 Tune the heap and direct memory limits applied to the test suite.
This branch includes the following new commits:
new b163ab7713 Add a 2GB swap file when setting up the Actions Runner.
new 84886c5df0 Remove /proc/sys/vm/drop_caches snake oil.
new 52eb8469a3 Tune the heap and direct memory limits applied to the test suite.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[drill] 02/03: Remove /proc/sys/vm/drop_caches snake oil.
Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch ci-mem-tweaks
in repository https://gitbox.apache.org/repos/asf/drill.git
commit 84886c5df0bf09739bf3e703495aa988c399c16d
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Tue Oct 24 08:45:25 2023 +0200
Remove /proc/sys/vm/drop_caches snake oil.
See https://www.linuxatemyram.com/.
---
.github/workflows/ci.yml | 3 ---
1 file changed, 3 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 335f3da8a2..212a8b6ab0 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -60,14 +60,11 @@ jobs:
"
- name: Build and test
run: |
- MAVEN_OPTS="-XX:+UseG1GC"
- sudo sh -c 'echo 1 > /proc/sys/vm/drop_caches' && \
mvn -P${{ matrix.profile }} install --batch-mode --no-transfer-progress \
-DexcludedGroups=org.apache.drill.categories.EasyOutOfMemory \
-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
- name: Test Specific Categories # EasyOutOfMemory
run: |
- sudo sh -c 'echo 1 > /proc/sys/vm/drop_caches' && \
mvn -P${{ matrix.profile }} test -pl org.apache.drill.exec:drill-java-exec \
-Dgroups=org.apache.drill.categories.EasyOutOfMemory \
-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
[drill] 01/03: Add a 2GB swap file when setting up the Actions Runner.
Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch ci-mem-tweaks
in repository https://gitbox.apache.org/repos/asf/drill.git
commit b163ab7713d3e5605b67b9cb84efbc7a7cc617fc
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Tue Oct 24 08:22:21 2023 +0200
Add a 2GB swap file when setting up the Actions Runner.
---
.github/workflows/ci.yml | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index e83615ec90..335f3da8a2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -48,8 +48,17 @@ jobs:
distribution: 'temurin'
java-version: ${{ matrix.java }}
cache: 'maven'
+ - name: Set up swap space
+ # Linux Action Runners come with 7GB of RAM which isn't quite enoough
+ # to run the test suite. Two workarounds are used below: a swap file is
+ # added to the Runner and memory hungry tests are run separately.
+ run: |
+ sudo sh -c "
+ fallocate -l 2G /tmp/swapfile
+ mkswap /tmp/swapfile
+ swapon /tmp/swapfile
+ "
- name: Build and test
- # The total GitHub Actions memory is 7000Mb. But GitHub CI requires some memory for the container to perform tests
run: |
MAVEN_OPTS="-XX:+UseG1GC"
sudo sh -c 'echo 1 > /proc/sys/vm/drop_caches' && \
@@ -62,6 +71,9 @@ jobs:
mvn -P${{ matrix.profile }} test -pl org.apache.drill.exec:drill-java-exec \
-Dgroups=org.apache.drill.categories.EasyOutOfMemory \
-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
+ - name: Remove swap space
+ run : |
+ sudo swapoff /tmp/swapfile
checkstyle_protobuf:
name: Run checkstyle and generate protobufs
runs-on: ubuntu-latest
[drill] 03/03: Tune the heap and direct memory limits applied to the test suite.
Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch ci-mem-tweaks
in repository https://gitbox.apache.org/repos/asf/drill.git
commit 52eb8469a31e69621df5d71a51d8af285e501eda
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Tue Oct 24 08:46:22 2023 +0200
Tune the heap and direct memory limits applied to the test suite.
---
pom.xml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/pom.xml b/pom.xml
index ecbf0a9bc9..a87d52c3d7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -90,8 +90,8 @@
<javax.validation.api>2.0.1.Final</javax.validation.api>
<asm.version>9.5</asm.version>
<excludedGroups />
- <memoryMb>2500</memoryMb>
- <directMemoryMb>4500</directMemoryMb>
+ <memoryMb>2560</memoryMb>
+ <directMemoryMb>2560</directMemoryMb>
<rat.skip>true</rat.skip>
<license.skip>true</license.skip>
<docker.repository>apache/drill</docker.repository>