You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "kevinw66 (via GitHub)" <gi...@apache.org> on 2024/04/07 09:37:56 UTC
[PR] [WIP] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
kevinw66 opened a new pull request, #1248:
URL: https://github.com/apache/bigtop/pull/1248
### Description of PR
The origin PR is #1158
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
- [ ] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1248:
URL: https://github.com/apache/bigtop/pull/1248#issuecomment-2095573623
@kevinw66 How about adding the author of #1158 as co-author on merging this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "kevinw66 (via GitHub)" <gi...@apache.org>.
kevinw66 commented on PR #1248:
URL: https://github.com/apache/bigtop/pull/1248#issuecomment-2080351254
@iwasakims @sekikn Could you please help review this? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "kevinw66 (via GitHub)" <gi...@apache.org>.
kevinw66 commented on PR #1248:
URL: https://github.com/apache/bigtop/pull/1248#issuecomment-2096124198
> @kevinw66 How about adding the author of #1158 as co-author on merging this? `root@localhost.localdomain` should be updated to appropriate author too.
I think this can be edited directly on the text area that appears after clicking `Squash and merge`, but I can change the commit history if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "kevinw66 (via GitHub)" <gi...@apache.org>.
kevinw66 commented on PR #1248:
URL: https://github.com/apache/bigtop/pull/1248#issuecomment-2080350951
Successfully run smoke tests on CentOS-7:
```
[root@localhost docker]# ./docker-hadoop.sh -d -dcp -C config_centos-7.yaml -G -L -m 4g -c 1 -k hdfs,yarn,mapreduce,hive -s hive
......
> Task :bigtop-tests:smoke-tests:hive:test
Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.pom to /tmp/gradle_download8121793616240671202bin
Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.pom to /tmp/gradle_download3858645288563285746bin
Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar to /tmp/gradle_download6871050970468328068bin
Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar to /tmp/gradle_download8258873549123127472bin
Caching disabled for task ':bigtop-tests:smoke-tests:hive:test' because:
Build cache is disabled
Task ':bigtop-tests:smoke-tests:hive:test' is not up-to-date because:
No history is available.
Starting process 'Gradle Test Executor 2'. Working directory: /bigtop-home/bigtop-tests/smoke-tests/hive Command: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/bin/java -Dawt.toolkit=sun.awt.X11.XToolkit -Dfile.encoding.pkg=sun.io -Dfile.separator=/ -Djava.awt.graphicsenv=sun.awt.X11GraphicsEnvironment -Djava.awt.printerjob=sun.print.PSPrinterJob -Djava.class.path=/root/.gradle/wrapper/dists/gradle-5.6.4-bin/c9880aa85176bf8c458862eb99f7e0a9/gradle-5.6.4/lib/gradle-launcher-5.6.4.jar -Djava.class.version=52.0 -Djava.endorsed.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/ext:/usr/java/packages/lib/ext -Djava.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib -Djava.runtime.name=OpenJDK Runtime Environment -Djava.runtime.version=1.8.0_402-b06 -Djava.s
ecurity.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djava.specification.maintenance.version=5 -Djava.specification.name=Java Platform API Specification -Djava.specification.vendor=Oracle Corporation -Djava.specification.version=1.8 -Djava.vendor=Red Hat, Inc. -Djava.vendor.url=https://www.redhat.com/ -Djava.vendor.url.bug=https://access.redhat.com/support/cases/ -Djava.version=1.8.0_402 -Djava.vm.info=mixed mode -Djava.vm.name=OpenJDK 64-Bit Server VM -Djava.vm.specification.name=Java Virtual Machine Specification -Djava.vm.specification.vendor=Oracle Corporation -Djava.vm.specification.version=1.8 -Djava.vm.vendor=Red Hat, Inc. -Djava.vm.version=25.402-b06 -Dlibrary.jansi.path=/root/.gradle/native/jansi/1.17.1/linux64 -Dline.separator=
-Dorg.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include=xxx -Dorg.gradle.appname=gradlew -Dorg.gradle.native=false -Dos.arch=amd64 -Dos.name=Linux -Dos.version=3.10.0-1160.el7.x86_64 -Dpath.separator=: -Dsun.arch.data.model=64 -Dsun.boot.class.path=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/classes -Dsun.boot.library.path=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/lib/amd64 -Dsun
.cpu.endian=little -Dsun.cpu.isalist -Dsun.io.unicode.encoding=UnicodeLittle -Dsun.java.command=org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.4 -Dsun.java.launcher=SUN_STANDARD -Dsun.jnu.encoding=ANSI_X3.4-1968 -Dsun.management.compiler=HotSpot 64-Bit Tiered Compilers -Dsun.os.patch.level=unknown -Duser.dir=/bigtop-home -Duser.home=/root -Duser.name=root -Duser.timezone -Xmx512m -Dfile.encoding=ANSI_X3.4-1968 -Djava.io.tmpdir=/tmp -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /root/.gradle/caches/5.6.4/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 2'
Successfully started process 'Gradle Test Executor 2'
Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.
> Task :bigtop-tests:smoke-tests:hive:test
Finished generating test XML results (0.008 secs) into: /bigtop-home/bigtop-tests/smoke-tests/hive/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.009 secs) into: /bigtop-home/bigtop-tests/smoke-tests/hive/build/reports/tests/test
Now testing...
:bigtop-tests:smoke-tests:hive:test (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 31.554 secs.
BUILD SUCCESSFUL in 4m 21s
28 actionable tasks: 6 executed, 22 up-to-date
Stopped 1 worker daemon(s).
+ rm -rf buildSrc/build/test-results/binary
+ rm -rf /bigtop-home/.gradle
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "kevinw66 (via GitHub)" <gi...@apache.org>.
kevinw66 commented on PR #1248:
URL: https://github.com/apache/bigtop/pull/1248#issuecomment-2080337076
Successfully run smoke tests on debian-11:
```
[root@localhost docker]# ./docker-hadoop.sh -d -dcp -C config_debian-11.yaml -G -L -m 4g -c 1 -k hdfs,yarn,mapreduce,hive -s hive
......
> Task :bigtop-tests:smoke-tests:hive:test
Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.pom to /tmp/gradle_download4722148391927244333bin
Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.pom to /tmp/gradle_download5205515868830083434bin
Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar to /tmp/gradle_download729204132602745612bin
Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar to /tmp/gradle_download8595248545323719120bin
Caching disabled for task ':bigtop-tests:smoke-tests:hive:test' because:
Build cache is disabled
Task ':bigtop-tests:smoke-tests:hive:test' is not up-to-date because:
No history is available.
Starting process 'Gradle Test Executor 2'. Working directory: /bigtop-home/bigtop-tests/smoke-tests/hive Command: /usr/lib/jvm/temurin-8-jdk-amd64/bin/java -Dawt.toolkit=sun.awt.X11.XToolkit -Dfile.encoding.pkg=sun.io -Dfile.separator=/ -Djava.awt.graphicsenv=sun.awt.X11GraphicsEnvironment -Djava.awt.printerjob=sun.print.PSPrinterJob -Djava.class.path=/root/.gradle/wrapper/dists/gradle-5.6.4-bin/c9880aa85176bf8c458862eb99f7e0a9/gradle-5.6.4/lib/gradle-launcher-5.6.4.jar -Djava.class.version=52.0 -Djava.endorsed.dirs=/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext -Djava.home=/usr/lib/jvm/temurin-8-jdk-amd64/jre -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib -Djava.runtime.name=OpenJDK Runtime Environment -Djava.runtime.version=1.8.0_412-b08 -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djava.specification.ma
intenance.version=5 -Djava.specification.name=Java Platform API Specification -Djava.specification.vendor=Oracle Corporation -Djava.specification.version=1.8 -Djava.vendor=Temurin -Djava.vendor.url=https://adoptium.net/ -Djava.vendor.url.bug=https://github.com/adoptium/adoptium-support/issues -Djava.version=1.8.0_412 -Djava.vm.info=mixed mode -Djava.vm.name=OpenJDK 64-Bit Server VM -Djava.vm.specification.name=Java Virtual Machine Specification -Djava.vm.specification.vendor=Oracle Corporation -Djava.vm.specification.version=1.8 -Djava.vm.vendor=Temurin -Djava.vm.version=25.412-b08 -Dlibrary.jansi.path=/root/.gradle/native/jansi/1.17.1/linux64 -Dline.separator=
-Dorg.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include=xxx -Dorg.gradle.appname=gradlew -Dorg.gradle.native=false -Dos.arch=amd64 -Dos.name=Linux -Dos.version=3.10.0-1160.el7.x86_64 -Dpath.separator=: -Dsun.arch.data.model=64 -Dsun.boot.class.path=/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/rt.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/temurin-8-jdk-amd64/jre/classes -Dsun.boot.library.path=/usr/lib/jvm/temurin-8-jdk-amd64/jre/lib/amd64 -Dsun.cpu.endian=little -Dsun.cpu.isalist -Dsun.io.unicode.encoding=UnicodeLittle -Dsun.java.command=org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.4 -Dsun.java.launcher=SUN_STANDARD -Dsun.jnu.encoding=ANSI_X3.4-1968 -Dsun.management.compiler=HotSpot
64-Bit Tiered Compilers -Dsun.os.patch.level=unknown -Duser.dir=/bigtop-home -Duser.home=/root -Duser.name=root -Duser.timezone -Xmx512m -Dfile.encoding=ANSI_X3.4-1968 -Djava.io.tmpdir=/tmp -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /root/.gradle/caches/5.6.4/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 2'
Successfully started process 'Gradle Test Executor 2'
Gradle Test Executor 2 finished executing tests.
> Task :bigtop-tests:smoke-tests:hive:test
Finished generating test XML results (0.016 secs) into: /bigtop-home/bigtop-tests/smoke-tests/hive/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.016 secs) into: /bigtop-home/bigtop-tests/smoke-tests/hive/build/reports/tests/test
Now testing...
:bigtop-tests:smoke-tests:hive:test (Thread[Execution worker for ':' Thread 3,5,main]) completed. Took 19.636 secs.
BUILD SUCCESSFUL in 1m 1s
28 actionable tasks: 4 executed, 24 up-to-date
Stopped 1 worker daemon(s).
+ rm -rf buildSrc/build/test-results/binary
+ rm -rf /bigtop-home/.gradle
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-3963: Merge hive memory leak PRs (for hive-3.1.3) [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims merged PR #1248:
URL: https://github.com/apache/bigtop/pull/1248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org