You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/01/31 19:35:59 UTC
[GitHub] [incubator-hudi] prashantwason opened a new pull request #1298:
[HUDI-587] Fixed generation of jacoco coverage reports.
prashantwason opened a new pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298
## What is the purpose of the pull request
Fix generation of jacococ coverage reports after unit tests are run.
## Brief change log
surefire plugin's argLine is moved into a property. This configuration allows jacoco plugin to modify the argLine to insert it's Java Agent's configuration during pre-unit-test stage.
## Verify this pull request
This pull request is already covered by existing tests, such as *(please describe tests)*.
Executed unit test and confirmed that coverage report is generated. See these logs:
mvn test -Dtest=TestHoodieActiveTimeline
10:33:33 [INFO] --- jacoco-maven-plugin:0.7.8:prepare-agent (pre-unit-test) @ hudi-common ---
10:33:33 [INFO] argLine set to -javaagent:/home/pwason/.m2/repository/org/jacoco/org.jacoco.agent/0.7.8/org.jacoco.agent-0.7.8-runtime.jar=destfile=/home/pwason/work/java/incubator-hudi/hudi-common/target/coverage-reports/jacoco-ut.exec -Xmx1024m -XX:MaxPermSize=256m
...
...
10:33:41 [INFO] --- jacoco-maven-plugin:0.7.8:report (post-unit-test) @ hudi-common ---
10:33:41 [INFO] Loading execution data file /home/pwason/work/java/incubator-hudi/hudi-common/target/coverage-reports/jacoco-ut.exec
10:33:42 [INFO] Analyzed bundle 'hudi-common' with 273 classes
ls -al hudi-common/target/coverage-reports/jacoco-ut.exec
-rw-rw-r-- 1 pwason pwason 49240 Jan 31 10:33 hudi-common/target/coverage-reports/jacoco-ut.exec
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on a change in pull
request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298#discussion_r374536313
##########
File path: pom.xml
##########
@@ -113,6 +113,7 @@
<spark.bundle.hive.shade.prefix></spark.bundle.hive.shade.prefix>
<utilities.bundle.hive.scope>provided</utilities.bundle.hive.scope>
<utilities.bundle.hive.shade.prefix></utilities.bundle.hive.shade.prefix>
+ <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
Review comment:
I don't think travis has the smarts to grok this and get the resource needs? We just get a VM with some fixed RAM https://docs.travis-ci.com/user/reference/overview/
`-Xmx` won't also grab the 1GB right off the bat.. do you see the builds slowing down? would be interesting to understand why..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] n3nash merged pull request #1298: [HUDI-587] Fixed
generation of jacoco coverage reports.
Posted by GitBox <gi...@apache.org>.
n3nash merged pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] n3nash commented on a change in pull request
#1298: [HUDI-587] Fixed generation of jacoco coverage reports.
Posted by GitBox <gi...@apache.org>.
n3nash commented on a change in pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298#discussion_r375543391
##########
File path: pom.xml
##########
@@ -113,6 +113,7 @@
<spark.bundle.hive.shade.prefix></spark.bundle.hive.shade.prefix>
<utilities.bundle.hive.scope>provided</utilities.bundle.hive.scope>
<utilities.bundle.hive.shade.prefix></utilities.bundle.hive.shade.prefix>
+ <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
Review comment:
Okay, makes sense.
@vinothchandar I don't see it slowing down, just had a hunch, merging this as this does not seem to have any side-effect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] n3nash commented on a change in pull request
#1298: [HUDI-587] Fixed generation of jacoco coverage reports.
Posted by GitBox <gi...@apache.org>.
n3nash commented on a change in pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298#discussion_r374442872
##########
File path: pom.xml
##########
@@ -113,6 +113,7 @@
<spark.bundle.hive.shade.prefix></spark.bundle.hive.shade.prefix>
<utilities.bundle.hive.scope>provided</utilities.bundle.hive.scope>
<utilities.bundle.hive.shade.prefix></utilities.bundle.hive.shade.prefix>
+ <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
Review comment:
@prashantwason this has a side effect of making the builds run slower. The reason is that when a build is submitted in travis, it needs to wait for resources to be available for the build to get triggered. My understanding is that if we ask for a larger JVM for non-IntegrationTest modules as well, this will make the process slower.
How much more memory is required for the coverage report ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] prashantwason commented on a change in pull
request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
Posted by GitBox <gi...@apache.org>.
prashantwason commented on a change in pull request #1298: [HUDI-587] Fixed generation of jacoco coverage reports.
URL: https://github.com/apache/incubator-hudi/pull/1298#discussion_r374913677
##########
File path: pom.xml
##########
@@ -113,6 +113,7 @@
<spark.bundle.hive.shade.prefix></spark.bundle.hive.shade.prefix>
<utilities.bundle.hive.scope>provided</utilities.bundle.hive.scope>
<utilities.bundle.hive.shade.prefix></utilities.bundle.hive.shade.prefix>
+ <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
Review comment:
I did not add this line. It was already part of the maven-surefire plugin's argLine. I just moved the definition to a property so that jacoco plugin can add its own java-agent config to it and the unit tests can execute with jacococ agent enabled.
This was added in the following commit 3 months ago:
https://github.com/apache/incubator-hudi/pull/979
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services