You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/11/02 04:42:56 UTC
[GitHub] [hudi] nsivabalan opened a new pull request, #7111: [DOCS][MINOR] Adding useful maven commands to website
nsivabalan opened a new pull request, #7111:
URL: https://github.com/apache/hudi/pull/7111
### Change Logs
Adding useful maven commands to our website.
### Impact
Improves productivity for devs.
### Risk level (write none, low medium or high below)
low.
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #7111: [DOCS][MINOR] Adding useful maven commands to website
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #7111:
URL: https://github.com/apache/hudi/pull/7111#discussion_r1011193279
##########
website/contribute/developer-setup.md:
##########
@@ -245,6 +245,49 @@ to effect change and source feedback, start a new email thread with the `[DISCUS
by a [vote](https://www.apache.org/foundation/voting) by a PMC member or others (depending on the specific scenario).
For technical suggestions, you can also leverage [our RFC Process](https://cwiki.apache.org/confluence/display/HUDI/RFC+Process) to outline your ideas in greater detail.
+### Useful Maven commands for developers.
+Listing out some of the maven commands that could be useful for developers.
+
+- Compile/build entire project
+
+```shell
+mvn clean package -DskipTests
+```
+Default profile is spark2 and scala2.11
+
+- For continuous development, you may want to build only the modules of interest. for eg, if you have been working with
+deltastreamer, you can build using this command instead of entire project. Majority of time goes into building all different bundles we have
+like flink bundle, presto bundle, trino bundle etc. But if you are developing something confined to hudi-utilties, you can achieve faster
+build times.
+
+```shell
+mvn package -DskipTests -pl packaging/hudi-utilities-bundle/ -am
+```
+
+To enable multi-threaded building, you can add -T.
+```shell
+mvn -T 2C package -DskipTests -pl packaging/hudi-utilities-bundle/ -am
+```
+This command will use 2 parallel threads to build.
+
+You can also confine the build to just one module if need be.
+```shell
+mvn -T 2C package -DskipTests -pl hudi-spark-datasource/hudi-spark -am
+```
+Note: "-am" will build all dependent modules as well.
+In local laptop, entire project build can take somewhere close to 7 to 10 mins. While buildig just hudi-spark-datasource/hudi-spark
Review Comment:
```suggestion
In local laptop, entire project build can take somewhere close to 7 to 10 mins. While buildig just hudi-spark-datasource/hudi-spark
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #7111: [DOCS][MINOR] Adding useful maven commands to website
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #7111:
URL: https://github.com/apache/hudi/pull/7111#discussion_r1011312575
##########
website/contribute/developer-setup.md:
##########
@@ -245,6 +245,49 @@ to effect change and source feedback, start a new email thread with the `[DISCUS
by a [vote](https://www.apache.org/foundation/voting) by a PMC member or others (depending on the specific scenario).
For technical suggestions, you can also leverage [our RFC Process](https://cwiki.apache.org/confluence/display/HUDI/RFC+Process) to outline your ideas in greater detail.
+### Useful Maven commands for developers.
+Listing out some of the maven commands that could be useful for developers.
+
+- Compile/build entire project
+
+```shell
+mvn clean package -DskipTests
+```
+Default profile is spark2 and scala2.11
+
+- For continuous development, you may want to build only the modules of interest. for eg, if you have been working with
+deltastreamer, you can build using this command instead of entire project. Majority of time goes into building all different bundles we have
+like flink bundle, presto bundle, trino bundle etc. But if you are developing something confined to hudi-utilties, you can achieve faster
+build times.
+
+```shell
+mvn package -DskipTests -pl packaging/hudi-utilities-bundle/ -am
+```
+
+To enable multi-threaded building, you can add -T.
+```shell
+mvn -T 2C package -DskipTests -pl packaging/hudi-utilities-bundle/ -am
+```
+This command will use 2 parallel threads to build.
+
+You can also confine the build to just one module if need be.
+```shell
+mvn -T 2C package -DskipTests -pl hudi-spark-datasource/hudi-spark -am
+```
+Note: "-am" will build all dependent modules as well.
+In local laptop, entire project build can take somewhere close to 7 to 10 mins. While buildig just hudi-spark-datasource/hudi-spark
+with multi-threaded, could get your compilation in 1.5 to 2 mins.
+
+If you wish to run any single test class in java.
+```shell
+mvn test -Punit-tests -pl hudi-spark-datasource/hudi-spark/ -am -B -DfailIfNoTests=false -Dtest=TestCleaner
+```
+
+If you wish to run a single test method in java.
+```shell
+mvn test -Punit-tests -pl hudi-spark-datasource/hudi-spark/ -am -B -DfailIfNoTests=false -Dtest=TestCleaner#testKeepLatestCommitsMOR
+```
Review Comment:
You need `-DwildcardSuites="abc"` to skip other scala tests to save time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7111: [DOCS][MINOR] Adding useful maven commands to website
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7111:
URL: https://github.com/apache/hudi/pull/7111#issuecomment-1305185470
@codope : feel free to land. addressed all comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #7111: [DOCS][MINOR] Adding useful maven commands to website
Posted by GitBox <gi...@apache.org>.
codope merged PR #7111:
URL: https://github.com/apache/hudi/pull/7111
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7111: [DOCS][MINOR] Adding useful maven commands to website
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7111:
URL: https://github.com/apache/hudi/pull/7111#issuecomment-1299560582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org