You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/09/15 18:34:39 UTC
[GitHub] [hudi] rahil-c opened a new pull request, #6684: Add Documentation for running unit tests/debugger with spark3
rahil-c opened a new pull request, #6684:
URL: https://github.com/apache/hudi/pull/6684
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
### Impact
_Describe any public API or user-facing feature change or any performance impact._
**Risk level: none | low | medium | high**
_Choose one. If medium or high, explain what verification was done to mitigate the risks._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
nsivabalan merged PR #6684:
URL: https://github.com/apache/hudi/pull/6684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
the-other-tim-brown commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r976959488
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,20 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+#### IMPORTANT REMINDER FOR BELOW STEPS: When submitting a PR please make sure to NOT commit the changes mentioned in these steps, instead once testing is done make sure to revert the changes and then submit a pr.
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
Review Comment:
We should note that the developer should click the checkbox for the "Skip Tests" option so they don't have to wait on the tests to run as part of the intellij build process
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r975866969
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
We should ideally avoid any local changes to POM as they inevitable will be getting committed occasionally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r976955462
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
> BTW, i just tried `TestMORDataSource` and it works fine for me w/o `skipTests` flag
got it, will remove `4.`
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
> BTW, i just tried `TestMORDataSource` and it works fine for me w/o `skipTests` flag
got it, will remove `4.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r976955970
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,20 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+#### IMPORTANT REMINDER FOR BELOW STEPS: When submitting a PR please make sure to NOT commit the changes mentioned in these steps, instead once testing is done make sure to revert the changes and then submit a pr.
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
Review Comment:
> Can we add `.mvn` to the git ignore to help prevent people from adding it to their PRs?
Yes thats a good idea will do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r976955462
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
> BTW, i just tried `TestMORDataSource` and it works fine for me w/o `skipTests` flag
got it, will remove the skip tests line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r975867133
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
BTW, i just tried `TestMORDataSource` and it works fine for me w/o `skipTests` flag
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6684: Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r972387855
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
Review Comment:
Let's add a screaming note, that people should NOT commit it and not forget to revert this before putting up the PR
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
I don't think this is necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r973242929
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
+4. In the root pom.xml under `maven-surefire-plugin`, under this plugin's `<configuration>` make sure to add `<skipTests>false<skipTests>`
Review Comment:
I think this is necessary for certain tests, when I tried running `TestMORDataSource` from IDE it would skip the test. Only when doing step 4 was it working for me. If you are able to run these tests without doing the step 4 i can remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on pull request #6684: Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on PR #6684:
URL: https://github.com/apache/hudi/pull/6684#issuecomment-1248459903
cc @yihua @alexeykudinkin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r976996005
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
Review Comment:
did you put up a PR to add .mvn to .gitignore file. we can land that in right instead of reminding folks here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
the-other-tim-brown commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r975892318
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,20 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+#### IMPORTANT REMINDER FOR BELOW STEPS: When submitting a PR please make sure to NOT commit the changes mentioned in these steps, instead once testing is done make sure to revert the changes and then submit a pr.
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
Review Comment:
Can we add `.mvn` to the git ignore to help prevent people from adding it to their PRs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rahil-c commented on a diff in pull request #6684: [DOCS] Add Documentation for running unit tests/debugger with spark3
Posted by GitBox <gi...@apache.org>.
rahil-c commented on code in PR #6684:
URL: https://github.com/apache/hudi/pull/6684#discussion_r973264735
##########
website/contribute/developer-setup.md:
##########
@@ -275,6 +275,18 @@ Discussion about contributing code to Hudi happens on the [dev@ mailing list](/c
This Quick Video will give a code walkthrough to start with [watch](https://www.youtube.com/watch?v=N2eDfU_rQ_U).
+## Running unit tests and local debugger via Intellij IDE
+
+0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
+1. Install the "Maven Helper" plugin from the Intellij IDE.
+2. Make sure IDEA uses Maven to build/run tests:
+ * You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
+ * Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
+3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org