You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2024/03/31 14:48:05 UTC
[PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
wombatu-kun opened a new pull request, #10942:
URL: https://github.com/apache/hudi/pull/10942
### Change Logs
Our constructor for user defined sort partitioner takes in write config, while some of the partitioners used in out of the box sort mode, does not account for it.
Lets fix the sort partitioners to ensure anything can be used as user defined partitioners.
For eg, NoneSortMode does not have a constructor that takes in write config
### Impact
none
### Risk level (write none, low medium or high below)
none
### Documentation Update
none
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1548807847
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
i think, yes it does. before the fix this partitioner had only default constructor, it was enough for it's instantiation and working, as it has no referencies to any writeconfig params
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545879218
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
Can you give an example how this partitioner got instantiated and customized?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545900885
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
This partitioner will be instantiated when user define write config property `hoodie.bulkinsert.user.defined.partitioner.class=org.apache.hudi.execution.bulkinsert.JavaGlobalSortPartitioner`. This constructor will be called via reflection in methods of DataSourceUtils class `createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)` and `createUserDefinedBulkInsertPartitionerWithRows(HoodieWriteConfig config)`:
`private static Option<BulkInsertPartitioner> createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)
throws HoodieException {
String bulkInsertPartitionerClass = config.getUserDefinedBulkInsertPartitionerClass();
try {
return StringUtils.isNullOrEmpty(bulkInsertPartitionerClass)
? Option.empty() :
Option.of((BulkInsertPartitioner) ReflectionUtils.loadClass(bulkInsertPartitionerClass, config));
} catch (Throwable e) {
throw new HoodieException("Could not create UserDefinedBulkInsertPartitioner class " + bulkInsertPartitionerClass, e);
}
}`
There is nothing to customize in this JavaGlobalSortPartitioner, but, for example, provided writeConfig is used for customization of RowSpatialCurveSortPartitioner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun closed pull request #10942: [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners
URL: https://github.com/apache/hudi/pull/10942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1549898777
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
@danny0405 can you please assign this PR to @nsivabalan to summon him to this discussion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2111593773
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060",
"triggerID" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"triggerType" : "PUSH"
}, {
"hash" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea11f68c1778f9ec23eab6a887076e51f60caa0b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060)
* 55fb13f601452d13f9ef2984e8ecef3ea95d7de6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2028820679
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060",
"triggerID" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea11f68c1778f9ec23eab6a887076e51f60caa0b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545958016
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
Yes, in this case HoodieWriteConfig is ignored just because this Partitioner is not configurable at all, but it does not mean that it should not be used as `UserDefinedBulkInsertPartitioner`.
So I think, the purpose of this task is not to make all BulkInsertPartitioners customizable with HoodieWriteConfig, but only to make them instantiable via reflection with already existing common approach for UserDefinedBulkInsertPartitioner (constructor with HoodieWriteConfig as the only parameter).
@nsivabalan am I right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1546190914
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
Before this fix:
if user wants to use JavaGlobalSortPartitioner and he set `hoodie.bulkinsert.user.defined.partitioner.class=org.apache.hudi.execution.bulkinsert.JavaGlobalSortPartitioner`, it will not work because this partitioner could not be instantiated via reflection (as it has no constructor with writeConfig parameter).
We create this constructor to add ability to use JavaGlobalSortPartitioner as user defined partitioner just by setting it's class name in writeConfig.
Don't know how to explain more clear. Let's wait for the author's reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545900885
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
This partitioner will be instantiated when user define write config property `hoodie.bulkinsert.user.defined.partitioner.class=org.apache.hudi.execution.bulkinsert.JavaGlobalSortPartitioner`. This constructor will be called via reflection in DataSourceUtils class: `createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)` and `createUserDefinedBulkInsertPartitionerWithRows(HoodieWriteConfig config)`.
There is nothing to customize in this JavaGlobalSortPartitioner, but, for example, provided writeConfig is used for customization of RowSpatialCurveSortPartitioner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545945929
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
I got confused because the "customized" `HoodieWriteConfig` does not really play a role here and it is ignored?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1545900885
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
This partitioner will be instantiated when user define write config property `hoodie.bulkinsert.user.defined.partitioner.class=org.apache.hudi.execution.bulkinsert.JavaGlobalSortPartitioner`.
This constructor will be called via reflection in methods of DataSourceUtils class `createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)` and `createUserDefinedBulkInsertPartitionerWithRows(HoodieWriteConfig config)`.
There is nothing to customize in this JavaGlobalSortPartitioner, but, for example, provided writeConfig is used for customization of RowSpatialCurveSortPartitioner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1546174541
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
> Yes, in this case HoodieWriteConfig is ignored just because this Partitioner is not configurable at all, but it does not mean that it should not be used as UserDefinedBulkInsertPartitioner
That does not make sense for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2111635895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060",
"triggerID" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"triggerType" : "PUSH"
}, {
"hash" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23943",
"triggerID" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea11f68c1778f9ec23eab6a887076e51f60caa0b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060)
* 55fb13f601452d13f9ef2984e8ecef3ea95d7de6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23943)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2058119247
@nsivabalan Hi! Sorry to bother you, but you are reporter of this task. Could you please review my PR?
Or close the PR if i totally misunderstood the task and did it wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2028787238
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea11f68c1778f9ec23eab6a887076e51f60caa0b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10942:
URL: https://github.com/apache/hudi/pull/10942#discussion_r1548797462
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaGlobalSortPartitioner.java:
##########
@@ -31,12 +32,21 @@
*
* @param <T> HoodieRecordPayload type
*/
-public class JavaGlobalSortPartitioner<T>
- implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+public class JavaGlobalSortPartitioner<T> implements BulkInsertPartitioner<List<HoodieRecord<T>>> {
+
+ public JavaGlobalSortPartitioner() {
+ }
+
+ /**
+ * Constructor to create as UserDefinedBulkInsertPartitioner class via reflection
+ * @param config HoodieWriteConfig
Review Comment:
> We create this constructor to add ability to use JavaGlobalSortPartitioner as user defined partitioner just by setting it's class name in writeConfig.
But does it work correctly with the write config being igored?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7526] Fix constructors for bulkinsert sort partitioners to ensure we could use it as user defined partitioners [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10942:
URL: https://github.com/apache/hudi/pull/10942#issuecomment-2111711975
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23060",
"triggerID" : "ea11f68c1778f9ec23eab6a887076e51f60caa0b",
"triggerType" : "PUSH"
}, {
"hash" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23943",
"triggerID" : "55fb13f601452d13f9ef2984e8ecef3ea95d7de6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55fb13f601452d13f9ef2984e8ecef3ea95d7de6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23943)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org