You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "HuangZhenQiu (via GitHub)" <gi...@apache.org> on 2025/04/12 00:04:23 UTC
[PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
HuangZhenQiu opened a new pull request, #13137:
URL: https://github.com/apache/hudi/pull/13137
### Change Logs
1. Add a new storage config to hoodie.parquet.flink.rowdata.write.support.class
2. Use reflection to load the class with default constructor
### Impact
none
### Risk level (write none, low medium or high below)
none
### Documentation Update
None
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798374684
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a6457e592fc1be6e376daac65d30a0e9692c38a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13137:
URL: https://github.com/apache/hudi/pull/13137#discussion_r2040515305
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieStorageConfig.java:
##########
@@ -245,6 +245,14 @@ public class HoodieStorageConfig extends HoodieConfig {
+ "and it is loaded at runtime. This is only required when trying to "
+ "override the existing write context when `hoodie.datasource.write.row.writer.enable=true`.");
+ public static final ConfigProperty<String> HOODIE_PARQUET_FLINK_ROW_DATA_WRITE_SUPPORT_CLASS = ConfigProperty
+ .key("hoodie.parquet.flink.rowdata.write.support.class")
+ .defaultValue("org.apache.hudi.io.storage.row.HoodieRowDataParquetWriteSupport")
+ .markAdvanced()
+ .sinceVersion("1.0.1")
Review Comment:
Did you mean 1.1.0? 1.0.1 has been released out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "HuangZhenQiu (via GitHub)" <gi...@apache.org>.
HuangZhenQiu commented on code in PR #13137:
URL: https://github.com/apache/hudi/pull/13137#discussion_r2040521932
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieStorageConfig.java:
##########
@@ -245,6 +245,14 @@ public class HoodieStorageConfig extends HoodieConfig {
+ "and it is loaded at runtime. This is only required when trying to "
+ "override the existing write context when `hoodie.datasource.write.row.writer.enable=true`.");
+ public static final ConfigProperty<String> HOODIE_PARQUET_FLINK_ROW_DATA_WRITE_SUPPORT_CLASS = ConfigProperty
+ .key("hoodie.parquet.flink.rowdata.write.support.class")
+ .defaultValue("org.apache.hudi.io.storage.row.HoodieRowDataParquetWriteSupport")
+ .markAdvanced()
+ .sinceVersion("1.0.1")
Review Comment:
Good catch. It is my mistake.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798408891
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
}, {
"hash" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a6457e592fc1be6e376daac65d30a0e9692c38a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821)
* 9da3ec31848c72d06db95bb043d237f151079a8d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798240109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a6457e592fc1be6e376daac65d30a0e9692c38a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798409494
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
}, {
"hash" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4822",
"triggerID" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a6457e592fc1be6e376daac65d30a0e9692c38a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821)
* 9da3ec31848c72d06db95bb043d237f151079a8d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4822)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798238895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a6457e592fc1be6e376daac65d30a0e9692c38a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13137:
URL: https://github.com/apache/hudi/pull/13137#issuecomment-2798450363
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4821",
"triggerID" : "5a6457e592fc1be6e376daac65d30a0e9692c38a",
"triggerType" : "PUSH"
}, {
"hash" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4822",
"triggerID" : "9da3ec31848c72d06db95bb043d237f151079a8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9da3ec31848c72d06db95bb043d237f151079a8d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4822)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9304] support configurable RowDataParquetWriteSupport for Flink [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #13137:
URL: https://github.com/apache/hudi/pull/13137
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org