You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/05 20:58:39 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request, #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
alexeykudinkin opened a new pull request, #6049:
URL: https://github.com/apache/hudi/pull/6049
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
Currently when doing bulk-insert using partition paths with slashes in it's being laid out incorrectly missing URL-encoding for the partition path, even though it's set to true.
This fix is purely a duct-tape until it's properly addressed by HUDI-3993
## Brief change log
See above
## Verify this pull request
This pull request is already covered by existing tests, such as *(please describe tests)*.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r919345001
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/internal/BulkInsertDataInternalWriterHelper.java:
##########
@@ -126,14 +125,17 @@ public void write(InternalRow record) throws IOException {
if (populateMetaFields) { // usual path where meta fields are pre populated in prep step.
partitionPath = String.valueOf(record.getUTF8String(HoodieRecord.PARTITION_PATH_META_FIELD_POS));
} else { // if meta columns are disabled.
+ // TODO(HUDI-3993) remove duplication, unify with HoodieDatasetBulkInsertHelper
if (!keyGeneratorOpt.isPresent()) { // NoPartitionerKeyGen
partitionPath = "";
} else if (simpleKeyGen) { // SimpleKeyGen
- Object parititionPathValue = record.get(simplePartitionFieldIndex, simplePartitionFieldDataType);
- partitionPath = parititionPathValue != null ? parititionPathValue.toString() : PartitionPathEncodeUtils.DEFAULT_PARTITION_PATH;
- if (writeConfig.isHiveStylePartitioningEnabled()) {
- partitionPath = (keyGeneratorOpt.get()).getPartitionPathFields().get(0) + "=" + partitionPath;
- }
+ Object partitionPathValue = record.get(simplePartitionFieldIndex, simplePartitionFieldDataType);
+ String partitionPathField = keyGeneratorOpt.get().getPartitionPathFields().get(0);
+ boolean shouldURLEncodePartitionPath = writeConfig.shouldURLEncodePartitionPath();
+ boolean hiveStylePartitioningEnabled = writeConfig.isHiveStylePartitioningEnabled();
+
+ partitionPath = KeyGenUtils.handlePartitionPathDecoration(partitionPathField,
+ partitionPathValue == null ? null : partitionPathValue.toString(), shouldURLEncodePartitionPath, hiveStylePartitioningEnabled);
Review Comment:
Yeah, done that initially but then decided to optimize it out given that this is a hot-path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182639408
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175570125
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9a917927cf0339a0271e04d474e91ac89254c8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736)
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182652664
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin closed pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin closed pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
URL: https://github.com/apache/hudi/pull/6049
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182663769
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TengHuo commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
TengHuo commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r920877138
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java:
##########
@@ -97,8 +96,31 @@ public static Dataset<Row> prepareHoodieDatasetForBulkInsert(SQLContext sqlConte
// simple fields for both record key and partition path: can directly use withColumn
String partitionPathField = keyGeneratorClass.equals(SimpleKeyGenerator.class.getName()) ? partitionPathFields :
partitionPathFields.substring(partitionPathFields.indexOf(":") + 1);
- rowDatasetWithRecordKeysAndPartitionPath = rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
- .withColumn(HoodieRecord.PARTITION_PATH_METADATA_FIELD, functions.col(partitionPathField).cast(DataTypes.StringType));
+
+ // TODO(HUDI-3993) cleanup duplication
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String partitionPathDecorationUDFName = PARTITION_PATH_UDF_FN + tableName;
+
+ boolean shouldURLEncodePartitionPath = config.shouldURLEncodePartitionPath();
+ boolean isHiveStylePartitioned = config.isHiveStylePartitioningEnabled();
+
+ if (shouldURLEncodePartitionPath || isHiveStylePartitioned) {
+ sqlContext.udf().register(
+ partitionPathDecorationUDFName,
+ (UDF1<String, String>) partitionPathValue ->
+ KeyGenUtils.handlePartitionPathDecoration(partitionPathField, partitionPathValue,
+ shouldURLEncodePartitionPath, isHiveStylePartitioned),
+ DataTypes.StringType);
+
+ rowDatasetWithRecordKeysAndPartitionPath =
+ rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
Review Comment:
In `ComplexKeyGenerator.getRecordKey(Row row)`, we setup `prefixFieldName` as `true` in method `RowKeyGeneratorHelper.getRecordKeyFromRow(row, getRecordKeyFields(), recordKeySchemaInfo, true)`, so the record key will have a prefix, which is record key field name, when we use `ComplexKeyGenerator`.
As I understand here, we use `withColumn` here for `recordKeyFields`, then we will get the same value in `RECORD_KEY_METADATA_FIELD` as the original `recordKeyFields`, so no prefix when key generator is `ComplexKeyGenerator`. Will it cause problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r922681411
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java:
##########
@@ -97,8 +96,31 @@ public static Dataset<Row> prepareHoodieDatasetForBulkInsert(SQLContext sqlConte
// simple fields for both record key and partition path: can directly use withColumn
String partitionPathField = keyGeneratorClass.equals(SimpleKeyGenerator.class.getName()) ? partitionPathFields :
partitionPathFields.substring(partitionPathFields.indexOf(":") + 1);
- rowDatasetWithRecordKeysAndPartitionPath = rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
- .withColumn(HoodieRecord.PARTITION_PATH_METADATA_FIELD, functions.col(partitionPathField).cast(DataTypes.StringType));
+
+ // TODO(HUDI-3993) cleanup duplication
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String partitionPathDecorationUDFName = PARTITION_PATH_UDF_FN + tableName;
+
+ boolean shouldURLEncodePartitionPath = config.shouldURLEncodePartitionPath();
+ boolean isHiveStylePartitioned = config.isHiveStylePartitioningEnabled();
+
+ if (shouldURLEncodePartitionPath || isHiveStylePartitioned) {
+ sqlContext.udf().register(
+ partitionPathDecorationUDFName,
+ (UDF1<String, String>) partitionPathValue ->
+ KeyGenUtils.handlePartitionPathDecoration(partitionPathField, partitionPathValue,
+ shouldURLEncodePartitionPath, isHiveStylePartitioned),
+ DataTypes.StringType);
+
+ rowDatasetWithRecordKeysAndPartitionPath =
+ rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
Review Comment:
@TengHuo That's a good point. It can be a problem if you mix the write operation type or switch row-writing config for a table. I would suggest filing another JIRA ticket to keep it consistent across. I don't deem it to be a blocker but would be good to keep it consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175515606
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9a917927cf0339a0271e04d474e91ac89254c8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1176850783
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182549868
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1184083526
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1183673241",
"triggerType" : "MANUAL"
}, {
"hash" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
* be896fedc6cef9d4f2f7529cdd31a18b98985f94 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1184087872
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1183673241",
"triggerType" : "MANUAL"
}, {
"hash" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9909",
"triggerID" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
* be896fedc6cef9d4f2f7529cdd31a18b98985f94 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9909)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175573690
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9a917927cf0339a0271e04d474e91ac89254c8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736)
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1183687424
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1183673241",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175512074
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9a917927cf0339a0271e04d474e91ac89254c8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175632658
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r917266599
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileSystemTestUtils.java:
##########
@@ -44,7 +44,7 @@ public class FileSystemTestUtils {
public static final String FORWARD_SLASH = "/";
public static final String FILE_SCHEME = "file";
public static final String COLON = ":";
- public static final Random RANDOM = new Random();
+ public static final Random RANDOM = new Random(0xDEED);
Review Comment:
+1
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/internal/BulkInsertDataInternalWriterHelper.java:
##########
@@ -126,14 +125,17 @@ public void write(InternalRow record) throws IOException {
if (populateMetaFields) { // usual path where meta fields are pre populated in prep step.
partitionPath = String.valueOf(record.getUTF8String(HoodieRecord.PARTITION_PATH_META_FIELD_POS));
} else { // if meta columns are disabled.
+ // TODO(HUDI-3993) remove duplication, unify with HoodieDatasetBulkInsertHelper
if (!keyGeneratorOpt.isPresent()) { // NoPartitionerKeyGen
partitionPath = "";
} else if (simpleKeyGen) { // SimpleKeyGen
- Object parititionPathValue = record.get(simplePartitionFieldIndex, simplePartitionFieldDataType);
- partitionPath = parititionPathValue != null ? parititionPathValue.toString() : PartitionPathEncodeUtils.DEFAULT_PARTITION_PATH;
- if (writeConfig.isHiveStylePartitioningEnabled()) {
- partitionPath = (keyGeneratorOpt.get()).getPartitionPathFields().get(0) + "=" + partitionPath;
- }
+ Object partitionPathValue = record.get(simplePartitionFieldIndex, simplePartitionFieldDataType);
+ String partitionPathField = keyGeneratorOpt.get().getPartitionPathFields().get(0);
+ boolean shouldURLEncodePartitionPath = writeConfig.shouldURLEncodePartitionPath();
+ boolean hiveStylePartitioningEnabled = writeConfig.isHiveStylePartitioningEnabled();
+
+ partitionPath = KeyGenUtils.handlePartitionPathDecoration(partitionPathField,
+ partitionPathValue == null ? null : partitionPathValue.toString(), shouldURLEncodePartitionPath, hiveStylePartitioningEnabled);
Review Comment:
Consider passing `Option.of(partitionPathValue)` instead of `null`
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java:
##########
@@ -97,8 +96,31 @@ public static Dataset<Row> prepareHoodieDatasetForBulkInsert(SQLContext sqlConte
// simple fields for both record key and partition path: can directly use withColumn
String partitionPathField = keyGeneratorClass.equals(SimpleKeyGenerator.class.getName()) ? partitionPathFields :
partitionPathFields.substring(partitionPathFields.indexOf(":") + 1);
- rowDatasetWithRecordKeysAndPartitionPath = rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
- .withColumn(HoodieRecord.PARTITION_PATH_METADATA_FIELD, functions.col(partitionPathField).cast(DataTypes.StringType));
+
+ // TODO(HUDI-3993) cleanup duplication
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String partitionPathDecorationUDFName = PARTITION_PATH_UDF_FN + tableName;
+
+ boolean shouldURLEncodePartitionPath = config.shouldURLEncodePartitionPath();
+ boolean isHiveStylePartitioned = config.isHiveStylePartitioningEnabled();
+
+ if (shouldURLEncodePartitionPath || isHiveStylePartitioned) {
+ sqlContext.udf().register(
+ partitionPathDecorationUDFName,
+ (UDF1<String, String>) partitionPathValue ->
Review Comment:
So, I assume this UDF registration would be gone after HUDI-3993?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] TengHuo commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
TengHuo commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r923259601
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java:
##########
@@ -97,8 +96,31 @@ public static Dataset<Row> prepareHoodieDatasetForBulkInsert(SQLContext sqlConte
// simple fields for both record key and partition path: can directly use withColumn
String partitionPathField = keyGeneratorClass.equals(SimpleKeyGenerator.class.getName()) ? partitionPathFields :
partitionPathFields.substring(partitionPathFields.indexOf(":") + 1);
- rowDatasetWithRecordKeysAndPartitionPath = rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
- .withColumn(HoodieRecord.PARTITION_PATH_METADATA_FIELD, functions.col(partitionPathField).cast(DataTypes.StringType));
+
+ // TODO(HUDI-3993) cleanup duplication
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String partitionPathDecorationUDFName = PARTITION_PATH_UDF_FN + tableName;
+
+ boolean shouldURLEncodePartitionPath = config.shouldURLEncodePartitionPath();
+ boolean isHiveStylePartitioned = config.isHiveStylePartitioningEnabled();
+
+ if (shouldURLEncodePartitionPath || isHiveStylePartitioned) {
+ sqlContext.udf().register(
+ partitionPathDecorationUDFName,
+ (UDF1<String, String>) partitionPathValue ->
+ KeyGenUtils.handlePartitionPathDecoration(partitionPathField, partitionPathValue,
+ shouldURLEncodePartitionPath, isHiveStylePartitioned),
+ DataTypes.StringType);
+
+ rowDatasetWithRecordKeysAndPartitionPath =
+ rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
Review Comment:
okay, got it
then think it will have duplicate data issue if user upgrade from 0.10 or older version when they only setup one column as record key and use `ComplexKeyGenerator `. The same issue as upgrading from 0.10 to 0.11.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1190840668
Closing in favor of https://github.com/apache/hudi/pull/5523
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1176848367
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1183673241
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1183691071
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1183673241",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182661811
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6049:
URL: https://github.com/apache/hudi/pull/6049#discussion_r919344127
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java:
##########
@@ -97,8 +96,31 @@ public static Dataset<Row> prepareHoodieDatasetForBulkInsert(SQLContext sqlConte
// simple fields for both record key and partition path: can directly use withColumn
String partitionPathField = keyGeneratorClass.equals(SimpleKeyGenerator.class.getName()) ? partitionPathFields :
partitionPathFields.substring(partitionPathFields.indexOf(":") + 1);
- rowDatasetWithRecordKeysAndPartitionPath = rows.withColumn(HoodieRecord.RECORD_KEY_METADATA_FIELD, functions.col(recordKeyFields).cast(DataTypes.StringType))
- .withColumn(HoodieRecord.PARTITION_PATH_METADATA_FIELD, functions.col(partitionPathField).cast(DataTypes.StringType));
+
+ // TODO(HUDI-3993) cleanup duplication
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String partitionPathDecorationUDFName = PARTITION_PATH_UDF_FN + tableName;
+
+ boolean shouldURLEncodePartitionPath = config.shouldURLEncodePartitionPath();
+ boolean isHiveStylePartitioned = config.isHiveStylePartitioningEnabled();
+
+ if (shouldURLEncodePartitionPath || isHiveStylePartitioned) {
+ sqlContext.udf().register(
+ partitionPathDecorationUDFName,
+ (UDF1<String, String>) partitionPathValue ->
Review Comment:
Correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182637413
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1184144477
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182637413",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1182652664",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "1183673241",
"triggerType" : "MANUAL"
}, {
"hash" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9909",
"triggerID" : "be896fedc6cef9d4f2f7529cdd31a18b98985f94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* be896fedc6cef9d4f2f7529cdd31a18b98985f94 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9909)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182552338
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1182559454
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
}, {
"hash" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875",
"triggerID" : "06d50fd9a820bc7f95f2f139f69dea97c1e3f06b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 06d50fd9a820bc7f95f2f139f69dea97c1e3f06b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9875)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1175599033
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1c9a917927cf0339a0271e04d474e91ac89254c8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736)
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6049: [HUDI-4365] Fixing URL-encoding in Bulk Insert row-writing path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6049:
URL: https://github.com/apache/hudi/pull/6049#issuecomment-1176853063
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9736",
"triggerID" : "1c9a917927cf0339a0271e04d474e91ac89254c8",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"triggerType" : "PUSH"
}, {
"hash" : "48dc9f122fb2e119ed8784cdd1bd8a8412e633bf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737",
"triggerID" : "1176848367",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 48dc9f122fb2e119ed8784cdd1bd8a8412e633bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9737)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org