You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jonvex (via GitHub)" <gi...@apache.org> on 2023/05/08 20:12:31 UTC
[GitHub] [hudi] jonvex opened a new pull request, #8666: [HUDI-915] Add missing partititonpath to records COW
jonvex opened a new pull request, #8666:
URL: https://github.com/apache/hudi/pull/8666
### Change Logs
In a bootstrapped table, If there was an upsert in a file group, the records in the file group that were not upserted would have null values for their partition column
### Impact
Now works correctly
### Risk level (write none, low medium or high below)
Medium
Lots of tests written
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1538998123
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7510bceed906d98b31bf44b4cd78a24a364c1288 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194307177
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/BaseMergeHelper.java:
##########
@@ -36,7 +37,7 @@ public abstract class BaseMergeHelper {
* @param upsertHandle Merge Handle
* @throws IOException in case of error
*/
- public abstract void runMerge(HoodieTable<?, ?, ?, ?> table, HoodieMergeHandle<?, ?, ?, ?> upsertHandle) throws IOException;
+ public abstract void runMerge(HoodieTable<?, ?, ?, ?> table, HoodieMergeHandle<?, ?, ?, ?> upsertHandle, Option<String[]> partitionFields, Object[] partitionValues) throws IOException;
Review Comment:
I agree that we shouldn't add the new params to the method. Would it be ok to add a fields to the merge handle and set those
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1543364782
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 66dd335158e2800359039917a9c1690450fa9c27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998)
* bfa1909d85b174cb466d205ee4ec2af09be1850d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1543402111
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 66dd335158e2800359039917a9c1690450fa9c27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998)
* bfa1909d85b174cb466d205ee4ec2af09be1850d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015)
* 840cfab05cacd7a4862b2ad9a1983af2953819a1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1543359810
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 66dd335158e2800359039917a9c1690450fa9c27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998)
* bfa1909d85b174cb466d205ee4ec2af09be1850d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1544703628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194314725
##########
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieBootstrapRecordIterator.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.model.MetadataValues;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.collection.ClosableIterator;
+
+import org.apache.avro.Schema;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
+
+public abstract class HoodieBootstrapRecordIterator<T> implements ClosableIterator<HoodieRecord<T>> {
+
+ protected ClosableIterator<HoodieRecord<T>> skeletonIterator;
+ protected ClosableIterator<HoodieRecord<T>> dataFileIterator;
+ private final Option<String[]> partitionFields;
+ private final Object[] partitionValues;
+
+ protected Schema schema;
+
+ public HoodieBootstrapRecordIterator(ClosableIterator<HoodieRecord<T>> skeletonIterator,
+ ClosableIterator<HoodieRecord<T>> dataFileIterator,
+ Schema schema,
+ Option<String[]> partitionFields,
+ Object[] partitionValues) {
+ this.skeletonIterator = skeletonIterator;
+ this.dataFileIterator = dataFileIterator;
+ this.schema = schema;
+ this.partitionFields = partitionFields;
+ this.partitionValues = partitionValues;
+ }
+
+ @Override
+ public void close() {
+
+ }
+
+ @Override
+ public boolean hasNext() {
+ checkState(skeletonIterator.hasNext() == dataFileIterator.hasNext());
+ return skeletonIterator.hasNext();
+ }
+
+ @Override
+ public HoodieRecord<T> next() {
+ HoodieRecord<T> dataRecord = dataFileIterator.next();
+ HoodieRecord<T> skeletonRecord = skeletonIterator.next();
+ HoodieRecord<T> ret = dataRecord.prependMetaFields(schema, schema,
+ new MetadataValues().setCommitTime(skeletonRecord.getRecordKey(schema, HoodieRecord.COMMIT_TIME_METADATA_FIELD))
+ .setCommitSeqno(skeletonRecord.getRecordKey(schema, HoodieRecord.COMMIT_SEQNO_METADATA_FIELD))
+ .setRecordKey(skeletonRecord.getRecordKey(schema, HoodieRecord.RECORD_KEY_METADATA_FIELD))
+ .setPartitionPath(skeletonRecord.getRecordKey(schema, HoodieRecord.PARTITION_PATH_METADATA_FIELD))
+ .setFileName(skeletonRecord.getRecordKey(schema, HoodieRecord.FILENAME_METADATA_FIELD)), null);
+ if (partitionFields.isPresent()) {
+ for (int i = 0; i < partitionValues.length; i++) {
+ int position = schema.getField(partitionFields.get()[i]).pos();
+ setThePartitionField(position, partitionValues[i], ret.getData());
+ }
+ }
+ return ret;
+ }
+
+ protected abstract void setThePartitionField(int position, Object fieldValue, T row);
Review Comment:
I originally had it as that, but it was getting confused when calling the setPartitionField method when implementing what is now setThePartitionField in the bootstrap reader. It thought I was doing a recursive call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1195160560
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestBootstrapRead.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.functional;
+
+import org.apache.hudi.DataSourceReadOptions;
+import org.apache.hudi.DataSourceWriteOptions;
+import org.apache.hudi.client.bootstrap.selector.BootstrapRegexModeSelector;
+import org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector;
+import org.apache.hudi.client.bootstrap.selector.MetadataOnlyBootstrapModeSelector;
+import org.apache.hudi.common.testutils.HoodieTestDataGenerator;
+import org.apache.hudi.config.HoodieBootstrapConfig;
+import org.apache.hudi.config.HoodieCompactionConfig;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.keygen.ComplexKeyGenerator;
+import org.apache.hudi.keygen.NonpartitionedKeyGenerator;
+import org.apache.hudi.keygen.SimpleKeyGenerator;
+import org.apache.hudi.testutils.HoodieSparkClientTestBase;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.apache.spark.sql.SaveMode;
+import org.apache.spark.sql.functions;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static org.apache.hudi.common.testutils.RawTripTestPayload.recordToString;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Tests different layouts for bootstrap base path
+ */
+@Tag("functional")
+public class TestBootstrapRead extends HoodieSparkClientTestBase {
+
+ @TempDir
+ public java.nio.file.Path tmpFolder;
+ protected String bootstrapBasePath = null;
+ protected String bootstrapTargetPath = null;
+ protected String hudiBasePath = null;
+
+ protected static int nInserts = 100;
+ protected static int nUpdates = 20;
+ protected static String[] dashPartitionPaths = {"2016-03-15", "2015-03-16", "2015-03-17"};
+ protected static String[] slashPartitionPaths = {"2016/03/15", "2015/03/16", "2015/03/17"};
+ protected String bootstrapType;
+ protected Boolean dashPartitions;
+ protected String tableType;
+ protected Integer nPartitions;
+
+ protected String[] partitionCols;
+ protected static String[] dropColumns = {"_hoodie_commit_time", "_hoodie_commit_seqno", "_hoodie_record_key", "_hoodie_file_name", "city_to_state"};
+
+ @BeforeEach
+ public void setUp() throws Exception {
+ bootstrapBasePath = tmpFolder.toAbsolutePath() + "/bootstrapBasePath";
+ hudiBasePath = tmpFolder.toAbsolutePath() + "/hudiBasePath";
+ bootstrapTargetPath = tmpFolder.toAbsolutePath() + "/bootstrapTargetPath";
+ initSparkContexts();
+ }
+
+ @AfterEach
+ public void tearDown() throws IOException {
+ cleanupSparkContexts();
+ cleanupClients();
+ cleanupTestDataGenerator();
+ }
+
+ private static Stream<Arguments> testArgs() {
+ Stream.Builder<Arguments> b = Stream.builder();
+ //TODO: add dash partitions false with [HUDI-4944]
Review Comment:
I changed the line below to:
Boolean[] dashPartitions = {true/*,false*/};
The code is already there to test that, it just doesn't work currently because of the double decoding issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194311447
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/PartitionUtils.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.client.bootstrap;
+
+import org.apache.hudi.AvroConversionUtils;
+import org.apache.hudi.HoodieSparkUtils;
+import org.apache.hudi.SparkAdapterSupport$;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.hadoop.CachingPath;
+
+import org.apache.avro.Schema;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.spark.sql.execution.datasources.SparkParsePartitionUtil;
+import org.apache.spark.sql.internal.SQLConf;
+
+public class PartitionUtils {
+
+ public static Object[] getPartitionFieldVals(Option<String[]> partitionFields,
+ String partitionPath,
+ String bootstrapBasePath,
+ Schema writerSchema,
+ Configuration hadoopConf) {
+ if (!partitionFields.isPresent()) {
+ return new Object[0];
+ }
+ SparkParsePartitionUtil sparkParsePartitionUtil = SparkAdapterSupport$.MODULE$.sparkAdapter().getSparkParsePartitionUtil();
+ return HoodieSparkUtils.parsePartitionColumnValues(
+ partitionFields.get(),
+ partitionPath,
+ new CachingPath(bootstrapBasePath),
Review Comment:
I got rid of the orElse. Since it is only called in the if block that `upsertHandle.baseFileForMerge().getBootstrapBaseFile().isPresent()`, it means that there will always be a bootstrap base path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1548552010
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cc420c8790dc736da431714aa78750be1c502a9d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087)
* 62deeb26c94c9a073330488e1c7ea774dd5c7147 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092)
* c20d2780e542b92fbf95a2a3fb2ab94512943780 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1545849363
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024)
* bb7e42e3225e97863cb3e96953784385e19ec638 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1546064468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb7e42e3225e97863cb3e96953784385e19ec638 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1548561788
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 62deeb26c94c9a073330488e1c7ea774dd5c7147 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092)
* c20d2780e542b92fbf95a2a3fb2ab94512943780 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1539080730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7510bceed906d98b31bf44b4cd78a24a364c1288 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955)
* 5d1b90a6e91fbfe1229556377831d0c52d9c7613 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1539008630
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7510bceed906d98b31bf44b4cd78a24a364c1288 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1550400018
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
}, {
"hash" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17127",
"triggerID" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6ba0740939112a80201fdaacaa2b2a1e383d1c85 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17127)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1543409451
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bfa1909d85b174cb466d205ee4ec2af09be1850d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015)
* 840cfab05cacd7a4862b2ad9a1983af2953819a1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1544489175
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 840cfab05cacd7a4862b2ad9a1983af2953819a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016)
* 1b2f28447ac507b35f82a0534ebd958a8fd8980d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1542624333
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 66dd335158e2800359039917a9c1690450fa9c27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1549713227
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
}, {
"hash" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c20d2780e542b92fbf95a2a3fb2ab94512943780 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099)
* 6ba0740939112a80201fdaacaa2b2a1e383d1c85 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope merged PR #8666:
URL: https://github.com/apache/hudi/pull/8666
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1539069610
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7510bceed906d98b31bf44b4cd78a24a364c1288 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955)
* 5d1b90a6e91fbfe1229556377831d0c52d9c7613 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1542404565
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d1b90a6e91fbfe1229556377831d0c52d9c7613 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956)
* 66dd335158e2800359039917a9c1690450fa9c27 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1549732276
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
}, {
"hash" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17127",
"triggerID" : "6ba0740939112a80201fdaacaa2b2a1e383d1c85",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c20d2780e542b92fbf95a2a3fb2ab94512943780 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099)
* 6ba0740939112a80201fdaacaa2b2a1e383d1c85 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17127)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194170081
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/HoodieFlinkCopyOnWriteTable.java:
##########
@@ -371,7 +371,7 @@ protected Iterator<List<WriteStatus>> handleUpdateInternal(HoodieMergeHandle<?,
throw new HoodieUpsertException(
"Error in finding the old file path at commit " + instantTime + " for fileId: " + fileId);
} else {
- HoodieMergeHelper.newInstance().runMerge(this, upsertHandle);
+ HoodieMergeHelper.newInstance().runMerge(this, upsertHandle, Option.empty(), new Object[0]);
Review Comment:
The problem is that we have been using spark to get those values and MergeHelper is supposed to be engine agnostic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1548214753
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cc420c8790dc736da431714aa78750be1c502a9d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087)
* 62deeb26c94c9a073330488e1c7ea774dd5c7147 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1543738560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 840cfab05cacd7a4862b2ad9a1983af2953819a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194322685
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -71,13 +71,21 @@ public static HoodieTableMetaClient init(String basePath, HoodieTableType tableT
return init(getDefaultHadoopConf(), basePath, tableType, properties);
}
- public static HoodieTableMetaClient init(String basePath, HoodieTableType tableType, String bootstrapBasePath, boolean bootstrapIndexEnable) throws IOException {
+ public static HoodieTableMetaClient init(String basePath, HoodieTableType tableType, String bootstrapBasePath, boolean bootstrapIndexEnable, String keyGenerator) throws IOException {
Properties props = new Properties();
props.setProperty(HoodieTableConfig.BOOTSTRAP_BASE_PATH.key(), bootstrapBasePath);
props.put(HoodieTableConfig.BOOTSTRAP_INDEX_ENABLE.key(), bootstrapIndexEnable);
+ if (keyGenerator != null) {
+ props.put("hoodie.datasource.write.keygenerator.class", keyGenerator);
+ props.put("hoodie.datasource.write.partitionpath.field", "datestr");
+ }
Review Comment:
1. I needed to do this because https://github.com/apache/hudi/pull/6016 sets the partitionpath incorrectly. I made the change this way so that it doesn't affect other tests.
2. This is in hoodie common and writeconfigs are in hoodie client common so we can't add that dependency. And again, I wouldn't need to do this in the first place if that other pr didn't modify the test utils to set configs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194957406
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/BaseMergeHelper.java:
##########
@@ -36,7 +37,7 @@ public abstract class BaseMergeHelper {
* @param upsertHandle Merge Handle
* @throws IOException in case of error
*/
- public abstract void runMerge(HoodieTable<?, ?, ?, ?> table, HoodieMergeHandle<?, ?, ?, ?> upsertHandle) throws IOException;
+ public abstract void runMerge(HoodieTable<?, ?, ?, ?> table, HoodieMergeHandle<?, ?, ?, ?> upsertHandle, Option<String[]> partitionFields, Object[] partitionValues) throws IOException;
Review Comment:
Yeah I think that's fine.
##########
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieBootstrapRecordIterator.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.model.MetadataValues;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.collection.ClosableIterator;
+
+import org.apache.avro.Schema;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
+
+public abstract class HoodieBootstrapRecordIterator<T> implements ClosableIterator<HoodieRecord<T>> {
+
+ protected ClosableIterator<HoodieRecord<T>> skeletonIterator;
+ protected ClosableIterator<HoodieRecord<T>> dataFileIterator;
+ private final Option<String[]> partitionFields;
+ private final Object[] partitionValues;
+
+ protected Schema schema;
+
+ public HoodieBootstrapRecordIterator(ClosableIterator<HoodieRecord<T>> skeletonIterator,
+ ClosableIterator<HoodieRecord<T>> dataFileIterator,
+ Schema schema,
+ Option<String[]> partitionFields,
+ Object[] partitionValues) {
+ this.skeletonIterator = skeletonIterator;
+ this.dataFileIterator = dataFileIterator;
+ this.schema = schema;
+ this.partitionFields = partitionFields;
+ this.partitionValues = partitionValues;
+ }
+
+ @Override
+ public void close() {
+
+ }
+
+ @Override
+ public boolean hasNext() {
+ checkState(skeletonIterator.hasNext() == dataFileIterator.hasNext());
+ return skeletonIterator.hasNext();
+ }
+
+ @Override
+ public HoodieRecord<T> next() {
+ HoodieRecord<T> dataRecord = dataFileIterator.next();
+ HoodieRecord<T> skeletonRecord = skeletonIterator.next();
+ HoodieRecord<T> ret = dataRecord.prependMetaFields(schema, schema,
+ new MetadataValues().setCommitTime(skeletonRecord.getRecordKey(schema, HoodieRecord.COMMIT_TIME_METADATA_FIELD))
+ .setCommitSeqno(skeletonRecord.getRecordKey(schema, HoodieRecord.COMMIT_SEQNO_METADATA_FIELD))
+ .setRecordKey(skeletonRecord.getRecordKey(schema, HoodieRecord.RECORD_KEY_METADATA_FIELD))
+ .setPartitionPath(skeletonRecord.getRecordKey(schema, HoodieRecord.PARTITION_PATH_METADATA_FIELD))
+ .setFileName(skeletonRecord.getRecordKey(schema, HoodieRecord.FILENAME_METADATA_FIELD)), null);
+ if (partitionFields.isPresent()) {
+ for (int i = 0; i < partitionValues.length; i++) {
+ int position = schema.getField(partitionFields.get()[i]).pos();
+ setThePartitionField(position, partitionValues[i], ret.getData());
+ }
+ }
+ return ret;
+ }
+
+ protected abstract void setThePartitionField(int position, Object fieldValue, T row);
Review Comment:
Ok then rename to `setPartitionPathField`? Typically, we don't use articles in variables, method or class names.
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -71,13 +71,21 @@ public static HoodieTableMetaClient init(String basePath, HoodieTableType tableT
return init(getDefaultHadoopConf(), basePath, tableType, properties);
}
- public static HoodieTableMetaClient init(String basePath, HoodieTableType tableType, String bootstrapBasePath, boolean bootstrapIndexEnable) throws IOException {
+ public static HoodieTableMetaClient init(String basePath, HoodieTableType tableType, String bootstrapBasePath, boolean bootstrapIndexEnable, String keyGenerator) throws IOException {
Properties props = new Properties();
props.setProperty(HoodieTableConfig.BOOTSTRAP_BASE_PATH.key(), bootstrapBasePath);
props.put(HoodieTableConfig.BOOTSTRAP_INDEX_ENABLE.key(), bootstrapIndexEnable);
+ if (keyGenerator != null) {
+ props.put("hoodie.datasource.write.keygenerator.class", keyGenerator);
+ props.put("hoodie.datasource.write.partitionpath.field", "datestr");
+ }
Review Comment:
I see. Thanks for the explanation.
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestBootstrapRead.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.functional;
+
+import org.apache.hudi.DataSourceReadOptions;
+import org.apache.hudi.DataSourceWriteOptions;
+import org.apache.hudi.client.bootstrap.selector.BootstrapRegexModeSelector;
+import org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector;
+import org.apache.hudi.client.bootstrap.selector.MetadataOnlyBootstrapModeSelector;
+import org.apache.hudi.common.testutils.HoodieTestDataGenerator;
+import org.apache.hudi.config.HoodieBootstrapConfig;
+import org.apache.hudi.config.HoodieCompactionConfig;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.keygen.ComplexKeyGenerator;
+import org.apache.hudi.keygen.NonpartitionedKeyGenerator;
+import org.apache.hudi.keygen.SimpleKeyGenerator;
+import org.apache.hudi.testutils.HoodieSparkClientTestBase;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.apache.spark.sql.SaveMode;
+import org.apache.spark.sql.functions;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static org.apache.hudi.common.testutils.RawTripTestPayload.recordToString;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Tests different layouts for bootstrap base path
+ */
+@Tag("functional")
+public class TestBootstrapRead extends HoodieSparkClientTestBase {
+
+ @TempDir
+ public java.nio.file.Path tmpFolder;
+ protected String bootstrapBasePath = null;
+ protected String bootstrapTargetPath = null;
+ protected String hudiBasePath = null;
+
+ protected static int nInserts = 100;
+ protected static int nUpdates = 20;
+ protected static String[] dashPartitionPaths = {"2016-03-15", "2015-03-16", "2015-03-17"};
+ protected static String[] slashPartitionPaths = {"2016/03/15", "2015/03/16", "2015/03/17"};
+ protected String bootstrapType;
+ protected Boolean dashPartitions;
+ protected String tableType;
+ protected Integer nPartitions;
+
+ protected String[] partitionCols;
+ protected static String[] dropColumns = {"_hoodie_commit_time", "_hoodie_commit_seqno", "_hoodie_record_key", "_hoodie_file_name", "city_to_state"};
+
+ @BeforeEach
+ public void setUp() throws Exception {
+ bootstrapBasePath = tmpFolder.toAbsolutePath() + "/bootstrapBasePath";
+ hudiBasePath = tmpFolder.toAbsolutePath() + "/hudiBasePath";
+ bootstrapTargetPath = tmpFolder.toAbsolutePath() + "/bootstrapTargetPath";
+ initSparkContexts();
+ }
+
+ @AfterEach
+ public void tearDown() throws IOException {
+ cleanupSparkContexts();
+ cleanupClients();
+ cleanupTestDataGenerator();
+ }
+
+ private static Stream<Arguments> testArgs() {
+ Stream.Builder<Arguments> b = Stream.builder();
+ //TODO: add dash partitions false with [HUDI-4944]
Review Comment:
Ok let's tackle that issue separately but I don't see more details in the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1542415477
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d1b90a6e91fbfe1229556377831d0c52d9c7613 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956)
* 66dd335158e2800359039917a9c1690450fa9c27 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1544499232
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 840cfab05cacd7a4862b2ad9a1983af2953819a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016)
* 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1548126045
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb7e42e3225e97863cb3e96953784385e19ec638 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040)
* cc420c8790dc736da431714aa78750be1c502a9d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #8666:
URL: https://github.com/apache/hudi/pull/8666#discussion_r1194337010
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestBootstrapRead.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.functional;
+
+import org.apache.hudi.DataSourceReadOptions;
+import org.apache.hudi.DataSourceWriteOptions;
+import org.apache.hudi.client.bootstrap.selector.BootstrapRegexModeSelector;
+import org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector;
+import org.apache.hudi.client.bootstrap.selector.MetadataOnlyBootstrapModeSelector;
+import org.apache.hudi.common.testutils.HoodieTestDataGenerator;
+import org.apache.hudi.config.HoodieBootstrapConfig;
+import org.apache.hudi.config.HoodieCompactionConfig;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.keygen.ComplexKeyGenerator;
+import org.apache.hudi.keygen.NonpartitionedKeyGenerator;
+import org.apache.hudi.keygen.SimpleKeyGenerator;
+import org.apache.hudi.testutils.HoodieSparkClientTestBase;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.apache.spark.sql.SaveMode;
+import org.apache.spark.sql.functions;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static org.apache.hudi.common.testutils.RawTripTestPayload.recordToString;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Tests different layouts for bootstrap base path
+ */
+@Tag("functional")
+public class TestBootstrapRead extends HoodieSparkClientTestBase {
+
+ @TempDir
+ public java.nio.file.Path tmpFolder;
+ protected String bootstrapBasePath = null;
+ protected String bootstrapTargetPath = null;
+ protected String hudiBasePath = null;
+
+ protected static int nInserts = 100;
+ protected static int nUpdates = 20;
+ protected static String[] dashPartitionPaths = {"2016-03-15", "2015-03-16", "2015-03-17"};
+ protected static String[] slashPartitionPaths = {"2016/03/15", "2015/03/16", "2015/03/17"};
+ protected String bootstrapType;
+ protected Boolean dashPartitions;
+ protected String tableType;
+ protected Integer nPartitions;
+
+ protected String[] partitionCols;
+ protected static String[] dropColumns = {"_hoodie_commit_time", "_hoodie_commit_seqno", "_hoodie_record_key", "_hoodie_file_name", "city_to_state"};
+
+ @BeforeEach
+ public void setUp() throws Exception {
+ bootstrapBasePath = tmpFolder.toAbsolutePath() + "/bootstrapBasePath";
+ hudiBasePath = tmpFolder.toAbsolutePath() + "/hudiBasePath";
+ bootstrapTargetPath = tmpFolder.toAbsolutePath() + "/bootstrapTargetPath";
+ initSparkContexts();
+ }
+
+ @AfterEach
+ public void tearDown() throws IOException {
+ cleanupSparkContexts();
+ cleanupClients();
+ cleanupTestDataGenerator();
+ }
+
+ private static Stream<Arguments> testArgs() {
+ Stream.Builder<Arguments> b = Stream.builder();
+ //TODO: add dash partitions false with [HUDI-4944]
Review Comment:
1. fixed
2. I added a 5 second delay between bootstrapping and the update and was still getting failures at the same rate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1548929549
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
}, {
"hash" : "cc420c8790dc736da431714aa78750be1c502a9d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17087",
"triggerID" : "cc420c8790dc736da431714aa78750be1c502a9d",
"triggerType" : "PUSH"
}, {
"hash" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17092",
"triggerID" : "62deeb26c94c9a073330488e1c7ea774dd5c7147",
"triggerType" : "PUSH"
}, {
"hash" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099",
"triggerID" : "c20d2780e542b92fbf95a2a3fb2ab94512943780",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c20d2780e542b92fbf95a2a3fb2ab94512943780 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17099)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1545860703
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
}, {
"hash" : "66dd335158e2800359039917a9c1690450fa9c27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16998",
"triggerID" : "66dd335158e2800359039917a9c1690450fa9c27",
"triggerType" : "PUSH"
}, {
"hash" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17015",
"triggerID" : "bfa1909d85b174cb466d205ee4ec2af09be1850d",
"triggerType" : "PUSH"
}, {
"hash" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17016",
"triggerID" : "840cfab05cacd7a4862b2ad9a1983af2953819a1",
"triggerType" : "PUSH"
}, {
"hash" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024",
"triggerID" : "1b2f28447ac507b35f82a0534ebd958a8fd8980d",
"triggerType" : "PUSH"
}, {
"hash" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040",
"triggerID" : "bb7e42e3225e97863cb3e96953784385e19ec638",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17024)
* bb7e42e3225e97863cb3e96953784385e19ec638 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8666:
URL: https://github.com/apache/hudi/pull/8666#issuecomment-1539369207
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16955",
"triggerID" : "7510bceed906d98b31bf44b4cd78a24a364c1288",
"triggerType" : "PUSH"
}, {
"hash" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956",
"triggerID" : "5d1b90a6e91fbfe1229556377831d0c52d9c7613",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5d1b90a6e91fbfe1229556377831d0c52d9c7613 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16956)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org