You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/10/18 08:48:01 UTC
[GitHub] [hudi] Carl-Zhou-CN opened a new pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Carl-Zhou-CN opened a new pull request #3817:
URL: https://github.com/apache/hudi/pull/3817
#3759 *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-947277899
@Carl-Zhou-CN : whats your apache id? I will assign the ticket to you. https://issues.apache.org/jira/browse/HUDI-2582
and thanks for the contribution. looks like a valid bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
novakov-alexey edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-951023252
I usually use git commit cherry-pick if I need to create new branch with particular commits from another branch. You can try it as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-956855393
@hudi-bot azure run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r734240126
##########
File path: hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestHoodieDatasetBulkInsertHelper.java
##########
@@ -84,14 +84,35 @@ public TestHoodieDatasetBulkInsertHelper() throws IOException {
}
private void init() throws IOException {
- schemaStr = FileIOUtils.readAsUTFString(getClass().getResourceAsStream("/exampleSchema.txt"));
- schema = DataSourceTestUtils.getStructTypeExampleSchema();
+ String schemaPath = "/exampleSchema.txt";
+ schemaStr = FileIOUtils.readAsUTFString(getClass().getResourceAsStream(schemaPath));
+ Schema schema = DataSourceTestUtils.getStructTypeExampleSchema(schemaPath);
structType = AvroConversionUtils.convertAvroSchemaToStructType(schema);
}
@Test
- public void testBulkInsertHelper() {
- HoodieWriteConfig config = getConfigBuilder(schemaStr).withProps(getPropsAllSet()).combineInput(false, false).build();
+ public void testBulkInsertHelperConcurrently() throws IOException {
+ String schema2Path = "/exampleSchema2.txt";
+ String schemaStr2 = FileIOUtils.readAsUTFString(getClass().getResourceAsStream(schema2Path));
+ StructType structType2 = AvroConversionUtils
+ .convertAvroSchemaToStructType(DataSourceTestUtils.getStructTypeExampleSchema(schema2Path));
+
+ IntStream.range(0, 10).parallel().forEach(i -> {
+ if (i % 2 == 0) {
+ testBulkInsertHelperFor(schemaStr, structType, "_row_key");
+ } else {
+ testBulkInsertHelperFor(schemaStr2, structType2, "_row_key2");
Review comment:
rather than introducing a new schema, can we re-use the same schema, but change key gen properties? may be choose "ts" as record key.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997",
"triggerID" : "955854010",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945602232
@novakov-alexey I originally wanted to quote your test case, but without your consent, I didn’t submit it in my code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2ae44db1ac12f0f966ab97d03a0547baafcf9dda Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701)
* ae74cf9f9e05aad6146b2b06da972fe476cb7994 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 07c5bf67d9d92cb65252e6b024b1dd09f4ce8289 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931)
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-951012177
not sure if there is any easy out. I see the diff as such is small. So, if its not a hassle, may be create a fresh branch from master and manually copy paste the changes from this patch. And push to this this branch in your remote origin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-950975892
Sorry, this is because the base branch I submitted is not the latest branch of my fork. I didn't use it to pull before submitting to git. Do you have any good suggestions for me to repair this piece of history?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3817:
URL: https://github.com/apache/hudi/pull/3817
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 07c5bf67d9d92cb65252e6b024b1dd09f4ce8289 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931)
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789)
* c005b08e4be6127f18364cb75d7f7f23d4e98ec9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c005b08e4be6127f18364cb75d7f7f23d4e98ec9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945583583
@Carl-Zhou-CN thanks for quick response. PR looks promising. Did you try to run that test I added to call HoodieDatasetBulkInsertHelper concurrently? I would also include it to this PR or at least some ideas from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r732380941
##########
File path: hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java
##########
@@ -80,17 +81,18 @@
String keyGeneratorClass = properties.getString(DataSourceWriteOptions.KEYGENERATOR_CLASS_NAME().key());
BuiltinKeyGenerator keyGenerator = (BuiltinKeyGenerator) ReflectionUtils.loadClass(keyGeneratorClass, properties);
StructType structTypeForUDF = rows.schema();
-
- sqlContext.udf().register(RECORD_KEY_UDF_FN, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
- sqlContext.udf().register(PARTITION_PATH_UDF_FN, (UDF1<Row, String>) keyGenerator::getPartitionPath, DataTypes.StringType);
+ String recordKeyUdfFnCurrent = RECORD_KEY_UDF_FN + UUID.randomUUID().toString();
+ String partitionPathUdfFnCurrent = PARTITION_PATH_UDF_FN + UUID.randomUUID().toString();
+ sqlContext.udf().register(recordKeyUdfFnCurrent, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
Review comment:
let me think about the actual fix. should we suffix table name or base path may be instead of random Id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on a change in pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r733490866
##########
File path: hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestHoodieDatasetBulkInsertHelper.java
##########
@@ -84,14 +84,35 @@ public TestHoodieDatasetBulkInsertHelper() throws IOException {
}
private void init() throws IOException {
- schemaStr = FileIOUtils.readAsUTFString(getClass().getResourceAsStream("/exampleSchema.txt"));
- schema = DataSourceTestUtils.getStructTypeExampleSchema();
+ String schemaPath = "/exampleSchema.txt";
+ schemaStr = FileIOUtils.readAsUTFString(getClass().getResourceAsStream(schemaPath));
+ Schema schema = DataSourceTestUtils.getStructTypeExampleSchema(schemaPath);
structType = AvroConversionUtils.convertAvroSchemaToStructType(schema);
}
@Test
- public void testBulkInsertHelper() {
- HoodieWriteConfig config = getConfigBuilder(schemaStr).withProps(getPropsAllSet()).combineInput(false, false).build();
+ public void testBulkInsertHelperConcurrently() throws IOException {
+ String schema2Path = "/exampleSchema2.txt";
+ String schemaStr2 = FileIOUtils.readAsUTFString(getClass().getResourceAsStream(schema2Path));
+ StructType structType2 = AvroConversionUtils
+ .convertAvroSchemaToStructType(DataSourceTestUtils.getStructTypeExampleSchema(schema2Path));
+
+ IntStream.range(0, 10).parallel().forEach(i -> {
Review comment:
feel free to think how to improve this concurrency test. On my machine it takes 15 seconds, so the Hudi build with tests will become probably for 15 seconds slower too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-951023252
I usually use git commit cherry-pick if I need to create new branch with particular commits from another branch. You can try is as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-957063012
@hudi-bot azure run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-948309212
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-948389410
@nsivabalan There was an error during the build. I have tried to rebuild, but it still hasn’t been resolved. Please help me to see it. The error seems to have nothing to do with the modification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae74cf9f9e05aad6146b2b06da972fe476cb7994 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-953632802
@nsivabalan I seem to succeed, is it the result you want?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae74cf9f9e05aad6146b2b06da972fe476cb7994 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759)
* 6f0322637300e0df2e8b216e3d63c4dddffeb041 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945590052
@novakov-alexey Use your test case
> @Carl-Zhou-CN感谢您的快速回复。公关看起来很有希望。您是否尝试运行我添加的用于同时调用 HoodieDatasetBulkInsertHelper 的测试？我也会把它包含在这个 PR 中，或者至少包含一些来自它的想法。
Yes, use your test case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945682795
@Carl-Zhou-CN feel free to take it to your PR. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-950961437
Looks like you have pulled in some extra commits too. Can you check whats happening and ensure we have commits only from this patch.
Patch as such looks good to me. Once commit history is fixed, we can merge it in.
<img width="1014" alt="Screen Shot 2021-10-25 at 7 01 13 AM" src="https://user-images.githubusercontent.com/513218/138710038-4fac286f-47a6-4241-a589-a11ff32ef811.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997",
"triggerID" : "955854010",
"triggerType" : "MANUAL"
}, {
"hash" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3039",
"triggerID" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997)
* c224ebc28a452bfd645d9844c28d36b26f28b885 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3039)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2ae44db1ac12f0f966ab97d03a0547baafcf9dda UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945601661
@Carl-Zhou-CN it seems fix is working. I have run my unit test several times in the master branch, it is green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c005b08e4be6127f18364cb75d7f7f23d4e98ec9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804)
* 07c5bf67d9d92cb65252e6b024b1dd09f4ce8289 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-956554404
thanks for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f0322637300e0df2e8b216e3d63c4dddffeb041 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763)
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-947298066
@nsivabalan Thank you for your recognition. I have assigned it to myself
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-951029767
Okay, thank you both for your suggestions, I will try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769)
* a11cd8809c5d83f7911c3074ab69b65fd6b0621b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c005b08e4be6127f18364cb75d7f7f23d4e98ec9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804)
* 07c5bf67d9d92cb65252e6b024b1dd09f4ce8289 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769)
* a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997",
"triggerID" : "955854010",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997",
"triggerID" : "955854010",
"triggerType" : "MANUAL"
}, {
"hash" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5da4b25ccfbb8994b9d3f4139ab6279c1f782093 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997)
* c224ebc28a452bfd645d9844c28d36b26f28b885 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-955854010
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2804",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
}, {
"hash" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2931",
"triggerID" : "07c5bf67d9d92cb65252e6b024b1dd09f4ce8289",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2932",
"triggerID" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"triggerType" : "PUSH"
}, {
"hash" : "5da4b25ccfbb8994b9d3f4139ab6279c1f782093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2997",
"triggerID" : "955854010",
"triggerType" : "MANUAL"
}, {
"hash" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3039",
"triggerID" : "c224ebc28a452bfd645d9844c28d36b26f28b885",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c224ebc28a452bfd645d9844c28d36b26f28b885 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3039)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on a change in pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r730741735
##########
File path: hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java
##########
@@ -80,17 +81,18 @@
String keyGeneratorClass = properties.getString(DataSourceWriteOptions.KEYGENERATOR_CLASS_NAME().key());
BuiltinKeyGenerator keyGenerator = (BuiltinKeyGenerator) ReflectionUtils.loadClass(keyGeneratorClass, properties);
StructType structTypeForUDF = rows.schema();
-
- sqlContext.udf().register(RECORD_KEY_UDF_FN, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
- sqlContext.udf().register(PARTITION_PATH_UDF_FN, (UDF1<Row, String>) keyGenerator::getPartitionPath, DataTypes.StringType);
+ String recordKeyUdfFnCurrent = RECORD_KEY_UDF_FN + UUID.randomUUID().toString();
+ String partitionPathUdfFnCurrent = PARTITION_PATH_UDF_FN + UUID.randomUUID().toString();
+ sqlContext.udf().register(recordKeyUdfFnCurrent, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
Review comment:
Does it mean that getRecordKey and getPartitionPath may be equal from different tables, so that adding random suffix to them help?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2ae44db1ac12f0f966ab97d03a0547baafcf9dda Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] novakov-alexey commented on a change in pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
novakov-alexey commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r730742484
##########
File path: hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java
##########
@@ -80,17 +81,18 @@
String keyGeneratorClass = properties.getString(DataSourceWriteOptions.KEYGENERATOR_CLASS_NAME().key());
BuiltinKeyGenerator keyGenerator = (BuiltinKeyGenerator) ReflectionUtils.loadClass(keyGeneratorClass, properties);
StructType structTypeForUDF = rows.schema();
-
- sqlContext.udf().register(RECORD_KEY_UDF_FN, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
- sqlContext.udf().register(PARTITION_PATH_UDF_FN, (UDF1<Row, String>) keyGenerator::getPartitionPath, DataTypes.StringType);
+ String recordKeyUdfFnCurrent = RECORD_KEY_UDF_FN + UUID.randomUUID().toString();
+ String partitionPathUdfFnCurrent = PARTITION_PATH_UDF_FN + UUID.randomUUID().toString();
+ sqlContext.udf().register(recordKeyUdfFnCurrent, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
Review comment:
I thought that `keyGenerator` is a unique object per table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Carl-Zhou-CN commented on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945548178
https://github.com/apache/hudi/issues/3759
Caused when resolving concurrency(HoodieKeyException: recordKey value: "null")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: fix: HoodieDatasetBulkInsertHelper concurrently rowkey not found
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2ae44db1ac12f0f966ab97d03a0547baafcf9dda Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae74cf9f9e05aad6146b2b06da972fe476cb7994 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759)
* 6f0322637300e0df2e8b216e3d63c4dddffeb041 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2ae44db1ac12f0f966ab97d03a0547baafcf9dda Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701)
* ae74cf9f9e05aad6146b2b06da972fe476cb7994 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#discussion_r734753400
##########
File path: hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/HoodieDatasetBulkInsertHelper.java
##########
@@ -79,18 +78,19 @@
properties.putAll(config.getProps());
String keyGeneratorClass = properties.getString(DataSourceWriteOptions.KEYGENERATOR_CLASS_NAME().key());
BuiltinKeyGenerator keyGenerator = (BuiltinKeyGenerator) ReflectionUtils.loadClass(keyGeneratorClass, properties);
- StructType structTypeForUDF = rows.schema();
-
- sqlContext.udf().register(RECORD_KEY_UDF_FN, (UDF1<Row, String>) keyGenerator::getRecordKey, DataTypes.StringType);
- sqlContext.udf().register(PARTITION_PATH_UDF_FN, (UDF1<Row, String>) keyGenerator::getPartitionPath, DataTypes.StringType);
+ String tableName = properties.getString(HoodieWriteConfig.TBL_NAME.key());
+ String recordKeyUdfFnCurrent = RECORD_KEY_UDF_FN + tableName;
Review comment:
we don't need to name the variable with "current" suffix. the value I understand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2769",
"triggerID" : "948309212",
"triggerType" : "MANUAL"
}, {
"hash" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789",
"triggerID" : "a11cd8809c5d83f7911c3074ab69b65fd6b0621b",
"triggerType" : "PUSH"
}, {
"hash" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c005b08e4be6127f18364cb75d7f7f23d4e98ec9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a11cd8809c5d83f7911c3074ab69b65fd6b0621b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2789)
* c005b08e4be6127f18364cb75d7f7f23d4e98ec9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f0322637300e0df2e8b216e3d63c4dddffeb041 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2701",
"triggerID" : "2ae44db1ac12f0f966ab97d03a0547baafcf9dda",
"triggerType" : "PUSH"
}, {
"hash" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2759",
"triggerID" : "ae74cf9f9e05aad6146b2b06da972fe476cb7994",
"triggerType" : "PUSH"
}, {
"hash" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763",
"triggerID" : "6f0322637300e0df2e8b216e3d63c4dddffeb041",
"triggerType" : "PUSH"
}, {
"hash" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767",
"triggerID" : "07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f0322637300e0df2e8b216e3d63c4dddffeb041 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2763)
* 07e911cc15ddfa2b811ee86a40f2cbc7c3d5c191 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2767)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-956554404
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-956554404
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3817: [HUDI-2582] Support concurrent key gen for different tables with row writer path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3817:
URL: https://github.com/apache/hudi/pull/3817#issuecomment-945545601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org