You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/07/27 07:52:42 UTC
[GitHub] [hudi] rmahindra123 opened a new pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
rmahindra123 opened a new pull request #3353:
URL: https://github.com/apache/hudi/pull/3353
## What is the purpose of the pull request
This is a PR for the changes that were done in #2475, with following fixes:
1. The actual logic is fixed to ensure it works
2. Code re-structure / cleanup and decouple the logic to detect full table reads from the current logic
3. Added new tests, and ensure tests actually verify the outputs
Original PR description: To read the hudi table, you need to specify the path, but the path is not only the tablePath corresponding to the table, but needs to be determined by the partition directory structure. Different keyGenerators correspond to different partition directory structures. The first-level partition directory uses path=.../table/*/*, the secondary partition directory path=.../table/*/*/*，so it is troublesome to let the user specify the data path, the user only needs to specify the tablePath: .../table
At the same time, after reading the hudi table by configuring path=.../table, it is more convenient to use sparksql to query the hudi table. You only need to add tab properties to the hive table metadata: spark.sql.sources.provider= hudi, you can automatically convert the hive table to the hudi table.
## Brief change log
Added logic in `createRelation()` method in `DefaultSource` to detect when a user only specifies the full table path and not a blob when reading the entire table. If detected correctly, the path is rewritten automatically as a blob that will ensure the rest of the logic works as before.
## Verify this pull request
Added unit and functional tests to cover all cases: no partition, single partition, date partition, and custom partition/ key generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
pengzhiwei2018 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-891489626
> please feel free give it a shot as well
> @vinothchandar It works for CoW, the final test in `TestMORDataSource.java` fails, so I have special cased it for CoW for now.
> @pengzhiwei2018 . I am planning on taking this over. If you have cycles, please feel free give it a shot as well.
Please feel free for this. I am currently busy with the sql integration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
pengzhiwei2018 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888779737
> That's pretty much the functionality, supporting reads of table without having to pass globs ("basePath/_/_/*") You can see the tests.
Yes, I see. But the PR #2651 has already supported reading table without pass globs. is there any difference?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rmahindra123 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
rmahindra123 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887901821
@vinothchandar It works for CoW, the final test in TestMor fails, so I have special cased it for CoW for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-891336366
@pengzhiwei2018 . I am planning on taking this over. If you have cycles, please feel free give it a shot as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
pengzhiwei2018 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888776637
Hi @rmahindra123 , the PR https://github.com/apache/hudi/pull/2651 has already support query by the table path. So I am not very understand what is the PR do, can you explain it for me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
pengzhiwei2018 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888883773
> > > That's pretty much the functionality, supporting reads of table without having to pass globs ("basePath/_/_/*") You can see the tests.
> >
> >
> > Yes, I see. But the PR #2651 has already supported reading table without pass globs. is there any difference?
>
> @pengzhiwei2018 Its similar functionality but there were a few bugs and the tests were not actually testing the functionality. It required some refactoring to work for different cases, so I put it in a new PR.
Thanks for your instructions @rmahindra123 ， can you implement this based on the `HoodieFileIndex`? By a simple review, I found some duplicate file list operation with the HoodieFileIndex.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888755403
@pengzhiwei2018 are you able to take one pass at this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fa243144df9e2bb93c02798e1e576dc7af31819a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rmahindra123 edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
rmahindra123 edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887901821
@vinothchandar It works for CoW, the final test in `TestMORDataSource.java` fails, so I have special cased it for CoW for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-961588028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189)
* fa243144df9e2bb93c02798e1e576dc7af31819a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fa243144df9e2bb93c02798e1e576dc7af31819a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192)
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rmahindra123 commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
rmahindra123 commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888805871
> > That's pretty much the functionality, supporting reads of table without having to pass globs ("basePath/_/_/*") You can see the tests.
>
> Yes, I see. But the PR #2651 has already supported reading table without pass globs. is there any difference?
@pengzhiwei2018 Its similar functionality but there were a few bugs and the tests were not actually testing the functionality. It required some refactoring to work for different cases, so I put it in a new PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fa243144df9e2bb93c02798e1e576dc7af31819a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192)
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189)
* fa243144df9e2bb93c02798e1e576dc7af31819a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-961588028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
}, {
"hash" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1192",
"triggerID" : "fa243144df9e2bb93c02798e1e576dc7af31819a",
"triggerType" : "PUSH"
}, {
"hash" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203",
"triggerID" : "540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 540a91e02b5d39bfe2c6c737fa8eefcfa5bf6b7d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-888777601
That's pretty much the functionality, supporting reads of table without having to pass globs ("basePath/*/*/*") You can see the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189",
"triggerID" : "cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfaf6084521f4f93d6680c1f9b0a3bc246b9fc99 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1189)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] rmahindra123 closed pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
rmahindra123 closed pull request #3353:
URL: https://github.com/apache/hudi/pull/3353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #3353: [HUDI-1527] Automatically infer the full partition path when user only specifies the table path
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3353:
URL: https://github.com/apache/hudi/pull/3353#issuecomment-887296516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org