You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org> on 2026/08/21 18:30:26 UTC
[PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
SEPURI-SAI-KRISHNA opened a new pull request, #19710:
URL: https://github.com/apache/hudi/pull/19710
### Describe the issue this Pull Request addresses
Closes #19668.
`HoodieTableConfigUtils#inferPartitionValueExtractorClass` picks
`SlashEncodedDayPartitionValueExtractor` for a table written with
`hoodie.datasource.write.slash.separated.date.partitioning=true`. That extractor requires the
partition path to be exactly three segments and throws otherwise:
```java
String[] splits = partitionPath.split("/");
if (splits.length != 3) {
throw new IllegalArgumentException("Partition path " + partitionPath + " is not in the form yyyy/mm/dd ");
}
```
A null or empty partition value is written to the single-segment `__HIVE_DEFAULT_PARTITION__`
directory, not to a `yyyy/MM/dd` one, so hive sync of a slash table that has ever seen a null
partition value fails with:
```
java.lang.IllegalArgumentException: Partition path __HIVE_DEFAULT_PARTITION__ is not in the form yyyy/mm/dd
```
A null partition value is ordinary — it is what every write path produces for a null or empty
date column, and #19648 adds coverage for exactly that directory.
### Summary and Changelog
Both slash-encoded extractors now recognise the default-partition directory and pass it through
instead of trying to parse it as a date. `__HIVE_DEFAULT_PARTITION__` is Hive's own representation
of a null partition value, so handing it back unchanged is what lets Hive map the partition to
NULL.
* `SlashEncodedDayPartitionValueExtractor#extractPartitionValuesInPath` returns
`Collections.singletonList(DEFAULT_PARTITION_PATH)` for the default-partition directory, before
the three-segment check.
* `SlashEncodedHourPartitionValueExtractor` gets the same treatment. It is not auto-inferred, but
it carries the identical defect against its own four-segment check, is user-configurable, and is
equally exposed to a null partition value — fixing one and leaving the other seemed worse than
fixing both.
* The check tolerates a Hive-style prefix (`datestr=__HIVE_DEFAULT_PARTITION__`), matching the way
the existing parsing already strips a `key=` prefix from each segment.
Tests in `TestPartitionValueExtractor`: the default-partition directory, bare and Hive-style
prefixed, for both extractors. `testDayPartition` is new — the day extractor had no test at all
despite being the auto-inferred one — and also pins the ordinary `yyyy/MM/dd` and
`datestr=yyyy/MM/dd` cases plus the existing rejection of a malformed path.
Both new cases fail on the unfixed code with the `IllegalArgumentException` quoted above.
### Impact
Hive sync of a slash-partitioned table containing a null partition value succeeds instead of
failing outright. No change for any other partition path: the new branch matches only the
default-partition marker, and everything else takes the pre-existing parsing.
The deprecated `default` marker is deliberately not handled here; the current write paths produce
`__HIVE_DEFAULT_PARTITION__`, and mapping legacy directories would be a separate behavior decision.
### Risk Level
low
One early-return in each of two extractors, matching a single constant, with the pre-existing
parsing untouched. Verified with `TestPartitionValueExtractor` (4 tests, all passing) and by
re-running the new cases against the unfixed code to confirm they fail there.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #19710:
URL: https://github.com/apache/hudi/pull/19710
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5380757277
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
}, {
"hash" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16483",
"triggerID" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8330fcc8530cf92c6215fb7383a3415e3e1e56a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16483)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5373933534
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b92920ad874ac7a19f55ba4463289120094db7b3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org>.
SEPURI-SAI-KRISHNA commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3835986292
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
}
+
+ private static boolean isDefaultPartition(String partitionPath) {
+ // a Hive-style directory carries the column name, e.g. "datestr=__HIVE_DEFAULT_PARTITION__"
+ int separator = partitionPath.indexOf('=');
+ String value = separator < 0 ? partitionPath : partitionPath.substring(separator + 1);
Review Comment:
Applied — both markers are now matched.
Two things made it worth doing rather than scoping out. `PartitionPathParser#parseValue` and `Type.java:103` already accept either marker on the read side, so matching only the
current one here was the odd one out. And while `FourToFiveUpgradeHandler` hard-fails the upgrade when a `default` partition exists, that check is skippable via
`hoodie.skip.default.partition.validation`, so a legacy table can still reach these extractors.
Folded into the shared helper from your other comment: `PartitionPathEncodeUtils#isDefaultPartitionValue` handles both. A legacy `default` directory still maps to
`__HIVE_DEFAULT_PARTITION__` as the returned value, since that is what Hive records for a null partition value — the directory name and the recorded value are independent.
`TestPartitionValueExtractor` covers bare `default` and `col=default` for both extractors; both new cases fail without the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org>.
SEPURI-SAI-KRISHNA commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3835988750
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
}
+
+ private static boolean isDefaultPartition(String partitionPath) {
Review Comment:
Applied — extracted to `PartitionPathEncodeUtils#isDefaultPartitionValue(String)` and used by both extractors, so the duplicated private copies are gone. It also handles the deprecated `default` marker per your other comment, so both concerns are fixed together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5378740625
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b92920ad874ac7a19f55ba4463289120094db7b3 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470)
* 505e6b3cb6d1113c2db078e09505f6adb9cf6e39 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5380334579
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
}, {
"hash" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 505e6b3cb6d1113c2db078e09505f6adb9cf6e39 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480)
* 8330fcc8530cf92c6215fb7383a3415e3e1e56a9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3846192354
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -52,6 +55,13 @@ private DateTimeFormatter getDtfOut() {
@Override
public List<String> extractPartitionValuesInPath(String partitionPath) {
+ // NOTE: A null or empty partition value lands in the default-partition directory, which is a
Review Comment:
🤖 nit: this NOTE block restates what's already covered by `isDefaultPartitionValue`'s Javadoc, and the same text is copy-pasted nearly verbatim into the hour extractor. Could you trim both to a short line like `// default-partition directory — pass straight through` and let the utility's Javadoc be the authoritative explanation?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5373997012
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b92920ad874ac7a19f55ba4463289120094db7b3 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org>.
SEPURI-SAI-KRISHNA commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3835990554
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
Review Comment:
Applied, extracted to `PartitionPathEncodeUtils#isDefaultPartitionValue(String)`, used by both extractors. (Same as the earlier thread on this file, which raised the identical point.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3835445189
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
Review Comment:
🤖 nit: `isDefaultPartition` is identical in both extractor classes — could you move it to a shared base class or a small static helper in e.g. `PartitionPathEncodeUtils` so the two implementations stay in sync automatically?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5373913281
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19710?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 75.58%. Comparing base ([`b6c4677`](https://app.codecov.io/gh/apache/hudi/commit/b6c4677206b4ba241c4b9b85110da60fcec7e1f3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b92920a`](https://app.codecov.io/gh/apache/hudi/commit/b92920ad874ac7a19f55ba4463289120094db7b3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 18 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19710 +/- ##
============================================
- Coverage 75.91% 75.58% -0.34%
- Complexity 32313 32816 +503
============================================
Files 2524 2527 +3
Lines 139550 142462 +2912
Branches 16793 17277 +484
============================================
+ Hits 105943 107675 +1732
- Misses 26095 27132 +1037
- Partials 7512 7655 +143
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `82.54% <ø> (-0.56%)` | :arrow_down: |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.40% <ø> (-1.07%)` | :arrow_down: |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.74% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.59% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.99% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.26% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.00% <ø> (-1.03%)` | :arrow_down: |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.58% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.34% <ø> (-0.61%)` | :arrow_down: |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.06% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19710/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19710/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19710/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.89% <ø> (+0.06%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19710?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...i/hive/SlashEncodedDayPartitionValueExtractor.java](https://app.codecov.io/gh/apache/hudi/pull/19710?src=pr&el=tree&filepath=hudi-sync%2Fhudi-hive-sync%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fhive%2FSlashEncodedDayPartitionValueExtractor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2xhc2hFbmNvZGVkRGF5UGFydGl0aW9uVmFsdWVFeHRyYWN0b3IuamF2YQ==) | `64.28% <ø> (ø)` | |
| [.../hive/SlashEncodedHourPartitionValueExtractor.java](https://app.codecov.io/gh/apache/hudi/pull/19710?src=pr&el=tree&filepath=hudi-sync%2Fhudi-hive-sync%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fhive%2FSlashEncodedHourPartitionValueExtractor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2xhc2hFbmNvZGVkSG91clBhcnRpdGlvblZhbHVlRXh0cmFjdG9yLmphdmE=) | `66.66% <ø> (ø)` | |
... and [37 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/19710/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19710:
URL: https://github.com/apache/hudi/pull/19710#discussion_r3832732842
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
}
+
+ private static boolean isDefaultPartition(String partitionPath) {
+ // a Hive-style directory carries the column name, e.g. "datestr=__HIVE_DEFAULT_PARTITION__"
+ int separator = partitionPath.indexOf('=');
+ String value = separator < 0 ? partitionPath : partitionPath.substring(separator + 1);
Review Comment:
🤖 This matches on `DEFAULT_PARTITION_PATH` (`__HIVE_DEFAULT_PARTITION__`), but `PartitionPathEncodeUtils` also defines `DEPRECATED_DEFAULT_PARTITION_PATH` (`"default"`) for older tables. A slash table written before the marker changed would land nulls in a single-segment `default` dir and still hit the 3-segment check. Is that legacy case out of scope here, or worth matching too?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/SlashEncodedDayPartitionValueExtractor.java:
##########
@@ -65,4 +73,11 @@ public List<String> extractPartitionValuesInPath(String partitionPath) {
return Collections.singletonList(dateTime.format(getDtfOut()));
}
+
+ private static boolean isDefaultPartition(String partitionPath) {
Review Comment:
🤖 nit: `isDefaultPartition` is identical in both `SlashEncodedDayPartitionValueExtractor` and `SlashEncodedHourPartitionValueExtractor` — could it live in a shared base class or a small `PartitionValueExtractorUtils` helper so the two implementations stay in sync?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5374638682
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b92920ad874ac7a19f55ba4463289120094db7b3 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5379496623
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 505e6b3cb6d1113c2db078e09505f6adb9cf6e39 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5380357314
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
}, {
"hash" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16483",
"triggerID" : "8330fcc8530cf92c6215fb7383a3415e3e1e56a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 505e6b3cb6d1113c2db078e09505f6adb9cf6e39 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480)
* 8330fcc8530cf92c6215fb7383a3415e3e1e56a9 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16483)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): pass the default partition through the slash-encoded value extractors [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19710:
URL: https://github.com/apache/hudi/pull/19710#issuecomment-5378813050
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470",
"triggerID" : "b92920ad874ac7a19f55ba4463289120094db7b3",
"triggerType" : "PUSH"
}, {
"hash" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480",
"triggerID" : "505e6b3cb6d1113c2db078e09505f6adb9cf6e39",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b92920ad874ac7a19f55ba4463289120094db7b3 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16470)
* 505e6b3cb6d1113c2db078e09505f6adb9cf6e39 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16480)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org