You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nicholas-fwang (via GitHub)" <gi...@apache.org> on 2023/03/24 11:01:55 UTC
[GitHub] [hudi] nicholas-fwang opened a new pull request, #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
nicholas-fwang opened a new pull request, #8284:
URL: https://github.com/apache/hudi/pull/8284
### Change Logs
Create instant time after setCommitTimezone if table exists, or after initTable if no table exists.
### Impact
When hoodie.table.timeline.timezone is UTC in not UTC LOCAL timezone, timeline action does not progress.
### Risk level (write none, low medium or high below)
none
### Documentation Update
The commit timezone is not updated when HoodieSparkSqlWriter write method.
For example, the LOCAL time zone is KST (UTC+9), and even if 'hoodie.table.timeline.timezone' is UTC, the first instance time is created as LOCAL (KST) and then initTable is called.
Then, the second instant time after initTable is created in UTC and waits because the first instant time is 9 hours ahead of the second KST.
And in other situations, a write method started when there is already an initialized table does not call setCommitTimezone.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1483741022
Yeah the build succeeded locally for me for Spark 3.3. I just pushed a commit after rebasing with master. Let's see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope merged PR #8284:
URL: https://github.com/apache/hudi/pull/8284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nicholas-fwang commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "nicholas-fwang (via GitHub)" <gi...@apache.org>.
nicholas-fwang commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1482900107
@codope thanks for review.
I'm trying to find a violation for failure of https://github.com/apache/hudi/actions/runs/4510471335/jobs/7943933966?pr=8284
but I couldn't find them in my build, just success. could you know what's checkstyle violation in this PR?
thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1492445298
also, lets ensure we follow this.
for a new table, we should honor what the user sets.
for subsequent commit, we should fetch from table config and honor the time zone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1482647182
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897",
"triggerID" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nicholas-fwang commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "nicholas-fwang (via GitHub)" <gi...@apache.org>.
nicholas-fwang commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1492548335
@yihua sure, I will add it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1482984410
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897",
"triggerID" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1483740825
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897",
"triggerID" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"triggerType" : "PUSH"
}, {
"hash" : "5b31410a0bb28ebb16d0af88f8c45662b6b8fd92",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5b31410a0bb28ebb16d0af88f8c45662b6b8fd92",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897)
* 5b31410a0bb28ebb16d0af88f8c45662b6b8fd92 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1483748587
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897",
"triggerID" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"triggerType" : "PUSH"
}, {
"hash" : "5b31410a0bb28ebb16d0af88f8c45662b6b8fd92",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15913",
"triggerID" : "5b31410a0bb28ebb16d0af88f8c45662b6b8fd92",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15897)
* 5b31410a0bb28ebb16d0af88f8c45662b6b8fd92 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15913)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1492420744
Hi @nicholas-fwang could you add a unit test for this? The unit test should fail without this PR and succeed after the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8284: [HUDI-5978] spark timeline timezone is not updated when hoodie.table.timeline.timezone is UTC
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8284:
URL: https://github.com/apache/hudi/pull/8284#issuecomment-1482638115
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b032ff4bd9e40fba4bf2bb318a1acaa3f7d0d87 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org