You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "boundarymate (via GitHub)" <gi...@apache.org> on 2023/03/29 08:19:19 UTC
[GitHub] [hudi] boundarymate opened a new issue, #8314: [SUPPORT] Why not save log files with timestamps less than savepoint time?
boundarymate opened a new issue, #8314:
URL: https://github.com/apache/hudi/issues/8314
# Background
When I made a savepoint for my hudi table, I found only basefiles with instant time less than the savepoint's instant time would be saved, but I think it is necessary to save the logfiles with instant time less than the savepoint's timestamp and greater than the basefile's timestamp.
Because the combination of the basefile and the logfiles is a full snapshot at the savepoint’s instant time.
# Code
SavepointActionExecutor#execute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan closed issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "xushiyan (via GitHub)" <gi...@apache.org>.
xushiyan closed issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
URL: https://github.com/apache/hudi/issues/8314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ad1happy2go commented on issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #8314:
URL: https://github.com/apache/hudi/issues/8314#issuecomment-1569762957
Closing out this issue since the JIRA is closed now. https://issues.apache.org/jira/browse/HUDI-6007
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boundarymate commented on issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "boundarymate (via GitHub)" <gi...@apache.org>.
boundarymate commented on issue #8314:
URL: https://github.com/apache/hudi/issues/8314#issuecomment-1488150363
Hudi version : 0.13.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boundarymate commented on issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "boundarymate (via GitHub)" <gi...@apache.org>.
boundarymate commented on issue #8314:
URL: https://github.com/apache/hudi/issues/8314#issuecomment-1491216309
> Yes, it is a bug, especially when using the MOR table with Flink, hudi savepoint is actually invalid. I created a [JIRA](https://issues.apache.org/jira/browse/HUDI-6007) and will try to fix it
Thank you for your comment, but I think savepoint is not completely invalid when using the MOR table with Flink, since mor table also have basefiles when finish compaction.No saving the logfiles just could potentially lead to a consistency issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on issue #8314:
URL: https://github.com/apache/hudi/issues/8314#issuecomment-1490554416
Yes, it is definitely a bug, I created a [JIRA](https://issues.apache.org/jira/browse/HUDI-6007) and will try to fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on issue #8314: [SUPPORT] Why not save log files with timestamp less than savepoint's instant time?
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on issue #8314:
URL: https://github.com/apache/hudi/issues/8314#issuecomment-1493821055
@boundarymate A problem was encountered during the implementation. Since the timestamp of the log file is consistent with the basefile, we cannot filter the log file by timestamp
```shell
-rw-r--r-- 1 zxy staff 20K 4 3 15:16 .638da311-b847-4412-8e90-938c4c575581-0_001.log.1_1-53-76
-rw-r--r-- 1 zxy staff 438K 4 3 15:15 638da311-b847-4412-8e90-938c4c575581-0_1-16-28_001.parquet
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org