You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2024/12/13 02:06:52 UTC
[PR] [WIP][core] Store `cardinality` `first` and `last` in the deletion vector meta [paimon]
Zouxxyy opened a new pull request, #4699:
URL: https://github.com/apache/paimon/pull/4699
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
Store `cardinality` `first` and `last` in the deletion vector meta
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Store `cardinality` in the deletion vector meta [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #4699:
URL: https://github.com/apache/paimon/pull/4699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Store `cardinality` `first` and `last` in the deletion vector meta [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #4699:
URL: https://github.com/apache/paimon/pull/4699#issuecomment-2540611270
Can you explain the usage for them separately?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Store `cardinality` in the deletion vector meta [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #4699:
URL: https://github.com/apache/paimon/pull/4699#discussion_r1883714404
##########
paimon-core/src/main/java/org/apache/paimon/table/sink/CommitMessageSerializer.java:
##########
@@ -107,48 +111,48 @@ public List<CommitMessage> deserializeList(int version, DataInputView view) thro
}
private CommitMessage deserialize(int version, DataInputView view) throws IOException {
- if (version >= 3) {
- IOExceptionSupplier<List<DataFileMeta>> fileDeserializer =
- () -> dataFileSerializer.deserializeList(view);
- if (version == 3) {
- DataFileMeta09Serializer serializer = new DataFileMeta09Serializer();
- fileDeserializer = () -> serializer.deserializeList(view);
- }
- return new CommitMessageImpl(
- deserializeBinaryRow(view),
- view.readInt(),
- new DataIncrement(
- fileDeserializer.get(), fileDeserializer.get(), fileDeserializer.get()),
- new CompactIncrement(
- fileDeserializer.get(), fileDeserializer.get(), fileDeserializer.get()),
- new IndexIncrement(
- indexEntrySerializer.deserializeList(view),
- indexEntrySerializer.deserializeList(view)));
- } else {
- return deserialize08(version, view);
- }
- }
-
- private CommitMessage deserialize08(int version, DataInputView view) throws IOException {
- if (dataFile08Serializer == null) {
- dataFile08Serializer = new DataFileMeta08Serializer();
- }
+ IOExceptionSupplier<List<DataFileMeta>> fileDeserializer = fileDeserializer(version, view);
+ IOExceptionSupplier<List<IndexFileMeta>> indexEntryDeserializer =
+ indexEntryDeserializer(version, view);
return new CommitMessageImpl(
deserializeBinaryRow(view),
view.readInt(),
new DataIncrement(
- dataFile08Serializer.deserializeList(view),
- dataFile08Serializer.deserializeList(view),
- dataFile08Serializer.deserializeList(view)),
+ fileDeserializer.get(), fileDeserializer.get(), fileDeserializer.get()),
new CompactIncrement(
- dataFile08Serializer.deserializeList(view),
- dataFile08Serializer.deserializeList(view),
- dataFile08Serializer.deserializeList(view)),
+ fileDeserializer.get(), fileDeserializer.get(), fileDeserializer.get()),
new IndexIncrement(
- indexEntrySerializer.deserializeList(view),
- version <= 2
- ? Collections.emptyList()
- : indexEntrySerializer.deserializeList(view)));
+ indexEntryDeserializer.get(),
+ version <= 2 ? Collections.emptyList() : indexEntryDeserializer.get()));
+ }
+
+ private IOExceptionSupplier<List<DataFileMeta>> fileDeserializer(
+ int version, DataInputView view) {
+ if (version > 3) {
Review Comment:
Maybe we should increase this CURRENT_VERSION
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org