You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "FangYongs (via GitHub)" <gi...@apache.org> on 2024/01/22 02:34:23 UTC
[PR] [core][branch] Add branches in tags table [incubator-paimon]
FangYongs opened a new pull request, #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #2753
Add branches column in tags table
### Tests
Added test in `TagsTableTest` to get branches
### API and Format
no
### Documentation
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "FangYongs (via GitHub)" <gi...@apache.org>.
FangYongs merged PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "TaoZex (via GitHub)" <gi...@apache.org>.
TaoZex commented on code in PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#discussion_r1461817526
##########
paimon-core/src/main/java/org/apache/paimon/table/system/TagsTable.java:
##########
@@ -202,15 +221,18 @@ public RecordReader<InternalRow> createReader(Split split) {
return new IteratorRecordReader<>(rows);
}
- private InternalRow toRow(Map.Entry<Snapshot, String> tag) {
+ private InternalRow toRow(
+ Map.Entry<Snapshot, String> tag, Map<String, List<String>> tagBranches) {
Snapshot snapshot = tag.getKey();
+ List<String> branches = tagBranches.get(tag.getValue());
return GenericRow.of(
BinaryString.fromString(tag.getValue()),
snapshot.id(),
snapshot.schemaId(),
Timestamp.fromLocalDateTime(
DateTimeUtils.toLocalDateTime(snapshot.timeMillis())),
- snapshot.totalRecordCount());
+ snapshot.totalRecordCount(),
+ BinaryString.fromString(branches == null ? "[]" : branches.toString()));
Review Comment:
same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "FangYongs (via GitHub)" <gi...@apache.org>.
FangYongs commented on PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#issuecomment-1914070754
This PR will wait for #2748
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "FangYongs (via GitHub)" <gi...@apache.org>.
FangYongs commented on code in PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#discussion_r1477280904
##########
docs/content/how-to/system-tables.md:
##########
@@ -198,12 +198,12 @@ and some historical information of the snapshots. You can also get all tag names
SELECT * FROM MyTable$tags;
/*
-+----------+-------------+-----------+-------------------------+--------------+
-| tag_name | snapshot_id | schema_id | commit_time | record_count |
-+----------+-------------+-----------+-------------------------+--------------+
-| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 |
-| tag3 | 3 | 0 | 2023-06-28 14:58:24.691 | 7 |
-+----------+-------------+-----------+-------------------------+--------------+
++----------+-------------+-----------+-------------------------+--------------+--------------+
+| tag_name | snapshot_id | schema_id | commit_time | record_count | branches |
++----------+-------------+-----------+-------------------------+--------------+--------------+
+| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 | [] |
Review Comment:
I think it's better to given empty branches instead of `main` if there is no branch for the tag
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "schnappi17 (via GitHub)" <gi...@apache.org>.
schnappi17 commented on code in PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#discussion_r1474386132
##########
paimon-core/src/main/java/org/apache/paimon/utils/TagManager.java:
##########
@@ -72,6 +72,26 @@ public Path branchTagPath(String branchName, String tagName) {
return new Path(getBranchPath(tablePath, branchName) + "/tag/" + TAG_PREFIX + tagName);
}
+ public String branchTag(String branchName) {
+ try {
+ List<Path> tagPaths =
+ listVersionedFileStatus(
+ fileIO,
+ new Path(getBranchPath(tablePath, branchName) + "/tag/"),
+ TAG_PREFIX)
+ .map(FileStatus::getPath)
+ .collect(Collectors.toList());
+ checkArgument(
+ tagPaths.size() == 1,
+ String.format(
+ "There are multiple tags %s for branch %s",
Review Comment:
I'm confused here, there should be allowed to generate snapshot but not tag on a branch? I don't get the point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "schnappi17 (via GitHub)" <gi...@apache.org>.
schnappi17 commented on PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#issuecomment-1931277839
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "schnappi17 (via GitHub)" <gi...@apache.org>.
schnappi17 commented on code in PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#discussion_r1474386132
##########
paimon-core/src/main/java/org/apache/paimon/utils/TagManager.java:
##########
@@ -72,6 +72,26 @@ public Path branchTagPath(String branchName, String tagName) {
return new Path(getBranchPath(tablePath, branchName) + "/tag/" + TAG_PREFIX + tagName);
}
+ public String branchTag(String branchName) {
+ try {
+ List<Path> tagPaths =
+ listVersionedFileStatus(
+ fileIO,
+ new Path(getBranchPath(tablePath, branchName) + "/tag/"),
+ TAG_PREFIX)
+ .map(FileStatus::getPath)
+ .collect(Collectors.toList());
+ checkArgument(
+ tagPaths.size() == 1,
+ String.format(
+ "There are multiple tags %s for branch %s",
Review Comment:
I'm confused here, why it's not allowed to generate more tags on branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][branch] Add branches in tags table [incubator-paimon]
Posted by "TaoZex (via GitHub)" <gi...@apache.org>.
TaoZex commented on code in PR #2754:
URL: https://github.com/apache/incubator-paimon/pull/2754#discussion_r1461817142
##########
docs/content/how-to/system-tables.md:
##########
@@ -198,12 +198,12 @@ and some historical information of the snapshots. You can also get all tag names
SELECT * FROM MyTable$tags;
/*
-+----------+-------------+-----------+-------------------------+--------------+
-| tag_name | snapshot_id | schema_id | commit_time | record_count |
-+----------+-------------+-----------+-------------------------+--------------+
-| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 |
-| tag3 | 3 | 0 | 2023-06-28 14:58:24.691 | 7 |
-+----------+-------------+-----------+-------------------------+--------------+
++----------+-------------+-----------+-------------------------+--------------+--------------+
+| tag_name | snapshot_id | schema_id | commit_time | record_count | branches |
++----------+-------------+-----------+-------------------------+--------------+--------------+
+| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 | [] |
Review Comment:
```suggestion
| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 | [main] |
```
How about use `main` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org