You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "MonsterChenzhuo (via GitHub)" <gi...@apache.org> on 2024/02/22 16:48:26 UTC
[I] [Feature] Snapshot supports the function of recording additional information of users [incubator-paimon]
MonsterChenzhuo opened a new issue, #2892:
URL: https://github.com/apache/incubator-paimon/issues/2892
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
To add an attribute for recording additional user-stored information in the existing snapshot structure, considering flexibility and expandability, a new attribute named extraInfo can be added to the top-level structure. This attribute stores a key-value pair structure, similar to a dictionary or Map, to facilitate the storage of extra user information.
```json
{
"extraInfo": {}
}
```
Design of extraInfo Structure
Basic Key-Value Pairs: For storing simple user information such as preference settings.
Nested Structure: Supports more complex data structures, such as lists or dictionaries, to store more detailed user data.
```json
{
"extraInfo": {
"preferences": {
"theme": "dark",
"language": "zh-CN"
},
"history": [
{
"action": "login",
"time": "2024-02-23T10:00:00Z"
}
]
}
}
```
Integration into the Existing Structure
The extraInfo attribute is integrated into the structure you provided, with the new structure as follows:
```json
{
"version": 3,
"id": 1,
"schemaId": 0,
"baseManifestList": "manifest-list-cd8aff2b-3fbb-430a-881f-1a5064fd1edd-0",
"deltaManifestList": "manifest-list-cd8aff2b-3fbb-430a-881f-1a5064fd1edd-1",
"changelogManifestList": null,
"indexManifest": "index-manifest-5ef89060-6da5-4561-8e06-0394432e61d4-0",
"commitUser": "bd9de2a7-64d0-4395-958e-f88c7c99383e",
"commitIdentifier": 9223372036854775807,
"commitKind": "APPEND",
"timeMillis": 1708617516783,
"logOffsets": {},
"totalRecordCount": 1,
"deltaRecordCount": 1,
"changelogRecordCount": 0,
"watermark": -9223372036854775808,
"extraInfo": {
"preferences": {
"theme": "dark",
"language": "zh-CN"
},
"history": [
{
"action": "login",
"time": "2024-02-23T10:00:00Z"
}
]
}
}
```
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Snapshot supports the function of recording additional information of users [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on issue #2892:
URL: https://github.com/apache/incubator-paimon/issues/2892#issuecomment-1959854174
@JingsongLi What do you think about this design?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Snapshot supports the function of recording additional information of users [paimon]
Posted by "vanliu-tx (via GitHub)" <gi...@apache.org>.
vanliu-tx commented on issue #2892:
URL: https://github.com/apache/paimon/issues/2892#issuecomment-2316550211
any update on this issue?
we have a scenario that use a Java client to write a paimon primary key table, and need a way to store some information in snapshot. These information would help when the task starts after stop/crash, the task could use the information in snapshot to seek the right position to recover.
@MonsterChenzhuo @JingsongLi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org