You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/09/13 23:19:00 UTC
[jira] [Work logged] (GOBBLIN-2157) Copy Iceberg table properties during iceberg table replication
[ https://issues.apache.org/jira/browse/GOBBLIN-2157?focusedWorklogId=934802&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-934802 ]
ASF GitHub Bot logged work on GOBBLIN-2157:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Sep/24 23:18
Start Date: 13/Sep/24 23:18
Worklog Time Spent: 10m
Work Description: Will-Lo opened a new pull request, #4056:
URL: https://github.com/apache/gobblin/pull/4056
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-2157
### Description
- [ ] Here are some details about my PR, including screenshots (if applicable):
The existing iceberg registration step has the following snippet:
```
if (dstMetadata != null) {
// use current destination metadata as 'base metadata' and source as 'updated metadata' while committing
this.tableOps.commit(dstMetadata, srcMetadata.replaceProperties(dstMetadata.properties())); <<<<<-----
Issue Time Tracking
-------------------
Worklog Id: (was: 934802)
Remaining Estimate: 0h
Time Spent: 10m
> Copy Iceberg table properties during iceberg table replication
> --------------------------------------------------------------
>
> Key: GOBBLIN-2157
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2157
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, Iceberg table copies do not replicate any properties from the source table. This can be unintuitive as the expectation for most pipelines is that the full table data and properties are replicated on both the source and destination.
> We want to copy all the properties from the source table while maintaining any pre-existing properties that were created from the destination table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)