Support relative paths in Iceberg

[Anurag Mantripragada <am...@apple.com.INVALID>]

Hi folks,

Pavan, Pani (cc) and I put together a design doc [1] to support relative paths in Iceberg metadata (Issue #1617 <https://github.com/apache/iceberg/issues/1617>) with three goals in mind.

Backup / Replication should not require ongoing rewrites of metadata. It is acceptable to have a single change when the first copy is performed.
No public API will change as a part of this proposal. 

We have thought through a few scenarios in the doc and also included an implementation. We will glad to to consider other ideas as well. 

[1] - https://docs.google.com/document/d/1RDEjJAVEXg1csRzyzTuM634L88vvI0iDHNQQK3kOVR0/edit?usp=sharing <https://docs.google.com/document/d/1RDEjJAVEXg1csRzyzTuM634L88vvI0iDHNQQK3kOVR0/edit?usp=sharing>

Thanks, 
Anurag

Re: Support relative paths in Iceberg

[Anurag Mantripragada <am...@apple.com.INVALID>]

Hi All, 

Thanks for all the initial feedback on this doc. I just wanted to bubble this up. We are looking for more feedback for this design.

Thank you for your time!

~ Anurag 

> On Jan 21, 2021, at 10:28 AM, Anurag Mantripragada <am...@apple.com.INVALID> wrote:
> 
> Hi folks,
> 
> Pavan, Pani (cc) and I put together a design doc [1] to support relative paths in Iceberg metadata (Issue #1617 <https://github.com/apache/iceberg/issues/1617>) with three goals in mind.
> Reduce the number of metadata files that need to be modified during backup/replication.
> No public API will change as a part of this proposal.
> Relative path changes are confined to metadata files only.
> 
> We have thought through a few scenarios in the doc and also included an implementation. We will be glad to to consider other ideas as well. 
> 
> [1] - https://docs.google.com/document/d/1RDEjJAVEXg1csRzyzTuM634L88vvI0iDHNQQK3kOVR0/edit?usp=sharing <https://docs.google.com/document/d/1RDEjJAVEXg1csRzyzTuM634L88vvI0iDHNQQK3kOVR0/edit?usp=sharing>
> 
> Thanks, 
> Anurag