[DISCUSS] Proposal for Relative Path Support In Table Spec

[Talat Uyarer <ta...@google.com.INVALID>]

Hi, Iceberg Community,

As mentioned at the last sync, Dan and I have been working on a proposal to
add support for relative paths, which has been a long requested feature.
There have been a number of discussions/proposals over the years, but we'd
like to scope down and refocus effort to make some meaningful progress on
this issue.

Please take a look at the linked doc and provide feedback. We'd love to
open up discussion on this topic at the next community sync and we can hold
one-off syncs on the topic if there's a lot of interest.

You can access Iceberg's First V4 Spec change from here :)

Proposal Issue: https://github.com/apache/iceberg/issues/13141
Doc: https://s.apache.org/iceberg-spec-relative-path

Talat

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[ally heev <al...@gmail.com>]

thanks!! missed the last sync. I have already subscribed to the calendar.
Can I know when is the relative path proposal meeting?(I don't find it in
my cal)

On Sat, Jul 5, 2025 at 12:04 AM Anurag Mantripragada
<am...@apple.com.invalid> wrote:

> In the last community sync, we have decided to discuss relative path
> proposal in a separate meeting. Please subscribe to Iceberg Dev Events[1]
> calendar to join the meeting.
>
>
> Thanks for your interest. Contributions are always welcome. Please reach
> out to authors to see how you can collaborate.
>
>
> [1] - https://iceberg.apache.org/community/#iceberg-community-events
>
> ~ Anurag Mantripragada
>
> On Jul 3, 2025, at 11:59 PM, ally heev <al...@gmail.com> wrote:
>
> Hi folks,
> Do you accept contributions from people other than authors on improvement
> proposals? I would really like to contribute on the implementation side.
> Also, is the spec definition finalized? I see a couple of unaddressed
> queries in the doc
>
>
> On Wed, May 28, 2025 at 3:44 AM Talat Uyarer <ta...@google.com.invalid>
> wrote:
>
>> Hi, Iceberg Community,
>>
>> As mentioned at the last sync, Dan and I have been working on a proposal
>> to add support for relative paths, which has been a long requested feature.
>> There have been a number of discussions/proposals over the years, but we'd
>> like to scope down and refocus effort to make some meaningful progress on
>> this issue.
>>
>> Please take a look at the linked doc and provide feedback. We'd love to
>> open up discussion on this topic at the next community sync and we can hold
>> one-off syncs on the topic if there's a lot of interest.
>>
>> You can access Iceberg's First V4 Spec change from here :)
>>
>> Proposal Issue: https://github.com/apache/iceberg/issues/13141
>> Doc: https://s.apache.org/iceberg-spec-relative-path
>>
>> Talat
>>
>
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Anurag Mantripragada <am...@apple.com.INVALID>]

In the last community sync, we have decided to discuss relative path proposal in a separate meeting. Please subscribe to Iceberg Dev Events[1] calendar to join the meeting. 


Thanks for your interest. Contributions are always welcome. Please reach out to authors to see how you can collaborate. 


[1] - https://iceberg.apache.org/community/#iceberg-community-events

~ Anurag Mantripragada

> On Jul 3, 2025, at 11:59 PM, ally heev <al...@gmail.com> wrote:
> 
> Hi folks,
> Do you accept contributions from people other than authors on improvement proposals? I would really like to contribute on the implementation side. Also, is the spec definition finalized? I see a couple of unaddressed queries in the doc
> 
> 
> On Wed, May 28, 2025 at 3:44 AM Talat Uyarer <ta...@google.com.invalid> wrote:
>> Hi, Iceberg Community,
>> 
>> As mentioned at the last sync, Dan and I have been working on a proposal to add support for relative paths, which has been a long requested feature. There have been a number of discussions/proposals over the years, but we'd like to scope down and refocus effort to make some meaningful progress on this issue.
>> 
>> Please take a look at the linked doc and provide feedback. We'd love to open up discussion on this topic at the next community sync and we can hold one-off syncs on the topic if there's a lot of interest.
>> 
>> You can access Iceberg's First V4 Spec change from here :)
>> 
>> Proposal Issue: https://github.com/apache/iceberg/issues/13141
>> Doc: https://s.apache.org/iceberg-spec-relative-path
>> 
>> Talat

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[ally heev <al...@gmail.com>]

Hi folks,
Do you accept contributions from people other than authors on improvement
proposals? I would really like to contribute on the implementation side.
Also, is the spec definition finalized? I see a couple of unaddressed
queries in the doc


On Wed, May 28, 2025 at 3:44 AM Talat Uyarer <ta...@google.com.invalid>
wrote:

> Hi, Iceberg Community,
>
> As mentioned at the last sync, Dan and I have been working on a proposal
> to add support for relative paths, which has been a long requested feature.
> There have been a number of discussions/proposals over the years, but we'd
> like to scope down and refocus effort to make some meaningful progress on
> this issue.
>
> Please take a look at the linked doc and provide feedback. We'd love to
> open up discussion on this topic at the next community sync and we can hold
> one-off syncs on the topic if there's a lot of interest.
>
> You can access Iceberg's First V4 Spec change from here :)
>
> Proposal Issue: https://github.com/apache/iceberg/issues/13141
> Doc: https://s.apache.org/iceberg-spec-relative-path
>
> Talat
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Hey everyone,

I wanted to follow up before sending out a vote for the spec additions
regarding relative paths because the discussed proposal had a non-trivial
update.

Early in the proposal process we went back and forth on whether (and where)
to include separator characters.  After a few rounds of discussion we
settled on including the path separator in the relative portion of the path
and limiting references to that separator.

Anoop and Steven pointed out an issue when relativizing similar paths; for
example: 'table' vs 'table_v2' (more detail in this comment thread
<https://github.com/apache/iceberg/pull/16174#discussion_r3228742112>).
This also led us to reconsider other aspects of how we treat the
representation and separator character.  Ultimately, it's more clear in the
spec to explicitly call out handling of the separator character and join
the table location and relative path on the URI separator character, '/'.

This has a number of advantages in that it makes relativization
unambiguous, is more consistent with how people and other formats like
Delta, Lance, Paimon, and Hudi reference relative paths, and makes the
language in the spec around separators more clear.

Please take a look at the updated PR
<https://github.com/apache/iceberg/pull/15630/changes> if you have
questions/comments.

-Dan

On Mon, Apr 20, 2026 at 2:20 PM Daniel Weeks <dw...@apache.org> wrote:

> Thanks to everyone who provided feedback.
>
> I've incorporated feedback from the first round and updated the PR.
>
> Please take a second (or first) look.
>
> -Dan
>
> On Mon, Mar 23, 2026 at 1:05 PM Daniel Weeks <dw...@apache.org> wrote:
>
>> Hey everyone,
>>
>> If you're interested in the first round of spec related updates for
>> relative paths, please take a look and add comments:
>> https://github.com/apache/iceberg/pull/15630
>>
>> -Dan
>>
>> On Mon, Mar 23, 2026 at 1:04 PM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> Hey Steve,
>>>
>>> I'm not sure if you were able get an answer on this question in any of
>>> the follow up discussions we had on relative paths, but the situation you
>>> describe is inherent to the difference between absolute and relative paths.
>>>
>>> The spec isn't responsible for how you relocate/duplicate/etc data if
>>> the base component of the relative path is updated and is explicitly not
>>> covered by the design.  That's the responsibility of the catalog or
>>> implementation.
>>>
>>> If you want data persistence across metadata moves, you always have the
>>> ability to produce absolute paths to retain the v1-3 behavior.  However, I
>>> believe what we've learned through production deployments and in comparison
>>> to other formats, is that primary use case is to either relocate the entire
>>> dataset or duplicate the entire dataset, which is the basis for the
>>> relative path model described in the proposal.
>>>
>>> As to the catalog handling, most (all?) implementations either do not
>>> natively support rename (like HadoopCatalog) and others treat rename as a
>>> metadata only operation but do not change the table location.  The closest
>>> thing is probably register table in the REST catalog, but that is very much
>>> left up to the catalog implementation.  I think we can draw from this that
>>> most table relocations are being performed outside of the catalog an then
>>> registered in the catalog.
>>>
>>> -Dan
>>>
>>>
>>>
>>>
>>> On Wed, Feb 4, 2026 at 2:27 PM Steve <ho...@gmail.com> wrote:
>>>
>>>> Thanks all,
>>>>
>>>>   Following the relative path discussion last week, I want to raise a
>>>> question about lifecycle clean up operations in the context of table
>>>> location mutability.
>>>> The current proposal established that "*the table location is the
>>>> basis for all path resolution against persisted relative paths*".
>>>> Since location remains mutable, this creates a behavioral difference
>>>> between v3 and v4 tables that increases operational complexity. Here's a
>>>> concrete example:.
>>>>
>>>> *Scenario*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *CREATE TABLE prod.db.events (  event_id BIGINT,  event_time
>>>> TIMESTAMP,  payload STRING) USING icebergLOCATION
>>>> 's3://bucket-a/warehouse/events';-- Insert some dataINSERT INTO
>>>> prod.db.events VALUES (1, current_timestamp(), 'data1');INSERT INTO
>>>> prod.db.events VALUES (2, current_timestamp(), 'data2');-- User changes
>>>> location (Spark)ALTER TABLE prod.db.events SET location
>>>> 's3://bucket-b/warehouse/events';-- Write new dataINSERT INTO
>>>> prod.db.events VALUES (3, current_timestamp(), 'data3');*
>>>>
>>>> *Result for v3 table on absolute path *
>>>> Manifest entries:
>>>>   - s3://bucket-a/warehouse/events/data/file1.parquet  (absolute - old
>>>> location)
>>>>   - s3://bucket-a/warehouse/events/data/file2.parquet  (absolute - old
>>>> location)
>>>>   - s3://bucket-b/warehouse/events/data/file3.parquet  (absolute - new
>>>> location)
>>>> Reads work out of the box as path are absolute
>>>> Snapshot expiration will cover both locations before and after the
>>>> change as iceberg metadata tracks the path at the time of creation
>>>> Orphan removal is limited as it will only respect only the latest
>>>> location
>>>>
>>>> *Result for v4 table on relative path*
>>>> Manifest entries:
>>>>   - file1.parquet  (relative - written when location was bucket-a)
>>>>   - file2.parquet  (relative - written when location was bucket-a)
>>>>   - file3.parquet  (relative - written when location is bucket-b)
>>>> Path resolution for file1.parquet:
>>>>   Resolved: s3://bucket-b/warehouse/events/data/file1.parquet  ❌
>>>>   Actual:   s3://bucket-a/warehouse/events/data/file1.parquet
>>>> Reads will fail after location change unless files are physically moved
>>>> (either by catalog or by background process)
>>>> Snapshot expiration and orphan removal will not cover locations before
>>>> the update.
>>>>
>>>> *Question*
>>>> In v1-3, updating location is a lightweight, metadata-only operation
>>>> which only impacts future writes, and existing absolute paths continue to
>>>> resolve correctly for read. In v4, this is no longer the case. A location
>>>> update becomes a breaking change that requires physical file movement to
>>>> maintain correctness. From what I can tell, a catalog can either validate
>>>> and handle the movement, rewrite paths to absolute, or reject the update to
>>>> make location effectively immutable. Understandably, the iceberg spec does
>>>> not want to prescribe the catalog guidance, but should we acknowledge this
>>>> behavior change and document the lifecycle cleanup implications? Would be
>>>> great if we can disucss further before the spec is finalized.
>>>>
>>>> Thanks,
>>>> Steve Zhang
>>>>
>>>>
>>>>
>>>> On Thu, Jan 29, 2026 at 5:48 PM Talat Uyarer via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> We had a productive meeting today regarding the Relative Paths
>>>>> proposal.
>>>>>
>>>>> We've reached a general agreement on the approach. The changes will
>>>>> involve explicitly defining path terminology (such as "absolute location")
>>>>> and should be well-contained within a new section on Table Spec.
>>>>>
>>>>> The next step is to open a PR with the proposed changes, which may
>>>>> include knock-on effects for the REST specification, such as updates to
>>>>> register table and load table requests.
>>>>>
>>>>> If you'd like to access the meeting notes:
>>>>> https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing
>>>>>
>>>>> You can find the recording here:
>>>>> https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing
>>>>>
>>>>> Thanks for everyone
>>>>>
>>>>> Talat
>>>>>
>>>>> On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon
>>>>> <wy...@cloudera.com.invalid> wrote:
>>>>>
>>>>>> Dan,
>>>>>> Thanks for the clarifications.
>>>>>> Looking forward to the sync.
>>>>>> - Wing Yew
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Wing Yu
>>>>>>>
>>>>>>> I see that you have been updating the Google doc containing the
>>>>>>>> proposal.
>>>>>>>
>>>>>>>
>>>>>>> That's correct, I've been working with Talat to update the doc based
>>>>>>> on feedback from the comments and first round of discussion we had on this
>>>>>>> topic.
>>>>>>>
>>>>>>> Looking through it now, as far as I can tell, the basic idea (from
>>>>>>>> the original proposal) of inferring the table location from the path to the
>>>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>>>
>>>>>>>
>>>>>>> So far, nothing has changed about table location inference, but we
>>>>>>> will probably be revisiting this with respect to other
>>>>>>> updates/clarifications.  There are still a couple open comments related to
>>>>>>> this point, but it is one of the main goals.
>>>>>>>
>>>>>>> You have added clarification around how the path to the metadata is
>>>>>>>> constructed from table location (from which the table location is thus
>>>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>>>> the last sync to discuss the proposal.
>>>>>>>
>>>>>>>
>>>>>>> I don't believe this was the sticking point from the original
>>>>>>> discussion.  Having multiple copies/locations of the same data files under
>>>>>>> a single table's management is explicitly a non-goal.  It was discussed in
>>>>>>> the comments of the doc for caching/fallback use cases, but I think that's
>>>>>>> better handled by specific engine/fileio implementations.
>>>>>>>
>>>>>>> The main sticking points were confusion around the complexity of how
>>>>>>> paths are constructed/persisted and the interplay between
>>>>>>> table/metadata/data locations depending on how those values are set in the
>>>>>>> table metadata.  Based on that feedback, we're suggesting some changes,
>>>>>>> which is primarily consist of: 1) defining path construction, resolution,
>>>>>>> and relativization separately, 2) making all paths relative to the table
>>>>>>> location (which simplifies resolution/relativization, 3) address
>>>>>>> confusing/complex issues like path separators and expectations around
>>>>>>> separators.
>>>>>>>
>>>>>>> We're still in the process of updating the document, but we will
>>>>>>> schedule another sync to discuss these updates in detail and address a few
>>>>>>> points that are still outstanding.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dan
>>>>>>>
>>>>>>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon
>>>>>>> <wy...@cloudera.com.invalid> wrote:
>>>>>>>
>>>>>>>> Hi Daniel Weeks,
>>>>>>>> I see that you have been updating the Google doc containing the
>>>>>>>> proposal.
>>>>>>>> Looking through it now, as far as I can tell, the basic idea (from
>>>>>>>> the original proposal) of inferring the table location from the path to the
>>>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>>>> You have added clarification around how the path to the metadata is
>>>>>>>> constructed from table location (from which the table location is thus
>>>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>>>> the last sync to discuss the proposal.
>>>>>>>> Do you intend to have another sync to continue the discussion?
>>>>>>>> Thanks,
>>>>>>>> Wing Yew
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Thanks Kevin, yes, I see the recording link too but don’t have
>>>>>>>>> access. I have requested access.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ~ Anurag Mantripragada
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>>>>>>> there's already a link for the recording associated with the gcal event but
>>>>>>>>> I dont have access to it.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Kevin Liu
>>>>>>>>>
>>>>>>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>>>>
>>>>>>>>>> Hey folks, was the sync recorded? I missed it due to calendar
>>>>>>>>>> sync issues :(
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ~ Anurag Mantripragada
>>>>>>>>>>
>>>>>>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks. I can see it now
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I can see the new event on the dev calendar.
>>>>>>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>>>>>>
>>>>>>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Kevin Liu
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Ally (and everyone else).
>>>>>>>>>>>>
>>>>>>>>>>>> We hadn't scheduled the discussion for relative paths, but I
>>>>>>>>>>>> just added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>>>>>>
>>>>>>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>>>>>>
>>>>>>>>>>>> -Dan
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <
>>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Talat
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> JB
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Hi, Iceberg Community,
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > As mentioned at the last sync, Dan and I have been working
>>>>>>>>>>>>> on a proposal to add support for relative paths, which has been a long
>>>>>>>>>>>>> requested feature. There have been a number of discussions/proposals over
>>>>>>>>>>>>> the years, but we'd like to scope down and refocus effort to make some
>>>>>>>>>>>>> meaningful progress on this issue.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Please take a look at the linked doc and provide feedback.
>>>>>>>>>>>>> We'd love to open up discussion on this topic at the next community sync
>>>>>>>>>>>>> and we can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Proposal Issue:
>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13141
>>>>>>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Talat
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Thanks to everyone who provided feedback.

I've incorporated feedback from the first round and updated the PR.

Please take a second (or first) look.

-Dan

On Mon, Mar 23, 2026 at 1:05 PM Daniel Weeks <dw...@apache.org> wrote:

> Hey everyone,
>
> If you're interested in the first round of spec related updates for
> relative paths, please take a look and add comments:
> https://github.com/apache/iceberg/pull/15630
>
> -Dan
>
> On Mon, Mar 23, 2026 at 1:04 PM Daniel Weeks <dw...@apache.org> wrote:
>
>> Hey Steve,
>>
>> I'm not sure if you were able get an answer on this question in any of
>> the follow up discussions we had on relative paths, but the situation you
>> describe is inherent to the difference between absolute and relative paths.
>>
>> The spec isn't responsible for how you relocate/duplicate/etc data if the
>> base component of the relative path is updated and is explicitly not
>> covered by the design.  That's the responsibility of the catalog or
>> implementation.
>>
>> If you want data persistence across metadata moves, you always have the
>> ability to produce absolute paths to retain the v1-3 behavior.  However, I
>> believe what we've learned through production deployments and in comparison
>> to other formats, is that primary use case is to either relocate the entire
>> dataset or duplicate the entire dataset, which is the basis for the
>> relative path model described in the proposal.
>>
>> As to the catalog handling, most (all?) implementations either do not
>> natively support rename (like HadoopCatalog) and others treat rename as a
>> metadata only operation but do not change the table location.  The closest
>> thing is probably register table in the REST catalog, but that is very much
>> left up to the catalog implementation.  I think we can draw from this that
>> most table relocations are being performed outside of the catalog an then
>> registered in the catalog.
>>
>> -Dan
>>
>>
>>
>>
>> On Wed, Feb 4, 2026 at 2:27 PM Steve <ho...@gmail.com> wrote:
>>
>>> Thanks all,
>>>
>>>   Following the relative path discussion last week, I want to raise a
>>> question about lifecycle clean up operations in the context of table
>>> location mutability.
>>> The current proposal established that "*the table location is the basis
>>> for all path resolution against persisted relative paths*". Since
>>> location remains mutable, this creates a behavioral difference between v3
>>> and v4 tables that increases operational complexity. Here's a concrete
>>> example:.
>>>
>>> *Scenario*
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *CREATE TABLE prod.db.events (  event_id BIGINT,  event_time TIMESTAMP,
>>> payload STRING) USING icebergLOCATION 's3://bucket-a/warehouse/events';--
>>> Insert some dataINSERT INTO prod.db.events VALUES (1, current_timestamp(),
>>> 'data1');INSERT INTO prod.db.events VALUES (2, current_timestamp(),
>>> 'data2');-- User changes location (Spark)ALTER TABLE prod.db.events SET
>>> location 's3://bucket-b/warehouse/events';-- Write new dataINSERT INTO
>>> prod.db.events VALUES (3, current_timestamp(), 'data3');*
>>>
>>> *Result for v3 table on absolute path *
>>> Manifest entries:
>>>   - s3://bucket-a/warehouse/events/data/file1.parquet  (absolute - old
>>> location)
>>>   - s3://bucket-a/warehouse/events/data/file2.parquet  (absolute - old
>>> location)
>>>   - s3://bucket-b/warehouse/events/data/file3.parquet  (absolute - new
>>> location)
>>> Reads work out of the box as path are absolute
>>> Snapshot expiration will cover both locations before and after the
>>> change as iceberg metadata tracks the path at the time of creation
>>> Orphan removal is limited as it will only respect only the latest
>>> location
>>>
>>> *Result for v4 table on relative path*
>>> Manifest entries:
>>>   - file1.parquet  (relative - written when location was bucket-a)
>>>   - file2.parquet  (relative - written when location was bucket-a)
>>>   - file3.parquet  (relative - written when location is bucket-b)
>>> Path resolution for file1.parquet:
>>>   Resolved: s3://bucket-b/warehouse/events/data/file1.parquet  ❌
>>>   Actual:   s3://bucket-a/warehouse/events/data/file1.parquet
>>> Reads will fail after location change unless files are physically moved
>>> (either by catalog or by background process)
>>> Snapshot expiration and orphan removal will not cover locations before
>>> the update.
>>>
>>> *Question*
>>> In v1-3, updating location is a lightweight, metadata-only operation
>>> which only impacts future writes, and existing absolute paths continue to
>>> resolve correctly for read. In v4, this is no longer the case. A location
>>> update becomes a breaking change that requires physical file movement to
>>> maintain correctness. From what I can tell, a catalog can either validate
>>> and handle the movement, rewrite paths to absolute, or reject the update to
>>> make location effectively immutable. Understandably, the iceberg spec does
>>> not want to prescribe the catalog guidance, but should we acknowledge this
>>> behavior change and document the lifecycle cleanup implications? Would be
>>> great if we can disucss further before the spec is finalized.
>>>
>>> Thanks,
>>> Steve Zhang
>>>
>>>
>>>
>>> On Thu, Jan 29, 2026 at 5:48 PM Talat Uyarer via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Hi All,
>>>>
>>>> We had a productive meeting today regarding the Relative Paths proposal.
>>>>
>>>> We've reached a general agreement on the approach. The changes will
>>>> involve explicitly defining path terminology (such as "absolute location")
>>>> and should be well-contained within a new section on Table Spec.
>>>>
>>>> The next step is to open a PR with the proposed changes, which may
>>>> include knock-on effects for the REST specification, such as updates to
>>>> register table and load table requests.
>>>>
>>>> If you'd like to access the meeting notes:
>>>> https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing
>>>>
>>>> You can find the recording here:
>>>> https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing
>>>>
>>>> Thanks for everyone
>>>>
>>>> Talat
>>>>
>>>> On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon
>>>> <wy...@cloudera.com.invalid> wrote:
>>>>
>>>>> Dan,
>>>>> Thanks for the clarifications.
>>>>> Looking forward to the sync.
>>>>> - Wing Yew
>>>>>
>>>>>
>>>>> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:
>>>>>
>>>>>> Hey Wing Yu
>>>>>>
>>>>>> I see that you have been updating the Google doc containing the
>>>>>>> proposal.
>>>>>>
>>>>>>
>>>>>> That's correct, I've been working with Talat to update the doc based
>>>>>> on feedback from the comments and first round of discussion we had on this
>>>>>> topic.
>>>>>>
>>>>>> Looking through it now, as far as I can tell, the basic idea (from
>>>>>>> the original proposal) of inferring the table location from the path to the
>>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>>
>>>>>>
>>>>>> So far, nothing has changed about table location inference, but we
>>>>>> will probably be revisiting this with respect to other
>>>>>> updates/clarifications.  There are still a couple open comments related to
>>>>>> this point, but it is one of the main goals.
>>>>>>
>>>>>> You have added clarification around how the path to the metadata is
>>>>>>> constructed from table location (from which the table location is thus
>>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>>> the last sync to discuss the proposal.
>>>>>>
>>>>>>
>>>>>> I don't believe this was the sticking point from the original
>>>>>> discussion.  Having multiple copies/locations of the same data files under
>>>>>> a single table's management is explicitly a non-goal.  It was discussed in
>>>>>> the comments of the doc for caching/fallback use cases, but I think that's
>>>>>> better handled by specific engine/fileio implementations.
>>>>>>
>>>>>> The main sticking points were confusion around the complexity of how
>>>>>> paths are constructed/persisted and the interplay between
>>>>>> table/metadata/data locations depending on how those values are set in the
>>>>>> table metadata.  Based on that feedback, we're suggesting some changes,
>>>>>> which is primarily consist of: 1) defining path construction, resolution,
>>>>>> and relativization separately, 2) making all paths relative to the table
>>>>>> location (which simplifies resolution/relativization, 3) address
>>>>>> confusing/complex issues like path separators and expectations around
>>>>>> separators.
>>>>>>
>>>>>> We're still in the process of updating the document, but we will
>>>>>> schedule another sync to discuss these updates in detail and address a few
>>>>>> points that are still outstanding.
>>>>>>
>>>>>> Thanks,
>>>>>> Dan
>>>>>>
>>>>>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon
>>>>>> <wy...@cloudera.com.invalid> wrote:
>>>>>>
>>>>>>> Hi Daniel Weeks,
>>>>>>> I see that you have been updating the Google doc containing the
>>>>>>> proposal.
>>>>>>> Looking through it now, as far as I can tell, the basic idea (from
>>>>>>> the original proposal) of inferring the table location from the path to the
>>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>>> You have added clarification around how the path to the metadata is
>>>>>>> constructed from table location (from which the table location is thus
>>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>>> the last sync to discuss the proposal.
>>>>>>> Do you intend to have another sync to continue the discussion?
>>>>>>> Thanks,
>>>>>>> Wing Yew
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>>
>>>>>>>> Thanks Kevin, yes, I see the recording link too but don’t have
>>>>>>>> access. I have requested access.
>>>>>>>>
>>>>>>>>
>>>>>>>> ~ Anurag Mantripragada
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>>>>>> there's already a link for the recording associated with the gcal event but
>>>>>>>> I dont have access to it.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Kevin Liu
>>>>>>>>
>>>>>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>>>>>>> issues :(
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ~ Anurag Mantripragada
>>>>>>>>>
>>>>>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Thanks. I can see it now
>>>>>>>>>
>>>>>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I can see the new event on the dev calendar.
>>>>>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>>>>>
>>>>>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Kevin Liu
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Ally (and everyone else).
>>>>>>>>>>>
>>>>>>>>>>> We hadn't scheduled the discussion for relative paths, but I
>>>>>>>>>>> just added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>>>>>
>>>>>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>>>>>
>>>>>>>>>>> -Dan
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <
>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Talat
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>>> JB
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Hi, Iceberg Community,
>>>>>>>>>>>> >
>>>>>>>>>>>> > As mentioned at the last sync, Dan and I have been working on
>>>>>>>>>>>> a proposal to add support for relative paths, which has been a long
>>>>>>>>>>>> requested feature. There have been a number of discussions/proposals over
>>>>>>>>>>>> the years, but we'd like to scope down and refocus effort to make some
>>>>>>>>>>>> meaningful progress on this issue.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Please take a look at the linked doc and provide feedback.
>>>>>>>>>>>> We'd love to open up discussion on this topic at the next community sync
>>>>>>>>>>>> and we can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>>>>>> >
>>>>>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>>>>>> >
>>>>>>>>>>>> > Proposal Issue:
>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13141
>>>>>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>>>>>> >
>>>>>>>>>>>> > Talat
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Hey everyone,

If you're interested in the first round of spec related updates for
relative paths, please take a look and add comments:
https://github.com/apache/iceberg/pull/15630

-Dan

On Mon, Mar 23, 2026 at 1:04 PM Daniel Weeks <dw...@apache.org> wrote:

> Hey Steve,
>
> I'm not sure if you were able get an answer on this question in any of the
> follow up discussions we had on relative paths, but the situation you
> describe is inherent to the difference between absolute and relative paths.
>
> The spec isn't responsible for how you relocate/duplicate/etc data if the
> base component of the relative path is updated and is explicitly not
> covered by the design.  That's the responsibility of the catalog or
> implementation.
>
> If you want data persistence across metadata moves, you always have the
> ability to produce absolute paths to retain the v1-3 behavior.  However, I
> believe what we've learned through production deployments and in comparison
> to other formats, is that primary use case is to either relocate the entire
> dataset or duplicate the entire dataset, which is the basis for the
> relative path model described in the proposal.
>
> As to the catalog handling, most (all?) implementations either do not
> natively support rename (like HadoopCatalog) and others treat rename as a
> metadata only operation but do not change the table location.  The closest
> thing is probably register table in the REST catalog, but that is very much
> left up to the catalog implementation.  I think we can draw from this that
> most table relocations are being performed outside of the catalog an then
> registered in the catalog.
>
> -Dan
>
>
>
>
> On Wed, Feb 4, 2026 at 2:27 PM Steve <ho...@gmail.com> wrote:
>
>> Thanks all,
>>
>>   Following the relative path discussion last week, I want to raise a
>> question about lifecycle clean up operations in the context of table
>> location mutability.
>> The current proposal established that "*the table location is the basis
>> for all path resolution against persisted relative paths*". Since
>> location remains mutable, this creates a behavioral difference between v3
>> and v4 tables that increases operational complexity. Here's a concrete
>> example:.
>>
>> *Scenario*
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *CREATE TABLE prod.db.events (  event_id BIGINT,  event_time TIMESTAMP,
>> payload STRING) USING icebergLOCATION 's3://bucket-a/warehouse/events';--
>> Insert some dataINSERT INTO prod.db.events VALUES (1, current_timestamp(),
>> 'data1');INSERT INTO prod.db.events VALUES (2, current_timestamp(),
>> 'data2');-- User changes location (Spark)ALTER TABLE prod.db.events SET
>> location 's3://bucket-b/warehouse/events';-- Write new dataINSERT INTO
>> prod.db.events VALUES (3, current_timestamp(), 'data3');*
>>
>> *Result for v3 table on absolute path *
>> Manifest entries:
>>   - s3://bucket-a/warehouse/events/data/file1.parquet  (absolute - old
>> location)
>>   - s3://bucket-a/warehouse/events/data/file2.parquet  (absolute - old
>> location)
>>   - s3://bucket-b/warehouse/events/data/file3.parquet  (absolute - new
>> location)
>> Reads work out of the box as path are absolute
>> Snapshot expiration will cover both locations before and after the change
>> as iceberg metadata tracks the path at the time of creation
>> Orphan removal is limited as it will only respect only the latest location
>>
>> *Result for v4 table on relative path*
>> Manifest entries:
>>   - file1.parquet  (relative - written when location was bucket-a)
>>   - file2.parquet  (relative - written when location was bucket-a)
>>   - file3.parquet  (relative - written when location is bucket-b)
>> Path resolution for file1.parquet:
>>   Resolved: s3://bucket-b/warehouse/events/data/file1.parquet  ❌
>>   Actual:   s3://bucket-a/warehouse/events/data/file1.parquet
>> Reads will fail after location change unless files are physically moved
>> (either by catalog or by background process)
>> Snapshot expiration and orphan removal will not cover locations before
>> the update.
>>
>> *Question*
>> In v1-3, updating location is a lightweight, metadata-only operation
>> which only impacts future writes, and existing absolute paths continue to
>> resolve correctly for read. In v4, this is no longer the case. A location
>> update becomes a breaking change that requires physical file movement to
>> maintain correctness. From what I can tell, a catalog can either validate
>> and handle the movement, rewrite paths to absolute, or reject the update to
>> make location effectively immutable. Understandably, the iceberg spec does
>> not want to prescribe the catalog guidance, but should we acknowledge this
>> behavior change and document the lifecycle cleanup implications? Would be
>> great if we can disucss further before the spec is finalized.
>>
>> Thanks,
>> Steve Zhang
>>
>>
>>
>> On Thu, Jan 29, 2026 at 5:48 PM Talat Uyarer via dev <
>> dev@iceberg.apache.org> wrote:
>>
>>> Hi All,
>>>
>>> We had a productive meeting today regarding the Relative Paths proposal.
>>>
>>> We've reached a general agreement on the approach. The changes will
>>> involve explicitly defining path terminology (such as "absolute location")
>>> and should be well-contained within a new section on Table Spec.
>>>
>>> The next step is to open a PR with the proposed changes, which may
>>> include knock-on effects for the REST specification, such as updates to
>>> register table and load table requests.
>>>
>>> If you'd like to access the meeting notes:
>>> https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing
>>>
>>> You can find the recording here:
>>> https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing
>>>
>>> Thanks for everyone
>>>
>>> Talat
>>>
>>> On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon
>>> <wy...@cloudera.com.invalid> wrote:
>>>
>>>> Dan,
>>>> Thanks for the clarifications.
>>>> Looking forward to the sync.
>>>> - Wing Yew
>>>>
>>>>
>>>> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:
>>>>
>>>>> Hey Wing Yu
>>>>>
>>>>> I see that you have been updating the Google doc containing the
>>>>>> proposal.
>>>>>
>>>>>
>>>>> That's correct, I've been working with Talat to update the doc based
>>>>> on feedback from the comments and first round of discussion we had on this
>>>>> topic.
>>>>>
>>>>> Looking through it now, as far as I can tell, the basic idea (from the
>>>>>> original proposal) of inferring the table location from the path to the
>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>
>>>>>
>>>>> So far, nothing has changed about table location inference, but we
>>>>> will probably be revisiting this with respect to other
>>>>> updates/clarifications.  There are still a couple open comments related to
>>>>> this point, but it is one of the main goals.
>>>>>
>>>>> You have added clarification around how the path to the metadata is
>>>>>> constructed from table location (from which the table location is thus
>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>> the last sync to discuss the proposal.
>>>>>
>>>>>
>>>>> I don't believe this was the sticking point from the original
>>>>> discussion.  Having multiple copies/locations of the same data files under
>>>>> a single table's management is explicitly a non-goal.  It was discussed in
>>>>> the comments of the doc for caching/fallback use cases, but I think that's
>>>>> better handled by specific engine/fileio implementations.
>>>>>
>>>>> The main sticking points were confusion around the complexity of how
>>>>> paths are constructed/persisted and the interplay between
>>>>> table/metadata/data locations depending on how those values are set in the
>>>>> table metadata.  Based on that feedback, we're suggesting some changes,
>>>>> which is primarily consist of: 1) defining path construction, resolution,
>>>>> and relativization separately, 2) making all paths relative to the table
>>>>> location (which simplifies resolution/relativization, 3) address
>>>>> confusing/complex issues like path separators and expectations around
>>>>> separators.
>>>>>
>>>>> We're still in the process of updating the document, but we will
>>>>> schedule another sync to discuss these updates in detail and address a few
>>>>> points that are still outstanding.
>>>>>
>>>>> Thanks,
>>>>> Dan
>>>>>
>>>>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon
>>>>> <wy...@cloudera.com.invalid> wrote:
>>>>>
>>>>>> Hi Daniel Weeks,
>>>>>> I see that you have been updating the Google doc containing the
>>>>>> proposal.
>>>>>> Looking through it now, as far as I can tell, the basic idea (from
>>>>>> the original proposal) of inferring the table location from the path to the
>>>>>> current metadata.json has not changed. Is my reading correct?
>>>>>> You have added clarification around how the path to the metadata is
>>>>>> constructed from table location (from which the table location is thus
>>>>>> reverse engineered) and around path relativization, but the original idea
>>>>>> does not appear to have changed. In that case, the use case of having a
>>>>>> single copy of metadata but more than one copy of data (two or more
>>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>>> the last sync to discuss the proposal.
>>>>>> Do you intend to have another sync to continue the discussion?
>>>>>> Thanks,
>>>>>> Wing Yew
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>
>>>>>>> Thanks Kevin, yes, I see the recording link too but don’t have
>>>>>>> access. I have requested access.
>>>>>>>
>>>>>>>
>>>>>>> ~ Anurag Mantripragada
>>>>>>>
>>>>>>>
>>>>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>>>>> there's already a link for the recording associated with the gcal event but
>>>>>>> I dont have access to it.
>>>>>>>
>>>>>>> Best,
>>>>>>> Kevin Liu
>>>>>>>
>>>>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>>
>>>>>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>>>>>> issues :(
>>>>>>>>
>>>>>>>>
>>>>>>>> ~ Anurag Mantripragada
>>>>>>>>
>>>>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Thanks. I can see it now
>>>>>>>>
>>>>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I can see the new event on the dev calendar.
>>>>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>>>>
>>>>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Kevin Liu
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Ally (and everyone else).
>>>>>>>>>>
>>>>>>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>>>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>>>>
>>>>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>>>>
>>>>>>>>>> -Dan
>>>>>>>>>>
>>>>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <
>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Talat
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> JB
>>>>>>>>>>>
>>>>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> > Hi, Iceberg Community,
>>>>>>>>>>> >
>>>>>>>>>>> > As mentioned at the last sync, Dan and I have been working on
>>>>>>>>>>> a proposal to add support for relative paths, which has been a long
>>>>>>>>>>> requested feature. There have been a number of discussions/proposals over
>>>>>>>>>>> the years, but we'd like to scope down and refocus effort to make some
>>>>>>>>>>> meaningful progress on this issue.
>>>>>>>>>>> >
>>>>>>>>>>> > Please take a look at the linked doc and provide feedback.
>>>>>>>>>>> We'd love to open up discussion on this topic at the next community sync
>>>>>>>>>>> and we can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>>>>> >
>>>>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>>>>> >
>>>>>>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>>>>> >
>>>>>>>>>>> > Talat
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Hey Steve,

I'm not sure if you were able get an answer on this question in any of the
follow up discussions we had on relative paths, but the situation you
describe is inherent to the difference between absolute and relative paths.

The spec isn't responsible for how you relocate/duplicate/etc data if the
base component of the relative path is updated and is explicitly not
covered by the design.  That's the responsibility of the catalog or
implementation.

If you want data persistence across metadata moves, you always have the
ability to produce absolute paths to retain the v1-3 behavior.  However, I
believe what we've learned through production deployments and in comparison
to other formats, is that primary use case is to either relocate the entire
dataset or duplicate the entire dataset, which is the basis for the
relative path model described in the proposal.

As to the catalog handling, most (all?) implementations either do not
natively support rename (like HadoopCatalog) and others treat rename as a
metadata only operation but do not change the table location.  The closest
thing is probably register table in the REST catalog, but that is very much
left up to the catalog implementation.  I think we can draw from this that
most table relocations are being performed outside of the catalog an then
registered in the catalog.

-Dan




On Wed, Feb 4, 2026 at 2:27 PM Steve <ho...@gmail.com> wrote:

> Thanks all,
>
>   Following the relative path discussion last week, I want to raise a
> question about lifecycle clean up operations in the context of table
> location mutability.
> The current proposal established that "*the table location is the basis
> for all path resolution against persisted relative paths*". Since
> location remains mutable, this creates a behavioral difference between v3
> and v4 tables that increases operational complexity. Here's a concrete
> example:.
>
> *Scenario*
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *CREATE TABLE prod.db.events (  event_id BIGINT,  event_time TIMESTAMP,
> payload STRING) USING icebergLOCATION 's3://bucket-a/warehouse/events';--
> Insert some dataINSERT INTO prod.db.events VALUES (1, current_timestamp(),
> 'data1');INSERT INTO prod.db.events VALUES (2, current_timestamp(),
> 'data2');-- User changes location (Spark)ALTER TABLE prod.db.events SET
> location 's3://bucket-b/warehouse/events';-- Write new dataINSERT INTO
> prod.db.events VALUES (3, current_timestamp(), 'data3');*
>
> *Result for v3 table on absolute path *
> Manifest entries:
>   - s3://bucket-a/warehouse/events/data/file1.parquet  (absolute - old
> location)
>   - s3://bucket-a/warehouse/events/data/file2.parquet  (absolute - old
> location)
>   - s3://bucket-b/warehouse/events/data/file3.parquet  (absolute - new
> location)
> Reads work out of the box as path are absolute
> Snapshot expiration will cover both locations before and after the change
> as iceberg metadata tracks the path at the time of creation
> Orphan removal is limited as it will only respect only the latest location
>
> *Result for v4 table on relative path*
> Manifest entries:
>   - file1.parquet  (relative - written when location was bucket-a)
>   - file2.parquet  (relative - written when location was bucket-a)
>   - file3.parquet  (relative - written when location is bucket-b)
> Path resolution for file1.parquet:
>   Resolved: s3://bucket-b/warehouse/events/data/file1.parquet  ❌
>   Actual:   s3://bucket-a/warehouse/events/data/file1.parquet
> Reads will fail after location change unless files are physically moved
> (either by catalog or by background process)
> Snapshot expiration and orphan removal will not cover locations before the
> update.
>
> *Question*
> In v1-3, updating location is a lightweight, metadata-only operation which
> only impacts future writes, and existing absolute paths continue to resolve
> correctly for read. In v4, this is no longer the case. A location update
> becomes a breaking change that requires physical file movement to maintain
> correctness. From what I can tell, a catalog can either validate and handle
> the movement, rewrite paths to absolute, or reject the update to make
> location effectively immutable. Understandably, the iceberg spec does not
> want to prescribe the catalog guidance, but should we acknowledge this
> behavior change and document the lifecycle cleanup implications? Would be
> great if we can disucss further before the spec is finalized.
>
> Thanks,
> Steve Zhang
>
>
>
> On Thu, Jan 29, 2026 at 5:48 PM Talat Uyarer via dev <
> dev@iceberg.apache.org> wrote:
>
>> Hi All,
>>
>> We had a productive meeting today regarding the Relative Paths proposal.
>>
>> We've reached a general agreement on the approach. The changes will
>> involve explicitly defining path terminology (such as "absolute location")
>> and should be well-contained within a new section on Table Spec.
>>
>> The next step is to open a PR with the proposed changes, which may
>> include knock-on effects for the REST specification, such as updates to
>> register table and load table requests.
>>
>> If you'd like to access the meeting notes:
>> https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing
>>
>> You can find the recording here:
>> https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing
>>
>> Thanks for everyone
>>
>> Talat
>>
>> On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon <wy...@cloudera.com.invalid>
>> wrote:
>>
>>> Dan,
>>> Thanks for the clarifications.
>>> Looking forward to the sync.
>>> - Wing Yew
>>>
>>>
>>> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:
>>>
>>>> Hey Wing Yu
>>>>
>>>> I see that you have been updating the Google doc containing the
>>>>> proposal.
>>>>
>>>>
>>>> That's correct, I've been working with Talat to update the doc based on
>>>> feedback from the comments and first round of discussion we had on this
>>>> topic.
>>>>
>>>> Looking through it now, as far as I can tell, the basic idea (from the
>>>>> original proposal) of inferring the table location from the path to the
>>>>> current metadata.json has not changed. Is my reading correct?
>>>>
>>>>
>>>> So far, nothing has changed about table location inference, but we will
>>>> probably be revisiting this with respect to other updates/clarifications.
>>>> There are still a couple open comments related to this point, but it is one
>>>> of the main goals.
>>>>
>>>> You have added clarification around how the path to the metadata is
>>>>> constructed from table location (from which the table location is thus
>>>>> reverse engineered) and around path relativization, but the original idea
>>>>> does not appear to have changed. In that case, the use case of having a
>>>>> single copy of metadata but more than one copy of data (two or more
>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>> the last sync to discuss the proposal.
>>>>
>>>>
>>>> I don't believe this was the sticking point from the original
>>>> discussion.  Having multiple copies/locations of the same data files under
>>>> a single table's management is explicitly a non-goal.  It was discussed in
>>>> the comments of the doc for caching/fallback use cases, but I think that's
>>>> better handled by specific engine/fileio implementations.
>>>>
>>>> The main sticking points were confusion around the complexity of how
>>>> paths are constructed/persisted and the interplay between
>>>> table/metadata/data locations depending on how those values are set in the
>>>> table metadata.  Based on that feedback, we're suggesting some changes,
>>>> which is primarily consist of: 1) defining path construction, resolution,
>>>> and relativization separately, 2) making all paths relative to the table
>>>> location (which simplifies resolution/relativization, 3) address
>>>> confusing/complex issues like path separators and expectations around
>>>> separators.
>>>>
>>>> We're still in the process of updating the document, but we will
>>>> schedule another sync to discuss these updates in detail and address a few
>>>> points that are still outstanding.
>>>>
>>>> Thanks,
>>>> Dan
>>>>
>>>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon
>>>> <wy...@cloudera.com.invalid> wrote:
>>>>
>>>>> Hi Daniel Weeks,
>>>>> I see that you have been updating the Google doc containing the
>>>>> proposal.
>>>>> Looking through it now, as far as I can tell, the basic idea (from the
>>>>> original proposal) of inferring the table location from the path to the
>>>>> current metadata.json has not changed. Is my reading correct?
>>>>> You have added clarification around how the path to the metadata is
>>>>> constructed from table location (from which the table location is thus
>>>>> reverse engineered) and around path relativization, but the original idea
>>>>> does not appear to have changed. In that case, the use case of having a
>>>>> single copy of metadata but more than one copy of data (two or more
>>>>> locations) is not supported by the proposal. This was the sticking point in
>>>>> the last sync to discuss the proposal.
>>>>> Do you intend to have another sync to continue the discussion?
>>>>> Thanks,
>>>>> Wing Yew
>>>>>
>>>>>
>>>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>>>> <am...@apple.com.invalid> wrote:
>>>>>
>>>>>> Thanks Kevin, yes, I see the recording link too but don’t have
>>>>>> access. I have requested access.
>>>>>>
>>>>>>
>>>>>> ~ Anurag Mantripragada
>>>>>>
>>>>>>
>>>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>>>>>>
>>>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>>>> there's already a link for the recording associated with the gcal event but
>>>>>> I dont have access to it.
>>>>>>
>>>>>> Best,
>>>>>> Kevin Liu
>>>>>>
>>>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>>>> <am...@apple.com.invalid> wrote:
>>>>>>
>>>>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>>>>> issues :(
>>>>>>>
>>>>>>>
>>>>>>> ~ Anurag Mantripragada
>>>>>>>
>>>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>>>
>>>>>>> Thanks. I can see it now
>>>>>>>
>>>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> I can see the new event on the dev calendar.
>>>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>>>
>>>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Kevin Liu
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Ally (and everyone else).
>>>>>>>>>
>>>>>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>>>
>>>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>>>
>>>>>>>>> -Dan
>>>>>>>>>
>>>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <
>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Talat
>>>>>>>>>>
>>>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> JB
>>>>>>>>>>
>>>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Hi, Iceberg Community,
>>>>>>>>>> >
>>>>>>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>>>>>>> proposal to add support for relative paths, which has been a long requested
>>>>>>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>>>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>>>>>>> progress on this issue.
>>>>>>>>>> >
>>>>>>>>>> > Please take a look at the linked doc and provide feedback. We'd
>>>>>>>>>> love to open up discussion on this topic at the next community sync and we
>>>>>>>>>> can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>>>> >
>>>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>>>> >
>>>>>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>>>> >
>>>>>>>>>> > Talat
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Steve <ho...@gmail.com>]

Thanks all,

  Following the relative path discussion last week, I want to raise a
question about lifecycle clean up operations in the context of table
location mutability.
The current proposal established that "*the table location is the basis for
all path resolution against persisted relative paths*". Since location
remains mutable, this creates a behavioral difference between v3 and v4
tables that increases operational complexity. Here's a concrete example:.

*Scenario*















*CREATE TABLE prod.db.events (  event_id BIGINT,  event_time TIMESTAMP,
payload STRING) USING icebergLOCATION 's3://bucket-a/warehouse/events';--
Insert some dataINSERT INTO prod.db.events VALUES (1, current_timestamp(),
'data1');INSERT INTO prod.db.events VALUES (2, current_timestamp(),
'data2');-- User changes location (Spark)ALTER TABLE prod.db.events SET
location 's3://bucket-b/warehouse/events';-- Write new dataINSERT INTO
prod.db.events VALUES (3, current_timestamp(), 'data3');*

*Result for v3 table on absolute path *
Manifest entries:
  - s3://bucket-a/warehouse/events/data/file1.parquet  (absolute - old
location)
  - s3://bucket-a/warehouse/events/data/file2.parquet  (absolute - old
location)
  - s3://bucket-b/warehouse/events/data/file3.parquet  (absolute - new
location)
Reads work out of the box as path are absolute
Snapshot expiration will cover both locations before and after the change
as iceberg metadata tracks the path at the time of creation
Orphan removal is limited as it will only respect only the latest location

*Result for v4 table on relative path*
Manifest entries:
  - file1.parquet  (relative - written when location was bucket-a)
  - file2.parquet  (relative - written when location was bucket-a)
  - file3.parquet  (relative - written when location is bucket-b)
Path resolution for file1.parquet:
  Resolved: s3://bucket-b/warehouse/events/data/file1.parquet  ❌
  Actual:   s3://bucket-a/warehouse/events/data/file1.parquet
Reads will fail after location change unless files are physically moved
(either by catalog or by background process)
Snapshot expiration and orphan removal will not cover locations before the
update.

*Question*
In v1-3, updating location is a lightweight, metadata-only operation which
only impacts future writes, and existing absolute paths continue to resolve
correctly for read. In v4, this is no longer the case. A location update
becomes a breaking change that requires physical file movement to maintain
correctness. From what I can tell, a catalog can either validate and handle
the movement, rewrite paths to absolute, or reject the update to make
location effectively immutable. Understandably, the iceberg spec does not
want to prescribe the catalog guidance, but should we acknowledge this
behavior change and document the lifecycle cleanup implications? Would be
great if we can disucss further before the spec is finalized.

Thanks,
Steve Zhang



On Thu, Jan 29, 2026 at 5:48 PM Talat Uyarer via dev <de...@iceberg.apache.org>
wrote:

> Hi All,
>
> We had a productive meeting today regarding the Relative Paths proposal.
>
> We've reached a general agreement on the approach. The changes will
> involve explicitly defining path terminology (such as "absolute location")
> and should be well-contained within a new section on Table Spec.
>
> The next step is to open a PR with the proposed changes, which may include
> knock-on effects for the REST specification, such as updates to register
> table and load table requests.
>
> If you'd like to access the meeting notes:
> https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing
>
> You can find the recording here:
> https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing
>
> Thanks for everyone
>
> Talat
>
> On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon <wy...@cloudera.com.invalid>
> wrote:
>
>> Dan,
>> Thanks for the clarifications.
>> Looking forward to the sync.
>> - Wing Yew
>>
>>
>> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> Hey Wing Yu
>>>
>>> I see that you have been updating the Google doc containing the proposal.
>>>
>>>
>>> That's correct, I've been working with Talat to update the doc based on
>>> feedback from the comments and first round of discussion we had on this
>>> topic.
>>>
>>> Looking through it now, as far as I can tell, the basic idea (from the
>>>> original proposal) of inferring the table location from the path to the
>>>> current metadata.json has not changed. Is my reading correct?
>>>
>>>
>>> So far, nothing has changed about table location inference, but we will
>>> probably be revisiting this with respect to other updates/clarifications.
>>> There are still a couple open comments related to this point, but it is one
>>> of the main goals.
>>>
>>> You have added clarification around how the path to the metadata is
>>>> constructed from table location (from which the table location is thus
>>>> reverse engineered) and around path relativization, but the original idea
>>>> does not appear to have changed. In that case, the use case of having a
>>>> single copy of metadata but more than one copy of data (two or more
>>>> locations) is not supported by the proposal. This was the sticking point in
>>>> the last sync to discuss the proposal.
>>>
>>>
>>> I don't believe this was the sticking point from the original
>>> discussion.  Having multiple copies/locations of the same data files under
>>> a single table's management is explicitly a non-goal.  It was discussed in
>>> the comments of the doc for caching/fallback use cases, but I think that's
>>> better handled by specific engine/fileio implementations.
>>>
>>> The main sticking points were confusion around the complexity of how
>>> paths are constructed/persisted and the interplay between
>>> table/metadata/data locations depending on how those values are set in the
>>> table metadata.  Based on that feedback, we're suggesting some changes,
>>> which is primarily consist of: 1) defining path construction, resolution,
>>> and relativization separately, 2) making all paths relative to the table
>>> location (which simplifies resolution/relativization, 3) address
>>> confusing/complex issues like path separators and expectations around
>>> separators.
>>>
>>> We're still in the process of updating the document, but we will
>>> schedule another sync to discuss these updates in detail and address a few
>>> points that are still outstanding.
>>>
>>> Thanks,
>>> Dan
>>>
>>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon
>>> <wy...@cloudera.com.invalid> wrote:
>>>
>>>> Hi Daniel Weeks,
>>>> I see that you have been updating the Google doc containing the
>>>> proposal.
>>>> Looking through it now, as far as I can tell, the basic idea (from the
>>>> original proposal) of inferring the table location from the path to the
>>>> current metadata.json has not changed. Is my reading correct?
>>>> You have added clarification around how the path to the metadata is
>>>> constructed from table location (from which the table location is thus
>>>> reverse engineered) and around path relativization, but the original idea
>>>> does not appear to have changed. In that case, the use case of having a
>>>> single copy of metadata but more than one copy of data (two or more
>>>> locations) is not supported by the proposal. This was the sticking point in
>>>> the last sync to discuss the proposal.
>>>> Do you intend to have another sync to continue the discussion?
>>>> Thanks,
>>>> Wing Yew
>>>>
>>>>
>>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>>> <am...@apple.com.invalid> wrote:
>>>>
>>>>> Thanks Kevin, yes, I see the recording link too but don’t have access.
>>>>> I have requested access.
>>>>>
>>>>>
>>>>> ~ Anurag Mantripragada
>>>>>
>>>>>
>>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>>>>>
>>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>>> there's already a link for the recording associated with the gcal event but
>>>>> I dont have access to it.
>>>>>
>>>>> Best,
>>>>> Kevin Liu
>>>>>
>>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>>> <am...@apple.com.invalid> wrote:
>>>>>
>>>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>>>> issues :(
>>>>>>
>>>>>>
>>>>>> ~ Anurag Mantripragada
>>>>>>
>>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>>
>>>>>> Thanks. I can see it now
>>>>>>
>>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> I can see the new event on the dev calendar.
>>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>>
>>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>>
>>>>>>> Best,
>>>>>>> Kevin Liu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey Ally (and everyone else).
>>>>>>>>
>>>>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>>
>>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>>
>>>>>>>> -Dan
>>>>>>>>
>>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <
>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>
>>>>>>>>> Hi Talat
>>>>>>>>>
>>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi, Iceberg Community,
>>>>>>>>> >
>>>>>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>>>>>> proposal to add support for relative paths, which has been a long requested
>>>>>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>>>>>> progress on this issue.
>>>>>>>>> >
>>>>>>>>> > Please take a look at the linked doc and provide feedback. We'd
>>>>>>>>> love to open up discussion on this topic at the next community sync and we
>>>>>>>>> can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>>> >
>>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>>> >
>>>>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>>> >
>>>>>>>>> > Talat
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Talat Uyarer via dev <de...@iceberg.apache.org>]

Hi All,

We had a productive meeting today regarding the Relative Paths proposal.

We've reached a general agreement on the approach. The changes will involve
explicitly defining path terminology (such as "absolute location") and
should be well-contained within a new section on Table Spec.

The next step is to open a PR with the proposed changes, which may include
knock-on effects for the REST specification, such as updates to register
table and load table requests.

If you'd like to access the meeting notes:
https://docs.google.com/document/d/1t0RxrK-nsCT83zXeD66kmGx_TMU2X8_xfN1A_k6dCV0/edit?usp=sharing

You can find the recording here:
https://drive.google.com/file/d/11q65achM_3vCfaEVYsxmfAdbKQJb2drA/view?usp=sharing

Thanks for everyone

Talat

On Fri, Aug 1, 2025 at 10:50 AM Wing Yew Poon <wy...@cloudera.com.invalid>
wrote:

> Dan,
> Thanks for the clarifications.
> Looking forward to the sync.
> - Wing Yew
>
>
> On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:
>
>> Hey Wing Yu
>>
>> I see that you have been updating the Google doc containing the proposal.
>>
>>
>> That's correct, I've been working with Talat to update the doc based on
>> feedback from the comments and first round of discussion we had on this
>> topic.
>>
>> Looking through it now, as far as I can tell, the basic idea (from the
>>> original proposal) of inferring the table location from the path to the
>>> current metadata.json has not changed. Is my reading correct?
>>
>>
>> So far, nothing has changed about table location inference, but we will
>> probably be revisiting this with respect to other updates/clarifications.
>> There are still a couple open comments related to this point, but it is one
>> of the main goals.
>>
>> You have added clarification around how the path to the metadata is
>>> constructed from table location (from which the table location is thus
>>> reverse engineered) and around path relativization, but the original idea
>>> does not appear to have changed. In that case, the use case of having a
>>> single copy of metadata but more than one copy of data (two or more
>>> locations) is not supported by the proposal. This was the sticking point in
>>> the last sync to discuss the proposal.
>>
>>
>> I don't believe this was the sticking point from the original
>> discussion.  Having multiple copies/locations of the same data files under
>> a single table's management is explicitly a non-goal.  It was discussed in
>> the comments of the doc for caching/fallback use cases, but I think that's
>> better handled by specific engine/fileio implementations.
>>
>> The main sticking points were confusion around the complexity of how
>> paths are constructed/persisted and the interplay between
>> table/metadata/data locations depending on how those values are set in the
>> table metadata.  Based on that feedback, we're suggesting some changes,
>> which is primarily consist of: 1) defining path construction, resolution,
>> and relativization separately, 2) making all paths relative to the table
>> location (which simplifies resolution/relativization, 3) address
>> confusing/complex issues like path separators and expectations around
>> separators.
>>
>> We're still in the process of updating the document, but we will schedule
>> another sync to discuss these updates in detail and address a few points
>> that are still outstanding.
>>
>> Thanks,
>> Dan
>>
>> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon <wy...@cloudera.com.invalid>
>> wrote:
>>
>>> Hi Daniel Weeks,
>>> I see that you have been updating the Google doc containing the proposal.
>>> Looking through it now, as far as I can tell, the basic idea (from the
>>> original proposal) of inferring the table location from the path to the
>>> current metadata.json has not changed. Is my reading correct?
>>> You have added clarification around how the path to the metadata is
>>> constructed from table location (from which the table location is thus
>>> reverse engineered) and around path relativization, but the original idea
>>> does not appear to have changed. In that case, the use case of having a
>>> single copy of metadata but more than one copy of data (two or more
>>> locations) is not supported by the proposal. This was the sticking point in
>>> the last sync to discuss the proposal.
>>> Do you intend to have another sync to continue the discussion?
>>> Thanks,
>>> Wing Yew
>>>
>>>
>>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>>> <am...@apple.com.invalid> wrote:
>>>
>>>> Thanks Kevin, yes, I see the recording link too but don’t have access.
>>>> I have requested access.
>>>>
>>>>
>>>> ~ Anurag Mantripragada
>>>>
>>>>
>>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>>>>
>>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>>> there's already a link for the recording associated with the gcal event but
>>>> I dont have access to it.
>>>>
>>>> Best,
>>>> Kevin Liu
>>>>
>>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>>> <am...@apple.com.invalid> wrote:
>>>>
>>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>>> issues :(
>>>>>
>>>>>
>>>>> ~ Anurag Mantripragada
>>>>>
>>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>>
>>>>> Thanks. I can see it now
>>>>>
>>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> I can see the new event on the dev calendar.
>>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>>
>>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>>
>>>>>> Best,
>>>>>> Kevin Liu
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Ally (and everyone else).
>>>>>>>
>>>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>>
>>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>>
>>>>>>> -Dan
>>>>>>>
>>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Talat
>>>>>>>>
>>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>>> >
>>>>>>>> > Hi, Iceberg Community,
>>>>>>>> >
>>>>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>>>>> proposal to add support for relative paths, which has been a long requested
>>>>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>>>>> progress on this issue.
>>>>>>>> >
>>>>>>>> > Please take a look at the linked doc and provide feedback. We'd
>>>>>>>> love to open up discussion on this topic at the next community sync and we
>>>>>>>> can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>>> >
>>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>>> >
>>>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>>> >
>>>>>>>> > Talat
>>>>>>>>
>>>>>>>
>>>>>
>>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Wing Yew Poon <wy...@cloudera.com.INVALID>]

Dan,
Thanks for the clarifications.
Looking forward to the sync.
- Wing Yew


On Fri, Aug 1, 2025 at 8:43 AM Daniel Weeks <dw...@apache.org> wrote:

> Hey Wing Yu
>
> I see that you have been updating the Google doc containing the proposal.
>
>
> That's correct, I've been working with Talat to update the doc based on
> feedback from the comments and first round of discussion we had on this
> topic.
>
> Looking through it now, as far as I can tell, the basic idea (from the
>> original proposal) of inferring the table location from the path to the
>> current metadata.json has not changed. Is my reading correct?
>
>
> So far, nothing has changed about table location inference, but we will
> probably be revisiting this with respect to other updates/clarifications.
> There are still a couple open comments related to this point, but it is one
> of the main goals.
>
> You have added clarification around how the path to the metadata is
>> constructed from table location (from which the table location is thus
>> reverse engineered) and around path relativization, but the original idea
>> does not appear to have changed. In that case, the use case of having a
>> single copy of metadata but more than one copy of data (two or more
>> locations) is not supported by the proposal. This was the sticking point in
>> the last sync to discuss the proposal.
>
>
> I don't believe this was the sticking point from the original discussion.
> Having multiple copies/locations of the same data files under a single
> table's management is explicitly a non-goal.  It was discussed in the
> comments of the doc for caching/fallback use cases, but I think that's
> better handled by specific engine/fileio implementations.
>
> The main sticking points were confusion around the complexity of how paths
> are constructed/persisted and the interplay between table/metadata/data
> locations depending on how those values are set in the table metadata.
> Based on that feedback, we're suggesting some changes, which is primarily
> consist of: 1) defining path construction, resolution, and relativization
> separately, 2) making all paths relative to the table location (which
> simplifies resolution/relativization, 3) address confusing/complex issues
> like path separators and expectations around separators.
>
> We're still in the process of updating the document, but we will schedule
> another sync to discuss these updates in detail and address a few points
> that are still outstanding.
>
> Thanks,
> Dan
>
> On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon <wy...@cloudera.com.invalid>
> wrote:
>
>> Hi Daniel Weeks,
>> I see that you have been updating the Google doc containing the proposal.
>> Looking through it now, as far as I can tell, the basic idea (from the
>> original proposal) of inferring the table location from the path to the
>> current metadata.json has not changed. Is my reading correct?
>> You have added clarification around how the path to the metadata is
>> constructed from table location (from which the table location is thus
>> reverse engineered) and around path relativization, but the original idea
>> does not appear to have changed. In that case, the use case of having a
>> single copy of metadata but more than one copy of data (two or more
>> locations) is not supported by the proposal. This was the sticking point in
>> the last sync to discuss the proposal.
>> Do you intend to have another sync to continue the discussion?
>> Thanks,
>> Wing Yew
>>
>>
>> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
>> <am...@apple.com.invalid> wrote:
>>
>>> Thanks Kevin, yes, I see the recording link too but don’t have access. I
>>> have requested access.
>>>
>>>
>>> ~ Anurag Mantripragada
>>>
>>>
>>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>>>
>>> Yes it was recorded. Dan or Talat should have the recording. I see
>>> there's already a link for the recording associated with the gcal event but
>>> I dont have access to it.
>>>
>>> Best,
>>> Kevin Liu
>>>
>>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>>> <am...@apple.com.invalid> wrote:
>>>
>>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>>> issues :(
>>>>
>>>>
>>>> ~ Anurag Mantripragada
>>>>
>>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>>
>>>> Thanks. I can see it now
>>>>
>>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org>
>>>> wrote:
>>>>
>>>>>
>>>>> I can see the new event on the dev calendar.
>>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>>
>>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>>
>>>>> Best,
>>>>> Kevin Liu
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hey Ally (and everyone else).
>>>>>>
>>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>>
>>>>>> Let me know if you still don't see it on the calendar.
>>>>>>
>>>>>> -Dan
>>>>>>
>>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Talat
>>>>>>>
>>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>>> <ta...@google.com.invalid> wrote:
>>>>>>> >
>>>>>>> > Hi, Iceberg Community,
>>>>>>> >
>>>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>>>> proposal to add support for relative paths, which has been a long requested
>>>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>>>> progress on this issue.
>>>>>>> >
>>>>>>> > Please take a look at the linked doc and provide feedback. We'd
>>>>>>> love to open up discussion on this topic at the next community sync and we
>>>>>>> can hold one-off syncs on the topic if there's a lot of interest.
>>>>>>> >
>>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>>> >
>>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>>> >
>>>>>>> > Talat
>>>>>>>
>>>>>>
>>>>
>>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Hey Wing Yu

I see that you have been updating the Google doc containing the proposal.


That's correct, I've been working with Talat to update the doc based on
feedback from the comments and first round of discussion we had on this
topic.

Looking through it now, as far as I can tell, the basic idea (from the
> original proposal) of inferring the table location from the path to the
> current metadata.json has not changed. Is my reading correct?


So far, nothing has changed about table location inference, but we will
probably be revisiting this with respect to other updates/clarifications.
There are still a couple open comments related to this point, but it is one
of the main goals.

You have added clarification around how the path to the metadata is
> constructed from table location (from which the table location is thus
> reverse engineered) and around path relativization, but the original idea
> does not appear to have changed. In that case, the use case of having a
> single copy of metadata but more than one copy of data (two or more
> locations) is not supported by the proposal. This was the sticking point in
> the last sync to discuss the proposal.


I don't believe this was the sticking point from the original discussion.
Having multiple copies/locations of the same data files under a single
table's management is explicitly a non-goal.  It was discussed in the
comments of the doc for caching/fallback use cases, but I think that's
better handled by specific engine/fileio implementations.

The main sticking points were confusion around the complexity of how paths
are constructed/persisted and the interplay between table/metadata/data
locations depending on how those values are set in the table metadata.
Based on that feedback, we're suggesting some changes, which is primarily
consist of: 1) defining path construction, resolution, and relativization
separately, 2) making all paths relative to the table location (which
simplifies resolution/relativization, 3) address confusing/complex issues
like path separators and expectations around separators.

We're still in the process of updating the document, but we will schedule
another sync to discuss these updates in detail and address a few points
that are still outstanding.

Thanks,
Dan

On Thu, Jul 31, 2025 at 5:47 PM Wing Yew Poon <wy...@cloudera.com.invalid>
wrote:

> Hi Daniel Weeks,
> I see that you have been updating the Google doc containing the proposal.
> Looking through it now, as far as I can tell, the basic idea (from the
> original proposal) of inferring the table location from the path to the
> current metadata.json has not changed. Is my reading correct?
> You have added clarification around how the path to the metadata is
> constructed from table location (from which the table location is thus
> reverse engineered) and around path relativization, but the original idea
> does not appear to have changed. In that case, the use case of having a
> single copy of metadata but more than one copy of data (two or more
> locations) is not supported by the proposal. This was the sticking point in
> the last sync to discuss the proposal.
> Do you intend to have another sync to continue the discussion?
> Thanks,
> Wing Yew
>
>
> On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
> <am...@apple.com.invalid> wrote:
>
>> Thanks Kevin, yes, I see the recording link too but don’t have access. I
>> have requested access.
>>
>>
>> ~ Anurag Mantripragada
>>
>>
>> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>>
>> Yes it was recorded. Dan or Talat should have the recording. I see
>> there's already a link for the recording associated with the gcal event but
>> I dont have access to it.
>>
>> Best,
>> Kevin Liu
>>
>> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
>> <am...@apple.com.invalid> wrote:
>>
>>> Hey folks, was the sync recorded? I missed it due to calendar sync
>>> issues :(
>>>
>>>
>>> ~ Anurag Mantripragada
>>>
>>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>>
>>> Thanks. I can see it now
>>>
>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org> wrote:
>>>
>>>>
>>>> I can see the new event on the dev calendar.
>>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>>
>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>>
>>>> Best,
>>>> Kevin Liu
>>>>
>>>>
>>>>
>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org> wrote:
>>>>
>>>>> Hey Ally (and everyone else).
>>>>>
>>>>> We hadn't scheduled the discussion for relative paths, but I just
>>>>> added an event to the dev calendar for Thursday at 9am (PT).
>>>>>
>>>>> Let me know if you still don't see it on the calendar.
>>>>>
>>>>> -Dan
>>>>>
>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>>> Hi Talat
>>>>>>
>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>>> <ta...@google.com.invalid> wrote:
>>>>>> >
>>>>>> > Hi, Iceberg Community,
>>>>>> >
>>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>>> proposal to add support for relative paths, which has been a long requested
>>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>>> progress on this issue.
>>>>>> >
>>>>>> > Please take a look at the linked doc and provide feedback. We'd
>>>>>> love to open up discussion on this topic at the next community sync and we
>>>>>> can hold one-off syncs on the topic if there's a lot of interest.
>>>>>> >
>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>> >
>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>> >
>>>>>> > Talat
>>>>>>
>>>>>
>>>
>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Wing Yew Poon <wy...@cloudera.com.INVALID>]

Hi Daniel Weeks,
I see that you have been updating the Google doc containing the proposal.
Looking through it now, as far as I can tell, the basic idea (from the
original proposal) of inferring the table location from the path to the
current metadata.json has not changed. Is my reading correct?
You have added clarification around how the path to the metadata is
constructed from table location (from which the table location is thus
reverse engineered) and around path relativization, but the original idea
does not appear to have changed. In that case, the use case of having a
single copy of metadata but more than one copy of data (two or more
locations) is not supported by the proposal. This was the sticking point in
the last sync to discuss the proposal.
Do you intend to have another sync to continue the discussion?
Thanks,
Wing Yew


On Thu, Jul 10, 2025 at 4:41 PM Anurag Mantripragada
<am...@apple.com.invalid> wrote:

> Thanks Kevin, yes, I see the recording link too but don’t have access. I
> have requested access.
>
>
> ~ Anurag Mantripragada
>
>
> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
>
> Yes it was recorded. Dan or Talat should have the recording. I see there's
> already a link for the recording associated with the gcal event but I dont
> have access to it.
>
> Best,
> Kevin Liu
>
> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
> <am...@apple.com.invalid> wrote:
>
>> Hey folks, was the sync recorded? I missed it due to calendar sync issues
>> :(
>>
>>
>> ~ Anurag Mantripragada
>>
>> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>>
>> Thanks. I can see it now
>>
>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org> wrote:
>>
>>>
>>> I can see the new event on the dev calendar.
>>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>>
>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>
>>> Best,
>>> Kevin Liu
>>>
>>>
>>>
>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org> wrote:
>>>
>>>> Hey Ally (and everyone else).
>>>>
>>>> We hadn't scheduled the discussion for relative paths, but I just added
>>>> an event to the dev calendar for Thursday at 9am (PT).
>>>>
>>>> Let me know if you still don't see it on the calendar.
>>>>
>>>> -Dan
>>>>
>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi Talat
>>>>>
>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer
>>>>> <ta...@google.com.invalid> wrote:
>>>>> >
>>>>> > Hi, Iceberg Community,
>>>>> >
>>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>>> proposal to add support for relative paths, which has been a long requested
>>>>> feature. There have been a number of discussions/proposals over the years,
>>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>>> progress on this issue.
>>>>> >
>>>>> > Please take a look at the linked doc and provide feedback. We'd love
>>>>> to open up discussion on this topic at the next community sync and we can
>>>>> hold one-off syncs on the topic if there's a lot of interest.
>>>>> >
>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>> >
>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>> >
>>>>> > Talat
>>>>>
>>>>
>>
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Anurag Mantripragada <am...@apple.com.INVALID>]

Thanks Kevin, yes, I see the recording link too but don’t have access. I have requested access. 


~ Anurag Mantripragada


> On Jul 10, 2025, at 2:43 PM, Kevin Liu <ke...@apache.org> wrote:
> 
> Yes it was recorded. Dan or Talat should have the recording. I see there's already a link for the recording associated with the gcal event but I dont have access to it. 
> 
> Best,
> Kevin Liu 
> 
> On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada <am...@apple.com.invalid> wrote:
>> Hey folks, was the sync recorded? I missed it due to calendar sync issues :( 
>> 
>> 
>> ~ Anurag Mantripragada
>> 
>>> On Jul 7, 2025, at 6:27 PM, ally heev <allyheev@gmail.com <ma...@gmail.com>> wrote:
>>> 
>>> Thanks. I can see it now
>>> 
>>> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <kevinjqliu@apache.org <ma...@apache.org>> wrote:
>>>> 
>>>> I can see the new event on the dev calendar. 
>>>> 
>>>> 
>>>> Subscribe to the "Iceberg Dev Events" calendar here:
>>>> https://iceberg.apache.org/community/#iceberg-community-events
>>>> 
>>>> Best,
>>>> Kevin Liu
>>>> 
>>>> 
>>>> 
>>>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dweeks@apache.org <ma...@apache.org>> wrote:
>>>>> Hey Ally (and everyone else).
>>>>> 
>>>>> We hadn't scheduled the discussion for relative paths, but I just added an event to the dev calendar for Thursday at 9am (PT).
>>>>> 
>>>>> Let me know if you still don't see it on the calendar.
>>>>> 
>>>>> -Dan
>>>>> 
>>>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>>>>>> Hi Talat
>>>>>> 
>>>>>> Thanks for the update. I will do a new pass on the doc.
>>>>>> 
>>>>>> Regards
>>>>>> JB
>>>>>> 
>>>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid> wrote:
>>>>>> >
>>>>>> > Hi, Iceberg Community,
>>>>>> >
>>>>>> > As mentioned at the last sync, Dan and I have been working on a proposal to add support for relative paths, which has been a long requested feature. There have been a number of discussions/proposals over the years, but we'd like to scope down and refocus effort to make some meaningful progress on this issue.
>>>>>> >
>>>>>> > Please take a look at the linked doc and provide feedback. We'd love to open up discussion on this topic at the next community sync and we can hold one-off syncs on the topic if there's a lot of interest.
>>>>>> >
>>>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>>>> >
>>>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>>>> >
>>>>>> > Talat
>> 

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Kevin Liu <ke...@apache.org>]

Yes it was recorded. Dan or Talat should have the recording. I see there's
already a link for the recording associated with the gcal event but I dont
have access to it.

Best,
Kevin Liu

On Thu, Jul 10, 2025 at 12:37 PM Anurag Mantripragada
<am...@apple.com.invalid> wrote:

> Hey folks, was the sync recorded? I missed it due to calendar sync issues
> :(
>
>
> ~ Anurag Mantripragada
>
> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
>
> Thanks. I can see it now
>
> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org> wrote:
>
>>
>> I can see the new event on the dev calendar.
>> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>>
>> Subscribe to the "Iceberg Dev Events" calendar here:
>> https://iceberg.apache.org/community/#iceberg-community-events
>>
>> Best,
>> Kevin Liu
>>
>>
>>
>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> Hey Ally (and everyone else).
>>>
>>> We hadn't scheduled the discussion for relative paths, but I just added
>>> an event to the dev calendar for Thursday at 9am (PT).
>>>
>>> Let me know if you still don't see it on the calendar.
>>>
>>> -Dan
>>>
>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi Talat
>>>>
>>>> Thanks for the update. I will do a new pass on the doc.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid>
>>>> wrote:
>>>> >
>>>> > Hi, Iceberg Community,
>>>> >
>>>> > As mentioned at the last sync, Dan and I have been working on a
>>>> proposal to add support for relative paths, which has been a long requested
>>>> feature. There have been a number of discussions/proposals over the years,
>>>> but we'd like to scope down and refocus effort to make some meaningful
>>>> progress on this issue.
>>>> >
>>>> > Please take a look at the linked doc and provide feedback. We'd love
>>>> to open up discussion on this topic at the next community sync and we can
>>>> hold one-off syncs on the topic if there's a lot of interest.
>>>> >
>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>> >
>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>> >
>>>> > Talat
>>>>
>>>
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Anurag Mantripragada <am...@apple.com.INVALID>]

Hey folks, was the sync recorded? I missed it due to calendar sync issues :( 


~ Anurag Mantripragada

> On Jul 7, 2025, at 6:27 PM, ally heev <al...@gmail.com> wrote:
> 
> Thanks. I can see it now
> 
> On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <kevinjqliu@apache.org <ma...@apache.org>> wrote:
>> 
>> I can see the new event on the dev calendar. 
>> 
>> 
>> Subscribe to the "Iceberg Dev Events" calendar here:
>> https://iceberg.apache.org/community/#iceberg-community-events
>> 
>> Best,
>> Kevin Liu
>> 
>> 
>> 
>> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dweeks@apache.org <ma...@apache.org>> wrote:
>>> Hey Ally (and everyone else).
>>> 
>>> We hadn't scheduled the discussion for relative paths, but I just added an event to the dev calendar for Thursday at 9am (PT).
>>> 
>>> Let me know if you still don't see it on the calendar.
>>> 
>>> -Dan
>>> 
>>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>>>> Hi Talat
>>>> 
>>>> Thanks for the update. I will do a new pass on the doc.
>>>> 
>>>> Regards
>>>> JB
>>>> 
>>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid> wrote:
>>>> >
>>>> > Hi, Iceberg Community,
>>>> >
>>>> > As mentioned at the last sync, Dan and I have been working on a proposal to add support for relative paths, which has been a long requested feature. There have been a number of discussions/proposals over the years, but we'd like to scope down and refocus effort to make some meaningful progress on this issue.
>>>> >
>>>> > Please take a look at the linked doc and provide feedback. We'd love to open up discussion on this topic at the next community sync and we can hold one-off syncs on the topic if there's a lot of interest.
>>>> >
>>>> > You can access Iceberg's First V4 Spec change from here :)
>>>> >
>>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>>> >
>>>> > Talat

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[ally heev <al...@gmail.com>]

Thanks. I can see it now

On Tue, Jul 8, 2025 at 12:37 AM Kevin Liu <ke...@apache.org> wrote:

>
> I can see the new event on the dev calendar.
> [image: Screenshot 2025-07-07 at 12.04.08 PM.png]
>
> Subscribe to the "Iceberg Dev Events" calendar here:
> https://iceberg.apache.org/community/#iceberg-community-events
>
> Best,
> Kevin Liu
>
>
>
> On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org> wrote:
>
>> Hey Ally (and everyone else).
>>
>> We hadn't scheduled the discussion for relative paths, but I just added
>> an event to the dev calendar for Thursday at 9am (PT).
>>
>> Let me know if you still don't see it on the calendar.
>>
>> -Dan
>>
>> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi Talat
>>>
>>> Thanks for the update. I will do a new pass on the doc.
>>>
>>> Regards
>>> JB
>>>
>>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid>
>>> wrote:
>>> >
>>> > Hi, Iceberg Community,
>>> >
>>> > As mentioned at the last sync, Dan and I have been working on a
>>> proposal to add support for relative paths, which has been a long requested
>>> feature. There have been a number of discussions/proposals over the years,
>>> but we'd like to scope down and refocus effort to make some meaningful
>>> progress on this issue.
>>> >
>>> > Please take a look at the linked doc and provide feedback. We'd love
>>> to open up discussion on this topic at the next community sync and we can
>>> hold one-off syncs on the topic if there's a lot of interest.
>>> >
>>> > You can access Iceberg's First V4 Spec change from here :)
>>> >
>>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>>> >
>>> > Talat
>>>
>>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Kevin Liu <ke...@apache.org>]

I can see the new event on the dev calendar.
[image: Screenshot 2025-07-07 at 12.04.08 PM.png]

Subscribe to the "Iceberg Dev Events" calendar here:
https://iceberg.apache.org/community/#iceberg-community-events

Best,
Kevin Liu



On Mon, Jul 7, 2025 at 11:38 AM Daniel Weeks <dw...@apache.org> wrote:

> Hey Ally (and everyone else).
>
> We hadn't scheduled the discussion for relative paths, but I just added an
> event to the dev calendar for Thursday at 9am (PT).
>
> Let me know if you still don't see it on the calendar.
>
> -Dan
>
> On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Talat
>>
>> Thanks for the update. I will do a new pass on the doc.
>>
>> Regards
>> JB
>>
>> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid>
>> wrote:
>> >
>> > Hi, Iceberg Community,
>> >
>> > As mentioned at the last sync, Dan and I have been working on a
>> proposal to add support for relative paths, which has been a long requested
>> feature. There have been a number of discussions/proposals over the years,
>> but we'd like to scope down and refocus effort to make some meaningful
>> progress on this issue.
>> >
>> > Please take a look at the linked doc and provide feedback. We'd love to
>> open up discussion on this topic at the next community sync and we can hold
>> one-off syncs on the topic if there's a lot of interest.
>> >
>> > You can access Iceberg's First V4 Spec change from here :)
>> >
>> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
>> > Doc: https://s.apache.org/iceberg-spec-relative-path
>> >
>> > Talat
>>
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Daniel Weeks <dw...@apache.org>]

Hey Ally (and everyone else).

We hadn't scheduled the discussion for relative paths, but I just added an
event to the dev calendar for Thursday at 9am (PT).

Let me know if you still don't see it on the calendar.

-Dan

On Sat, Jul 5, 2025 at 9:37 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:

> Hi Talat
>
> Thanks for the update. I will do a new pass on the doc.
>
> Regards
> JB
>
> On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid>
> wrote:
> >
> > Hi, Iceberg Community,
> >
> > As mentioned at the last sync, Dan and I have been working on a proposal
> to add support for relative paths, which has been a long requested feature.
> There have been a number of discussions/proposals over the years, but we'd
> like to scope down and refocus effort to make some meaningful progress on
> this issue.
> >
> > Please take a look at the linked doc and provide feedback. We'd love to
> open up discussion on this topic at the next community sync and we can hold
> one-off syncs on the topic if there's a lot of interest.
> >
> > You can access Iceberg's First V4 Spec change from here :)
> >
> > Proposal Issue: https://github.com/apache/iceberg/issues/13141
> > Doc: https://s.apache.org/iceberg-spec-relative-path
> >
> > Talat
>

Re: [DISCUSS] Proposal for Relative Path Support In Table Spec

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Hi Talat

Thanks for the update. I will do a new pass on the doc.

Regards
JB

On Wed, May 28, 2025 at 12:13 AM Talat Uyarer <ta...@google.com.invalid> wrote:
>
> Hi, Iceberg Community,
>
> As mentioned at the last sync, Dan and I have been working on a proposal to add support for relative paths, which has been a long requested feature. There have been a number of discussions/proposals over the years, but we'd like to scope down and refocus effort to make some meaningful progress on this issue.
>
> Please take a look at the linked doc and provide feedback. We'd love to open up discussion on this topic at the next community sync and we can hold one-off syncs on the topic if there's a lot of interest.
>
> You can access Iceberg's First V4 Spec change from here :)
>
> Proposal Issue: https://github.com/apache/iceberg/issues/13141
> Doc: https://s.apache.org/iceberg-spec-relative-path
>
> Talat