You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "elphastori (via GitHub)" <gi...@apache.org> on 2025/05/20 12:30:21 UTC

[PR] Docs: add column descriptions for entries metadata table #13076 [iceberg]

elphastori opened a new pull request, #13104:
URL: https://github.com/apache/iceberg/pull/13104

   Add column descriptions for entries metadata table at https://iceberg.apache.org/docs/nightly/spark-queries/#entries. Closes #13076


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table #13076 [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#issuecomment-2894830747

   @elphastori Can you a snapshot of the page built locally?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109211954


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   My bad, I meant to use this link: https://iceberg.apache.org/spec/#manifests. (The one I used has to be translated by the build)
   
   Btw, there's no deep link, so the link just goes to the main Manifests section in the spec. I think it's worth rearranging that section to have proper subheadings we can link to it like we have for [TableMetadata](https://iceberg.apache.org/spec/#table-metadata).
   
   I can try that if you think it's a good idea.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2121156956


##########
docs/docs/spark-queries.md:
##########
@@ -299,6 +299,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: A struct with metadata, the fields are also defined in the [manifest file schema](../../spec.md#manifests)

Review Comment:
   ```suggestion
       - `data_file`: A struct with metadata about the data file. The fields of the struct are defined in the [data_file schema](../../spec.md#manifests)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2111095523


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   The issue is we don't have anchors for `manifest_entry` and `data_file` to link to.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2103074320


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   Sure, since `data_file` has 24 fields defined in https://iceberg.apache.org/spec/#manifests , can I improve the inline description and provide a link to the schema? It would look like this:
   
   > - `data_file`: A struct with metadata, the fields are also defined in the [manifest file schema](../../spec.md#manifests) 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2110731586


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   @nastra @Fokko What do you think?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2110970830


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   > Sure, since `data_file` has 24 fields defined in https://iceberg.apache.org/spec/#manifests , can I improve the inline description and provide a link to the schema? It would look like this:
   > 
   > > * `data_file`: A struct with metadata, the fields are also defined in the [manifest file schema](../../spec.md#manifests)
   
   This makes sense to me, +1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2121343055


##########
docs/docs/spark-queries.md:
##########
@@ -299,6 +299,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: A struct with metadata, the fields are also defined in the [manifest file schema](../../spec.md#manifests)

Review Comment:
   Thanks! I've applied the suggestion, it reads much better:
   
   <img width="740" alt="data_file" src="https://github.com/user-attachments/assets/2b6e4ea1-a103-4ba2-b88e-c60642b0514f" />
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra merged PR #13104:
URL: https://github.com/apache/iceberg/pull/13104


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2102813536


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   Can you also add description of this struct?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109211954


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   My bad, I meant to use this link: https://iceberg.apache.org/spec/#manifests. (The one I used needs to be processed by the build system.)
   
   By the way, there’s no deep link available, so the URL just points to the main Manifests section in the spec. I think it might be worth reorganizing that section with proper subheadings that we can link to, similar to how it’s done for [TableMetadata](https://iceberg.apache.org/spec/#table-metadata).
   
   I’m happy to take a stab at that if you think it’s a good idea.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2120177480


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   I don't think that's too much of an issue. We can link to the manifest right now and update the link later when we have proper anchors



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table #13076 [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#issuecomment-2895003759

   @manuzhang Thanks, here's the screenshot.
   
   <img width="798" alt="entries-screenshot" src="https://github.com/user-attachments/assets/8f802b95-8cf1-4dd9-b9db-0be8cd9d0e0f" />
   
   The "manifest file schema" URL also works as expected!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2121067022


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   I've updated the description for `data_file` and added the link:
   
   <img width="734" alt="data_file" src="https://github.com/user-attachments/assets/37316c95-f0e0-4480-9e6d-45045bedfb44" />
   
   I will follow-up with a separate PR to add anchors after this one has been merged.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "elphastori (via GitHub)" <gi...@apache.org>.
elphastori closed pull request #13104: Docs: add column descriptions for entries metadata table
URL: https://github.com/apache/iceberg/pull/13104


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #13104:
URL: https://github.com/apache/iceberg/pull/13104#discussion_r2109157835


##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   Yeah, but I can't find a link to the data_file definition.



##########
docs/docs/spark-queries.md:
##########
@@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries;
 | -- | -- | -- | -- | -- | -- |
 | 2 | 57897183625154 | 0 | 0 | {"content":0,"file_path":"s3:/.../table/data/00047-25-833044d0-127b-415c-b874-038a4f978c29-00612.parquet","file_format":"PARQUET","spec_id":0,"record_count":15,"file_size_in_bytes":473,"column_sizes":{1:103},"value_counts":{1:15},"null_value_counts":{1:0},"nan_value_counts":{},"lower_bounds":{1:},"upper_bounds":{1:},"key_metadata":null,"split_offsets":[4],"equality_ids":null,"sort_order_id":0} | {"c1":{"column_size":103,"value_count":15,"null_value_count":0,"nan_value_count":null,"lower_bound":1,"upper_bound":3}} |
 
+Note:
+
+1. The columns of the `entries` table correspond to the fields of the `manifest_entry` struct (see the [manifest file schema](../../spec.md#manifests) for the full definition):
+    - `status`: Used to track additions and deletions
+    - `snapshot_id`: The ID of the snapshot in which the file was added or removed
+    - `sequence_number`: Used for ordering changes across snapshots
+    - `file_sequence_number`: Indicates when the file was added
+    - `data_file`: Metadata about the data file

Review Comment:
   Yeah, but I can't find a link to the `data_file` definition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org