You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2024/07/04 18:49:22 UTC
[I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Fokko opened a new issue, #893:
URL: https://github.com/apache/iceberg-python/issues/893
### Feature Request / Improvement
It looks like a misnamed field slipped in:
```json
{
"status": 1,
"snapshot_id": {
"long": 898025966831056900
},
"data_sequence_number": null,
"file_sequence_number": null,
"data_file": {
"content": 0,
"file_path": "/tmp/some.db/tablev2/data/00000-0-93717a88-1cea-4e3d-a69a-00ce3d087822.parquet",
"file_format": "PARQUET",
"partition": {},
"record_count": 3,
"file_size_in_bytes": 5459,
"column_sizes": { ... },
"value_counts": { ... },
"null_value_counts": { ... },
"nan_value_counts": { ... },
"lower_bounds": { ... },
"upper_bounds": { ... },
"key_metadata": null,
"split_offsets": {
"array": [
4
]
},
"equality_ids": null,
"sort_order_id": null
}
}
```
This should be `sequence_number`:
Luckily this still worked due to Iceberg's field-id based lookup, but would be good to get this cleaned up.
Relevant code:
https://github.com/apache/iceberg-python/blob/a8d3f17d42b00b507a3522714fe431a18124493e/pyiceberg/manifest.py#L380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on issue #893:
URL: https://github.com/apache/iceberg-python/issues/893#issuecomment-2211163215
@Fokko I would like to take a shot at this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on issue #893:
URL: https://github.com/apache/iceberg-python/issues/893#issuecomment-2211986191
@Fokko the PR https://github.com/apache/iceberg-python/pull/900 is ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #893:
URL: https://github.com/apache/iceberg-python/issues/893#issuecomment-2211165635
@soumya-ghosh Feel free to take a stab at it, let me know if you run into anything
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX closed issue #893: Rename `data_sequence_number` to `sequence_number`
URL: https://github.com/apache/iceberg-python/issues/893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #893:
URL: https://github.com/apache/iceberg-python/issues/893#issuecomment-2211166513
> Is there a way on the Java/spark side to turn metadata information into JSON? With https://github.com/apache/iceberg-python/issues/535, perhaps we can compare the two JSON results and check for mismatches like this one.
That would be an interesting idea. We could take the PySpark schema and turn it into an Iceberg schema and compare the two (or just compare the Avro schemas)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Rename `data_sequence_number` to `sequence_number` [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #893:
URL: https://github.com/apache/iceberg-python/issues/893#issuecomment-2209610949
Is there a way on the Java/spark side to turn metadata information into JSON? With #535, perhaps we can compare the two JSON results and check for mismatches like this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org