You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "soumya-ghosh (via GitHub)" <gi...@apache.org> on 2024/07/06 22:02:33 UTC

[PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

soumya-ghosh opened a new pull request, #900:
URL: https://github.com/apache/iceberg-python/pull/900

   Implements: https://github.com/apache/iceberg-python/issues/893


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1669065871


##########
pyiceberg/manifest.py:
##########
@@ -762,10 +762,10 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter:
 
         if (
             (entry.status == ManifestEntryStatus.ADDED or entry.status == ManifestEntryStatus.EXISTING)
-            and entry.data_sequence_number is not None
-            and (self._min_data_sequence_number is None or entry.data_sequence_number < self._min_data_sequence_number)
+            and entry.sequence_number is not None
+            and (self._min_data_sequence_number is None or entry.sequence_number < self._min_data_sequence_number)

Review Comment:
   Renamed _min_data_sequence_number to _min_sequence_number in ManifestWriter.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#issuecomment-2214953159

   @kevinjqliu missed to rename a variable, hence CI checks failed in mypy stage, it is fixed now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1669074287


##########
pyiceberg/manifest.py:
##########
@@ -762,10 +762,10 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter:
 
         if (
             (entry.status == ManifestEntryStatus.ADDED or entry.status == ManifestEntryStatus.EXISTING)
-            and entry.data_sequence_number is not None
-            and (self._min_data_sequence_number is None or entry.data_sequence_number < self._min_data_sequence_number)
+            and entry.sequence_number is not None
+            and (self._min_data_sequence_number is None or entry.sequence_number < self._min_data_sequence_number)

Review Comment:
   Also renamed `min_data_sequence_number` to `min_sequence_number` in init file pyiceberg.table package, no change required in tests as the variable usage was internal.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#issuecomment-2222132868

   Thanks @kevinjqliu @HonahX for your review comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX merged PR #900:
URL: https://github.com/apache/iceberg-python/pull/900


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1669065944


##########
tests/utils/test_manifest.py:
##########
@@ -66,7 +66,7 @@ def test_read_manifest_entry(generated_manifest_entry_file: str) -> None:
 
     assert manifest_entry.status == ManifestEntryStatus.ADDED

Review Comment:
   ah i see, thank you. it is implicitly checked in that test 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#issuecomment-2218427733

   @kevinjqliu Changes made as per review comment.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#issuecomment-2218488117

   r? @HonahX / @Fokko 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1667536334


##########
pyiceberg/manifest.py:
##########
@@ -762,10 +762,10 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter:
 
         if (
             (entry.status == ManifestEntryStatus.ADDED or entry.status == ManifestEntryStatus.EXISTING)
-            and entry.data_sequence_number is not None
-            and (self._min_data_sequence_number is None or entry.data_sequence_number < self._min_data_sequence_number)
+            and entry.sequence_number is not None
+            and (self._min_data_sequence_number is None or entry.sequence_number < self._min_data_sequence_number)

Review Comment:
   nit: there are a few variables named `min_data_sequence_number`, should we change those to match? 
   https://github.com/search?q=repo%3Aapache%2Ficeberg-python%20data_sequence_number&type=code



##########
tests/utils/test_manifest.py:
##########
@@ -66,7 +66,7 @@ def test_read_manifest_entry(generated_manifest_entry_file: str) -> None:
 
     assert manifest_entry.status == ManifestEntryStatus.ADDED

Review Comment:
   nit: add a test to check the string representation fo manifest_entry to show that the new field is called `sequence_number` 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1667687714


##########
pyiceberg/manifest.py:
##########
@@ -762,10 +762,10 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter:
 
         if (
             (entry.status == ManifestEntryStatus.ADDED or entry.status == ManifestEntryStatus.EXISTING)
-            and entry.data_sequence_number is not None
-            and (self._min_data_sequence_number is None or entry.data_sequence_number < self._min_data_sequence_number)
+            and entry.sequence_number is not None
+            and (self._min_data_sequence_number is None or entry.sequence_number < self._min_data_sequence_number)

Review Comment:
   That thought did cross my mind and then I checked the manifest list files and in the actual files it is correct (i.e. `sequence_number` and `min_sequence_number`). 
   We only have (private) variable `_min_data_sequence_number` in `ManifestWriter` class, so I left it as it is. 
   I guess we should standardize them and make the variable names same as fields in spec.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1668994905


##########
pyiceberg/manifest.py:
##########
@@ -762,10 +762,10 @@ def add_entry(self, entry: ManifestEntry) -> ManifestWriter:
 
         if (
             (entry.status == ManifestEntryStatus.ADDED or entry.status == ManifestEntryStatus.EXISTING)
-            and entry.data_sequence_number is not None
-            and (self._min_data_sequence_number is None or entry.data_sequence_number < self._min_data_sequence_number)
+            and entry.sequence_number is not None
+            and (self._min_data_sequence_number is None or entry.sequence_number < self._min_data_sequence_number)

Review Comment:
   i think standardizing them would make it easier to associate the two variables in the future



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on code in PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#discussion_r1669055951


##########
tests/utils/test_manifest.py:
##########
@@ -66,7 +66,7 @@ def test_read_manifest_entry(generated_manifest_entry_file: str) -> None:
 
     assert manifest_entry.status == ManifestEntryStatus.ADDED

Review Comment:
   I think this is already getting validated in [here](https://github.com/apache/iceberg-python/pull/900/files#diff-9a5a57e7ec723cc4ab6056ce1921f291a501158322d2e6ebf817d568c9621f4bL187).
   At the end of the test we are comparing the contents of avro file with the expected `ManifestEntry` object.
   Please correct me if my understanding is wrong.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Rename data_sequence_number to sequence_number in ManifestEntry [iceberg-python]

Posted by "soumya-ghosh (via GitHub)" <gi...@apache.org>.
soumya-ghosh commented on PR #900:
URL: https://github.com/apache/iceberg-python/pull/900#issuecomment-2220747725

   @HonahX Changes done to address failures due to upstream code


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org