You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "kevinjqliu (via GitHub)" <gi...@apache.org> on 2025/11/11 18:53:09 UTC

[I] fix pyiceberg-core nightly [iceberg-rust]

kevinjqliu opened a new issue, #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837

   ### Apache Iceberg Rust version
   
   None
   
   ### Describe the bug
   
   pyiceberg-core nightly has been broken since 11/3
   
   https://github.com/apache/iceberg-rust/actions/workflows/release_python_nightly.yml
   https://test.pypi.org/project/pyiceberg-core
   
   we finally found the underlying issue which is due to the size of the files
   ```
   Uploading 
   pyiceberg_core-0.7.0.dev20251111175828-cp39-abi3-macosx_10_12_x86_64.macosx_11_0
   _arm64.macosx_10_12_universal2.whl
   INFO     Response from https://test.pypi.org/legacy/:                           
            400 Bad Request                                                        
   INFO     <html>                                                                 
             <head>                                                                
              <title>400 Project size too large. Limit for project 'pyiceberg-core'
            total size is 10 GB. See                                               
            https://docs.pypi.org/project-management/storage-limits#requesting-a-pr
            oject-size-limit-increase</title>                                      
             </head>                                                               
             <body>                                                                
              <h1>400 Project size too large. Limit for project 'pyiceberg-core'   
            total size is 10 GB. See                                               
            https://docs.pypi.org/project-management/storage-limits#requesting-a-pr
            oject-size-limit-increase</h1>                                         
              The server could not comply with the request since it is either      
            malformed or otherwise incorrect.<br/><br/>                            
            Project size too large. Limit for project &#x27;pyiceberg-core&#x27;   
            total size is 10 GB. See                                               
            https://docs.pypi.org/project-management/storage-limits#requesting-a-pr
            oject-size-limit-increase                                              
                                                                                   
                                                                                   
             </body>                                                               
            </html>                                                                
   ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/          
            Bad Request                                                            
   ```
   
   ### To Reproduce
   
   _No response_
   
   ### Expected behavior
   
   _No response_
   
   ### Willingness to contribute
   
   None


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3518391093

   increase a file limit increase request to 200MB, for both pypi and testpypi
   https://github.com/pypi/support/issues/8198


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #1837: fix pyiceberg-core nightly
URL: https://github.com/apache/iceberg-rust/issues/1837


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3522774421

   nightly pipeline is now running successfully. 
   
   For the 100MB per-file limit, 
   - [x] tuned the wheels (see https://github.com/apache/iceberg-rust/issues/1839)
   - [x] requested an increase for file limit to 200MB (pypi/support#8198) for more buffer
   
   For the 10GB per project limit, 
   - [x] manually removed old releases on testpypi 
   - [ ] setup some kind of maintenance / automation to periodically prune old releases


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3518381954

   we're hitting the per file limit of 100MB. i added an issue to investigate https://github.com/apache/iceberg-rust/issues/1839 and potentially reduce the size. 
   
   I'll also file a request to increase the limit for testpypi (and pypi if we must)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3518339133

   We should aggressively retain away the old builds and also check why pyiceberg-core wheels are now 100MB each... 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3518374854

   https://docs.pypi.org/project-management/storage-limits/#requesting-a-project-size-limit-increase


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] fix pyiceberg-core nightly [iceberg-rust]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1837:
URL: https://github.com/apache/iceberg-rust/issues/1837#issuecomment-3518458879

   to trim down the total size, we can automatically retain away (delete) old releases for testpypi
   with https://github.com/arcivanov/pypi-cleanup
   ```
   uvx pypi-cleanup --package pyiceberg-core --host https://test.pypi.org/ --verbose -d 30 --query-only
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org