You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Abhimanuchauhan01 (via GitHub)" <gi...@apache.org> on 2025/01/24 17:51:09 UTC

[I] "Request for New Feature: Optimized Query Execution" [iceberg]

Abhimanuchauhan01 opened a new issue, #12097:
URL: https://github.com/apache/iceberg/issues/12097

   ### Feature Request / Improvement
   
   ### Feature Description
   We propose adding enhanced support for nested structures in Iceberg’s partitioning system. Currently, Iceberg’s partitioning support is limited to top-level columns, which restricts the flexibility of data partitioning in complex datasets. 
   
   ### Use Case
   In scenarios where large datasets include deeply nested structures (e.g., JSON data with arrays or maps), the ability to partition on nested fields would allow for more efficient data management and querying. This would improve performance for users who need to partition on these fields but are currently forced to flatten the data before partitioning.
   
   ### Motivation
   Supporting partitioning on nested structures would provide the following benefits:
   - **Improved performance**: Efficiently partition large datasets with complex nested structures.
   - **Better query optimization**: Queries on partitioned data would be more efficient by targeting specific subfields in nested structures.
   - **Increased flexibility**: Allow users to partition on any part of the dataset, not just top-level columns.
   
   ### Example Use Case
   A user working with nested JSON data could partition the data based on specific fields inside arrays or maps, improving the performance of queries that filter on these fields.
   
   ### Potential Challenges
   There may be performance trade-offs when partitioning on deeply nested fields, especially with large, complex datasets. A careful design of partitioning strategies would be necessary to avoid unnecessary overhead.
   
   
   ### Query engine
   
   Hive
   
   ### Willingness to contribute
   
   - [ ] I can contribute this improvement/feature independently
   - [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
   - [ ] I cannot contribute this improvement/feature at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] "Request for New Feature: Optimized Query Execution" [iceberg]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer closed issue #12097: "Request for New Feature: Optimized Query Execution"
URL: https://github.com/apache/iceberg/issues/12097


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] "Request for New Feature: Optimized Query Execution" [iceberg]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #12097:
URL: https://github.com/apache/iceberg/issues/12097#issuecomment-2613097485

   Closed, as I'm pretty sure this is a purely AI generated issue. I will reopen if i'm wrong. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] "Request for New Feature: Optimized Query Execution" [iceberg]

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #12097:
URL: https://github.com/apache/iceberg/issues/12097#issuecomment-2613101963

   As a quick note for anyone looking at this - Iceberg supports partitioning on any field in the schema. There are specific type restrictions like no complex types, removing that restriction would be very difficult and have limited value. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org