You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "MichielMortier (via GitHub)" <gi...@apache.org> on 2026/02/24 14:39:30 UTC

[I] `pqarrow.SchemaField.IsLeaf()` unreliable because `ColIndex` is never set to -1 for non-leaves [arrow-go]

MichielMortier opened a new issue, #673:
URL: https://github.com/apache/arrow-go/issues/673

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   **Package:** `github.com/apache/arrow/go/v18/parquet/pqarrow`
   
   **Summary:**  
   `SchemaField.IsLeaf()` is documented/implemented as `ColIndex != -1`, but the library never sets `ColIndex` to `-1` for non-leaf (nested) fields. As a result, non-leaf nodes can have a non-negative `ColIndex`, so `IsLeaf()` returns `true` when it should return `false`.
   
   **Expected behavior:**  
   - Leaf fields (actual Parquet columns): `ColIndex >= 0`, `IsLeaf() == true`.  
   - Non-leaf fields (nested/parent nodes): `ColIndex == -1` (or similar sentinel), `IsLeaf() == false`.
   
   **Actual behavior:**  
   Non-leaf fields often keep a default or incorrect `ColIndex` (e.g. 0 or another value). So `IsLeaf()` can be `true` for parent nodes, and code that uses `IsLeaf()` to decide “is this a physical column?” gets wrong results.
   
   **Workaround:**  
   Determine “leaf” by structure instead of `IsLeaf()`: treat a field as a leaf when it has no children, e.g. `len(f.Children) == 0`. Only use `f.ColIndex` for such nodes when collecting column indices.
   
   
   ### Component(s)
   
   Parquet


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `pqarrow.SchemaField.IsLeaf()` unreliable because `ColIndex` is never set to -1 for non-leaves [arrow-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on issue #673:
URL: https://github.com/apache/arrow-go/issues/673#issuecomment-3953338111

   All instances of `SchemaField` in `pqarrow` should be initialized via `makeSchemaFields` which always sets `ColIndex` to `-1`. In what scenario will a non-leaf node have a non-negative `ColIndex`? Can you create an example that shows this bug at all?
   
   > Non-leaf fields often keep a default or incorrect ColIndex (e.g. 0 or another value). So IsLeaf() can be true for parent nodes, and code that uses IsLeaf() to decide “is this a physical column?” gets wrong results.
   
   I'm not aware of any scenario in the code where this occurs, is this a bug you've encountered?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `pqarrow.SchemaField.IsLeaf()` unreliable because `ColIndex` is never set to -1 for non-leaves [arrow-go]

Posted by "MichielMortier (via GitHub)" <gi...@apache.org>.
MichielMortier commented on issue #673:
URL: https://github.com/apache/arrow-go/issues/673#issuecomment-3960051377

   Thanks for the response, when looking further I saw i was not on the newest version because my go import was using an old repo (github.com/apache/arrow/go/v18). The issue was fixed in https://github.com/apache/arrow-go/pull/591.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `pqarrow.SchemaField.IsLeaf()` unreliable because `ColIndex` is never set to -1 for non-leaves [arrow-go]

Posted by "MichielMortier (via GitHub)" <gi...@apache.org>.
MichielMortier closed issue #673: `pqarrow.SchemaField.IsLeaf()` unreliable because `ColIndex` is never set to -1 for non-leaves
URL: https://github.com/apache/arrow-go/issues/673


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org