You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "kentkwu (via GitHub)" <gi...@apache.org> on 2026/03/04 21:06:11 UTC

[PR] fix: tableFromJSON and tableFromArrays crash on arrays of strings [arrow-js]

kentkwu opened a new pull request, #405:
URL: https://github.com/apache/arrow-js/pull/405

   ## What's Changed
   
   Applies a fix for type inference crashing on encountering an array of strings.
   
   Added a `compareTypesForInference` helper that compares Dictionary types structurally, ignoring IDs. It recurses through List and Struct for nested cases. The public compareTypes is unchanged. ID equality still matters for IPC comparisons.
   
   
   **Root cause:**
   
   Strings infer to Dictionary<Utf8, Int32>, and every Dictionary construction auto-increments a global ID. The array homogeneity check in inferType calls compareTypes on separately inferred types, which requires matching IDs — so it always fails for string arrays.
   
   Closes #86, #50
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: tableFromJSON and tableFromArrays crash on arrays of strings [arrow-js]

Posted by "trxcllnt (via GitHub)" <gi...@apache.org>.
trxcllnt commented on code in PR #405:
URL: https://github.com/apache/arrow-js/pull/405#discussion_r2887644428


##########
src/factories.ts:
##########
@@ -149,7 +173,7 @@ function inferType(value: readonly unknown[]): dtypes.DataType {
     } else if (arraysCount + nullsCount === value.length) {
         const array = value as Array<unknown>[];
         const childType = inferType(array[array.findIndex((ary) => ary != null)]);
-        if (array.every((ary) => ary == null || compareTypes(childType, inferType(ary)))) {
+        if (array.every((ary) => ary == null || compareTypesForInference(childType, inferType(ary)))) {

Review Comment:
   What if we just passed the dictionaryId down through `inferType`? e.g. 
   ```suggestion
           if (array.every((ary) => ary == null || compareTypes(childType, inferType(ary, dtypes.DataType.isDictionary(childType) ? childType.id : null)))) {
   ```
   
   Then we could pass the dictionary id into the Dictionary constructor on L142?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: tableFromJSON and tableFromArrays crash on arrays of strings [arrow-js]

Posted by "trxcllnt (via GitHub)" <gi...@apache.org>.
trxcllnt commented on code in PR #405:
URL: https://github.com/apache/arrow-js/pull/405#discussion_r2887936539


##########
src/factories.ts:
##########
@@ -149,7 +173,7 @@ function inferType(value: readonly unknown[]): dtypes.DataType {
     } else if (arraysCount + nullsCount === value.length) {
         const array = value as Array<unknown>[];
         const childType = inferType(array[array.findIndex((ary) => ary != null)]);
-        if (array.every((ary) => ary == null || compareTypes(childType, inferType(ary)))) {
+        if (array.every((ary) => ary == null || compareTypesForInference(childType, inferType(ary)))) {

Review Comment:
   Would probably need to maintain a Map of key -> dictionary id in order to do the same thing for dictionary-encoded struct children.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: tableFromJSON and tableFromArrays crash on arrays of strings [arrow-js]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #405:
URL: https://github.com/apache/arrow-js/pull/405#issuecomment-4001719644

   @matheuzinoficial @hemkumarrao-ni @knight556 Could you try this fix?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org