[jira] [Commented] (ARROW-1693) [JS] Error reading dictionary-encoded integration test files

["Paul Taylor (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236855#comment-16236855 ] 

Paul Taylor commented on ARROW-1693:
------------------------------------

[~wesmckinn] digging into this now, yeah it looks like the the DictionaryBatch UTF8Vector fieldNode don't include the offsets buffer. Sounds like I should get those integration tests up and running.

I wanna offer some push back on your comment about brittleness though. Maybe I'm alone on this, but seems like a cross-platform ipc format should strictly enforce its own spec -- anything less and end up with a bunch of maybe-compatible implementations, right?

> [JS] Error reading dictionary-encoded integration test files
> ------------------------------------------------------------
>
>                 Key: ARROW-1693
>                 URL: https://issues.apache.org/jira/browse/ARROW-1693
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: JavaScript
>            Reporter: Brian Hulette
>            Assignee: Brian Hulette
>            Priority: Major
>             Fix For: 0.8.0
>
>         Attachments: dictionary-cpp.arrow, dictionary-java.arrow, dictionary.json
>
>
> The JS implementation crashes when reading the dictionary test case from the integration tests.
> To replicate, first generate the test files with java and cpp impls:
> {code}
> $ cd ${ARROW_HOME}/integration/
> $ python -c 'from integration_test import generate_dictionary_case; generate_dictionary_case().write("dictionary.json")'
> $ ../cpp/debug/debug/json-integration-test --integration --json=dictionary.json --arrow=dictionary-cpp.arrow --mode=JSON_TO_ARROW
> $ java -cp ../java/tools/target/arrow-tools-0.8.0-SNAPSHOT-jar-with-dependencies.jar org.apache.arrow.tools.Integration -c JSON_TO_ARROW -a dictionary-java.arrow -j dictionary.json
> {code}
> Attempt to read the files with the JS impl:
> {code}
> $ cd ${ARROW_HOME}/js/
> $ ./bin/arrow2csv.js -s dict1_0 -f ../integration/dictionary-{java,cpp}.arrow
> {code}
> Both files result in an error for me on [a8f51858|https://github.com/apache/arrow/commit/a8f518588fda471b2e3cc8e0f0064e7c4bb99899]:
> {{TypeError: Cannot read property 'buffer' of undefined}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)