You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Paul Taylor (Jira)" <ji...@apache.org> on 2021/02/05 05:36:00 UTC
[jira] [Commented] (ARROW-11326) utf8 vector buffers don't work if
allocated within Web Assembly memory of Node.js
[ https://issues.apache.org/jira/browse/ARROW-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279350#comment-17279350 ]
Paul Taylor commented on ARROW-11326:
-------------------------------------
This sounds like a bug in node's Buffer class?
We use the Buffer to utf8 encode and decode in node, because it was (at the time of authorship) dramatically faster than TextEncoder/TextDecoder.
> utf8 vector buffers don't work if allocated within Web Assembly memory of Node.js
> ---------------------------------------------------------------------------------
>
> Key: ARROW-11326
> URL: https://issues.apache.org/jira/browse/ARROW-11326
> Project: Apache Arrow
> Issue Type: Bug
> Components: JavaScript
> Environment: node.js in Mac book pro
> Reporter: Dmitri Bronnikov
> Priority: Major
>
> After making int32array of offsets = [0, 1] and uint8array of values = [ascii_code('A')], create a vector of strings:
> const vec = arrow.Vector.new(arrow.Data.new(new Utf8(), 0, 1, 0, [offsets, values, null, null])
> then access the first and only element:
> console.log(vec.get(0))
> Works within browsers. Works in node.js with fixed size types, e.g. float or integer.
> Fails in Node.js (v14.11.0.) with this callstack
> at ../../node_modules/@apache-arrow/es2015-umd/buffer/index.js:311:1
> at __proto__ (../../node_modules/@apache-arrow/es2015-umd/buffer/index.js:167:1)
> at Function._Buffer [as from] (../../node_modules/@apache-arrow/es2015-umd/buffer/index.js:154:1)
> at prototype (../../node_modules/@apache-arrow/es2015-umd/util/utf8.ts:43:31)
> at partial2 (../../node_modules/@apache-arrow/es2015-umd/visitor/get.ts:293:12)
> at go.isArray [as get] (../../node_modules/@apache-arrow/es2015-umd/vector/index.ts:175:43)
> at Sr.get (../../node_modules/@apache-arrow/es2015-umd/util/args.ts:27:7)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)