You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "rustyconover (via GitHub)" <gi...@apache.org> on 2026/03/04 13:55:53 UTC
[PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
rustyconover opened a new pull request, #402:
URL: https://github.com/apache/arrow-js/pull/402
## Summary
When a zero-column RecordBatch is deserialized from IPC, `ensureSameLengthData`
in the `RecordBatch` constructor recomputes length from children via
`chunks.reduce((max, col) => Math.max(max, col.length), 0)`. With zero children,
this always returns 0 — discarding the original length from the IPC message header.
Other Arrow implementations (PyArrow, Arrow Go, arrow-rs) correctly preserve
numRows for zero-column batches.
## Fix
Pass `this.data.length` to `ensureSameLengthData` as the explicit `maxLength`
parameter, which the function already accepts as an optional third argument.
For batches with columns, `this.data.length` already matches the max column
length, so there is no behavior change.
## Tests
- Read a PyArrow-generated zero-column IPC stream (100 rows) and verify numRows
- JS round-trip: write + read zero-column batch, verify numRows preserved
- Direct constructor: verify zero-column RecordBatch preserves length
Closes #401
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-4002416877
I'm writing a RPC layer using Arrow IPC (https://vgi-rpc.query.farm) that calls a function that takes no arguments 200 times, I want to serialize those arguments to those function calls (that take no arguments), so I wind up with a zero-column record batch that has 200 rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou merged PR #402:
URL: https://github.com/apache/arrow-js/pull/402
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on code in PR #402:
URL: https://github.com/apache/arrow-js/pull/402#discussion_r2889883363
##########
test/data/zero_column_batch.arrow:
##########
Review Comment:
Done — removed the binary file and the test now generates the IPC buffer on-demand using the JS writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-3997735569
@raulcd can you please trigger the CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "trxcllnt (via GitHub)" <gi...@apache.org>.
trxcllnt commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-4002620455
And that's not representable by a `Table<{ args: FixedSizeList(0) }>`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-4004965841
@raulcd could you please trigger the CI again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #402:
URL: https://github.com/apache/arrow-js/pull/402#discussion_r2887235440
##########
test/data/zero_column_batch.arrow:
##########
Review Comment:
Can we create this file by our writer in test instead of adding this file to the repository?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "trxcllnt (via GitHub)" <gi...@apache.org>.
trxcllnt commented on code in PR #402:
URL: https://github.com/apache/arrow-js/pull/402#discussion_r2887666114
##########
test/data/zero_column_batch.arrow:
##########
Review Comment:
Agreed, we should generate this file on-demand instead of checking it in. We'll need to rebase it out of the commit history before merging this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-4004844905
I'm trying to use a consistent API throughout the implementations. I'm only working with RecordBatches. Each argument to the function is a column, not a list. Arguments can vary in types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Preserve numRows for zero-column RecordBatch in IPC [arrow-js]
Posted by "rustyconover (via GitHub)" <gi...@apache.org>.
rustyconover commented on PR #402:
URL: https://github.com/apache/arrow-js/pull/402#issuecomment-4004943427
Addressed — removed the checked-in `zero_column_batch.arrow` file and updated the test to generate the IPC buffer on-demand using the JS writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org