You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "zeroshade (via GitHub)" <gi...@apache.org> on 2024/09/05 18:34:07 UTC
[PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
zeroshade opened a new pull request, #43976:
URL: https://github.com/apache/arrow/pull/43976
### Rationale for this change
Widening the Decimal128/256 type to allow for bitwidths of 32 and 64 allows for more interoperability with other libraries and utilities which already support these types. This provides even more opportunities for zero-copy interactions between things such as libcudf and various databases.
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
### What changes are included in this PR?
Updating the documentation in Schema.fbs to explicitly state that 32-bit and 64-bit is now allowed for bitwidths of Decimal types. This is the only area in the the spec that mentions the allowed decimal bitwidths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43976:
URL: https://github.com/apache/arrow/pull/43976#discussion_r1752176788
##########
docs/source/format/Columnar.rst:
##########
@@ -21,7 +21,7 @@
Arrow Columnar Format
*********************
-*Version: 1.4*
+*Version: 1.5*
.. seealso:: :ref:`Additions to the Arrow columnar format since version 1.0.0
<post-1-0-0-format-versions>`
Review Comment:
Can you also add a subsection for version 1.5 in https://github.com/apache/arrow/blob/b4b22a4635fcf9bb1a36bf26411c6bde2ae01732/docs/source/format/Versioning.rst#post-100-format-versions ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43976:
URL: https://github.com/apache/arrow/pull/43976#discussion_r1752290912
##########
docs/source/format/Versioning.rst:
##########
@@ -105,3 +105,8 @@ Version 1.4
* Added :ref:`listview-layout` and the associated ListView and LargeListView
types.
* Added :ref:`variadic-buffers`.
+
+Version 1.5
+-----------
+
+* Expanded Decimal type bit widths to allow 32-bit and 64-bit types
Review Comment:
Nittiest nit
```suggestion
* Expanded Decimal type bit widths to allow 32-bit and 64-bit types.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43976:
URL: https://github.com/apache/arrow/pull/43976#discussion_r1752272575
##########
docs/source/format/Columnar.rst:
##########
@@ -21,7 +21,7 @@
Arrow Columnar Format
*********************
-*Version: 1.4*
+*Version: 1.5*
.. seealso:: :ref:`Additions to the Arrow columnar format since version 1.0.0
<post-1-0-0-format-versions>`
Review Comment:
added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #43976:
URL: https://github.com/apache/arrow/pull/43976#issuecomment-2342680345
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit c4b68ee561cd3d2363f83f00b4085145bf3e3807.
There were 4 benchmark results indicating a performance regression:
- Commit Run on `amd64-m5-4xlarge-linux` at [2024-09-11 01:22:10Z](https://conbench.ursa.dev/compare/runs/90291c8e37104a7bb33a60aab73eb4cf...d9a16b92652e4b28b5d6a90f8e485157/)
- [`file-write` (R) with compression=uncompressed, dataset=nyctaxi_2010-01, file_type=parquet, input_type=dataframe, language=R](https://conbench.ursa.dev/compare/benchmarks/066e0dd2c24f7c5980007302f3c0ac50...066e107ff79b7f238000adcdf42052f1)
- [`dataset-serialize` (Python) with dataset=nyctaxi_multi_ipc_s3, format=parquet, selectivity=1pc](https://conbench.ursa.dev/compare/benchmarks/066e0c65ed4573708000e511cfc6f33f...066e0f0caf707b378000c8cd2c96eb4f)
- and 2 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/29973488343) has more details. It also includes information about 119 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43956: [Format] Allow Decimal32/Decimal64 in format [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #43976:
URL: https://github.com/apache/arrow/pull/43976
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org