You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "devinrsmith (via GitHub)" <gi...@apache.org> on 2023/04/05 17:19:44 UTC
[GitHub] [parquet-format] devinrsmith opened a new pull request, #198: PARQUET-2264: Allow scale == precision for DecimalType
devinrsmith opened a new pull request, #198:
URL: https://github.com/apache/parquet-format/pull/198
The majority of implementations allow for scale == precision.
See https://github.com/apache/arrow-rs/pull/1607 for further motivation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-format] wgtmac merged pull request #198: PARQUET-2264: Allow scale == precision for DecimalType
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac merged PR #198:
URL: https://github.com/apache/parquet-format/pull/198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-format] mapleFU commented on pull request #198: PARQUET-2264: Allow scale == precision for DecimalType
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #198:
URL: https://github.com/apache/parquet-format/pull/198#issuecomment-1499297787
```
diff --git a/cpp/examples/parquet/parquet_arrow/reader_writer.cc b/cpp/examples/parquet/parquet_arrow/reader_writer.cc
index f5d96ec16..a4c4e5f77 100644
--- a/cpp/examples/parquet/parquet_arrow/reader_writer.cc
+++ b/cpp/examples/parquet/parquet_arrow/reader_writer.cc
@@ -27,10 +27,12 @@
// To have some input data, we first create an Arrow Table that holds
// some data.
std::shared_ptr<arrow::Table> generate_table() {
- arrow::Int64Builder i64builder;
- PARQUET_THROW_NOT_OK(i64builder.AppendValues({1, 2, 3, 4, 5}));
- std::shared_ptr<arrow::Array> i64array;
- PARQUET_THROW_NOT_OK(i64builder.Finish(&i64array));
+ arrow::Decimal128Builder decimalBuilder(arrow::decimal(15, 15));
+ for (int i = 0; i < 5; ++i) {
+ PARQUET_THROW_NOT_OK(decimalBuilder.Append(arrow::Decimal128()));
+ }
+ std::shared_ptr<arrow::Array> decimalArray;
+ PARQUET_THROW_NOT_OK(decimalBuilder.Finish(&decimalArray));
arrow::StringBuilder strbuilder;
PARQUET_THROW_NOT_OK(strbuilder.Append("some"));
@@ -42,9 +44,9 @@ std::shared_ptr<arrow::Table> generate_table() {
PARQUET_THROW_NOT_OK(strbuilder.Finish(&strarray));
std::shared_ptr<arrow::Schema> schema = arrow::schema(
- {arrow::field("int", arrow::int64()), arrow::field("str", arrow::utf8())});
+ {arrow::field("decimal", arrow::decimal(15, 15)), arrow::field("str", arrow::utf8())});
- return arrow::Table::Make(schema, {i64array, strarray});
+ return arrow::Table::Make(schema, {decimalArray, strarray});
}
// #1 Write out the data as a Parquet file
```
I've do an experiment with C++ arrow parquet, and it's able to write decimal with `Decimal(15, 15)`. So I think this patch is ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-format] wgtmac commented on a diff in pull request #198: PARQUET-2264: Allow scale == precision for DecimalType
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #198:
URL: https://github.com/apache/parquet-format/pull/198#discussion_r1159239875
##########
LogicalTypes.md:
##########
@@ -219,7 +219,7 @@ decimal point, and the precision stores the maximum number of digits supported
in the unscaled value.
If not specified, the scale is 0. Scale must be zero or a positive integer less
-than the precision. Precision is required and must be a non-zero positive
+than or equal to the precision. Precision is required and must be a non-zero positive
Review Comment:
Thanks for the PR! Could you also add the comment to the thrift definition: https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L253 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-format] devinrsmith commented on a diff in pull request #198: PARQUET-2264: Allow scale == precision for DecimalType
Posted by "devinrsmith (via GitHub)" <gi...@apache.org>.
devinrsmith commented on code in PR #198:
URL: https://github.com/apache/parquet-format/pull/198#discussion_r1159871175
##########
LogicalTypes.md:
##########
@@ -219,7 +219,7 @@ decimal point, and the precision stores the maximum number of digits supported
in the unscaled value.
If not specified, the scale is 0. Scale must be zero or a positive integer less
-than the precision. Precision is required and must be a non-zero positive
+than or equal to the precision. Precision is required and must be a non-zero positive
Review Comment:
Will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org