You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/16 12:09:32 UTC
[GitHub] [arrow] rouault opened a new pull request, #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
rouault opened a new pull request, #13164:
URL: https://github.com/apache/arrow/pull/13164
The current documentation of Statistics::num_values() is a bit
ambiguous as it mentions the 'total number of values' and my initial
understanding is that it also included null values. But experimentation
and documentation of https://arrow.apache.org/docs/python/generated/pyarrow.parquet.Statistics.html
shows that it is the number of non-null values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1127591525
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1128442440
`num_values` is used extensively in Arrow code, IMO we can keep it.
Though the name is not very clear, looks it not rare that `values` excludes `nulls`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] rouault commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
rouault commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1128685860
> To replace `num_not_null` with `num_values`, and `num_null` with `null_count`
Good idea. done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1129484991
CI error is not related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13164: MINOR: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1129645976
Benchmark runs are scheduled for baseline = cb5c039a2cd0fe791e3fbfc7935e8a397366d092 and contender = ed084f6bcd3336bf08d326546883854244768f80. ed084f6bcd3336bf08d326546883854244768f80 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/4dd2715447044a4693d1238d54526984...799c02ca49794753848ba72c3ca738cb/)
[Failed :arrow_down:0.62% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/a7807333f9ca440ea4d715158f45de36...dc466ac7d61b4be9b026f13acba67e59/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/f2353929b8d14514840d13458fecece0...7cd8f57cc3f14784af92d5ff4a59c196/)
[Finished :arrow_down:0.16% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/7ecfd6510f904efd8270906a42f1015a...f93dbaa6d111467baae708109466c54c/)
Buildkite builds:
[Finished] [`ed084f6b` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/780)
[Failed] [`ed084f6b` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/777)
[Finished] [`ed084f6b` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/767)
[Finished] [`ed084f6b` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/783)
[Finished] [`cb5c039a` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/779)
[Failed] [`cb5c039a` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/776)
[Finished] [`cb5c039a` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/766)
[Finished] [`cb5c039a` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/782)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1128671770
Thanks for the explanation @rouault .
Instead of replacing `num_values` with `num_not_null`, does it make sense to reverse it? To replace `num_not_null` with `num_values`, and `num_null` with `null_count` (the function names are `IncrementNumValues` and `IncrementNullCount`). Will it make the code easier to reason?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] rouault commented on pull request #13164: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
rouault commented on PR #13164:
URL: https://github.com/apache/arrow/pull/13164#issuecomment-1128647690
> IMO we can keep it.
The change of name of the private member is to be consistent with the ``num_not_null`` argument of TypedStatisticsImpl::Update at https://github.com/apache/arrow/blob/52a051b1f01669647d8f4d16d7de30c10265dc51/cpp/src/parquet/statistics.cc#L565, which is how I realized what the semantics of num_values() was
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 closed pull request #13164: MINOR: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
Posted by GitBox <gi...@apache.org>.
cyb70289 closed pull request #13164: MINOR: [C++] cpp/parquet/Statistics: clarify that num_values() is the number of non-null values
URL: https://github.com/apache/arrow/pull/13164
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org