You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/08/15 11:36:33 UTC
[GitHub] [arrow-datafusion] Ted-Jiang opened a new pull request, #3146: Support number of histogram bins in approx_percentile_cont
Ted-Jiang opened a new pull request, #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146
# Which issue does this PR close?
Closes #3142, #3145
Now we support approx_percentile_cont(col_name, percentile value, number of histogram bins)
```
DataFusion CLI v10.0.0
❯ create external table order stored as parquet location '/Users/yangjiang/test-data/tpch-1g/orders';
0 rows in set. Query took 0.016 seconds.
❯ select o_orderstatus, approx_percentile_cont(o_totalprice, 0.5) from order group by o_orderstatus order by 1 limit 3;
+---------------+-------------------------------------------------------+
| o_orderstatus | APPROXPERCENTILECONT(order.o_totalprice,Float64(0.5)) |
+---------------+-------------------------------------------------------+
| F | 143415.09074751436 |
| O | 143177.55673372833 |
| P | 181407.62680871075 |
+---------------+-------------------------------------------------------+
3 rows in set. Query took 0.728 seconds.
❯ select o_orderstatus, approx_percentile_cont(o_totalprice, 0.5, 10000) from order group by o_orderstatus order by 1 limit 3;
+---------------+--------------------------------------------------------------------+
| o_orderstatus | APPROXPERCENTILECONT(order.o_totalprice,Float64(0.5),Int64(10000)) |
+---------------+--------------------------------------------------------------------+
| F | 143289.22569288642 |
| O | 143184.3388358634 |
| P | 181451.707578125 |
+---------------+--------------------------------------------------------------------+
3 rows in set. Query took 1.168 seconds.
❯
```
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3146: Support number of histogram bins in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#issuecomment-1214947988
# [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3146?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3146](https://codecov.io/gh/apache/arrow-datafusion/pull/3146?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4ffb103) into [master](https://codecov.io/gh/apache/arrow-datafusion/commit/48f9b7ab57edce2548997e358eb589c5f23e1bad?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (48f9b7a) will **decrease** coverage by `0.06%`.
> The diff coverage is `39.39%`.
```diff
@@ Coverage Diff @@
## master #3146 +/- ##
==========================================
- Coverage 85.95% 85.88% -0.07%
==========================================
Files 291 291
Lines 52382 52429 +47
==========================================
+ Hits 45025 45031 +6
- Misses 7357 7398 +41
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-datafusion/pull/3146?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...sical-expr/src/aggregate/approx\_percentile\_cont.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1leHByL3NyYy9hZ2dyZWdhdGUvYXBwcm94X3BlcmNlbnRpbGVfY29udC5ycw==) | `70.05% <32.60%> (-14.91%)` | :arrow_down: |
| [datafusion/expr/src/aggregate\_function.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9hZ2dyZWdhdGVfZnVuY3Rpb24ucnM=) | `90.93% <54.54%> (-1.38%)` | :arrow_down: |
| [datafusion/physical-expr/src/aggregate/build\_in.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1leHByL3NyYy9hZ2dyZWdhdGUvYnVpbGRfaW4ucnM=) | `89.29% <55.55%> (-0.64%)` | :arrow_down: |
| [datafusion/expr/src/logical\_plan/plan.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vcGxhbi5ycw==) | `77.25% <0.00%> (-0.53%)` | :arrow_down: |
| [datafusion/common/src/scalar.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/3146/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb21tb24vc3JjL3NjYWxhci5ycw==) | `84.94% <0.00%> (+0.06%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
alamb merged PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#discussion_r947653826
##########
datafusion/physical-expr/src/aggregate/approx_percentile_cont.rs:
##########
@@ -52,39 +53,35 @@ impl ApproxPercentileCont {
// Arguments should be [ColumnExpr, DesiredPercentileLiteral]
debug_assert_eq!(expr.len(), 2);
- // Extract the desired percentile literal
- let lit = expr[1]
- .as_any()
- .downcast_ref::<Literal>()
- .ok_or_else(|| {
- DataFusionError::Internal(
- "desired percentile argument must be float literal".to_string(),
- )
- })?
- .value();
- let percentile = match lit {
- ScalarValue::Float32(Some(q)) => *q as f64,
- ScalarValue::Float64(Some(q)) => *q as f64,
- got => return Err(DataFusionError::NotImplemented(format!(
- "Percentile value for 'APPROX_PERCENTILE_CONT' must be Float32 or Float64 literal (got data type {})",
- got
- )))
- };
+ let percentile = validate_input_percentile_expr(&expr[1])?;
Review Comment:
👍
##########
datafusion/core/tests/sql/aggregates.rs:
##########
@@ -925,6 +925,54 @@ async fn csv_query_approx_percentile_cont_with_weight() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn csv_query_approx_percentile_cont_with_histogram_bins() -> Result<()> {
+ let ctx = SessionContext::new();
+ register_aggregate_csv(&ctx).await?;
+
+ // compare approx_percentile_cont and approx_percentile_cont_with_weight
+ let sql = "SELECT c1, approx_percentile_cont(c3, 0.95, 200) AS c3_p95 FROM aggregate_test_100 GROUP BY 1 ORDER BY 1";
+ let actual = execute_to_batches(&ctx, sql).await;
+ let expected = vec![
+ "+----+--------+",
Review Comment:
I am not a statistics expert but I verified that this is different than the output generated by `SELECT c1, approx_percentile_cont_with_weight(c3, c2, 0.95) AS c3_p95 FROM aggregate_test_100 GROUP BY 1 ORDER BY 1` a few lines above
##########
datafusion/physical-expr/src/aggregate/approx_percentile_cont.rs:
##########
@@ -113,6 +116,64 @@ impl ApproxPercentileCont {
}
}
+fn validate_input_percentile_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<f64> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let percentile = match lit {
+ ScalarValue::Float32(Some(q)) => *q as f64,
+ ScalarValue::Float64(Some(q)) => *q as f64,
+ got => return Err(DataFusionError::NotImplemented(format!(
+ "Percentile value for 'APPROX_PERCENTILE_CONT' must be Float32 or Float64 literal (got data type {})",
+ got.get_datatype()
+ )))
+ };
+
+ // Ensure the percentile is between 0 and 1.
+ if !(0.0..=1.0).contains(&percentile) {
+ return Err(DataFusionError::Plan(format!(
+ "Percentile value must be between 0.0 and 1.0 inclusive, {} is invalid",
+ percentile
+ )));
+ }
+ Ok(percentile)
+}
+
+fn validate_input_max_size_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<usize> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let max_size = match lit {
+ ScalarValue::UInt8(Some(q)) => *q as usize,
Review Comment:
Yeah, I think we should file a ticket to allow creating unsigned int types from SQL (like in `CREATE TABLE`) statements.
I ran into this limitation while I was fixing https://github.com/apache/arrow-datafusion/pull/3167 -- https://github.com/apache/arrow-datafusion/pull/3167/files#r946145318
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on pull request #3146: Support number of histogram bins in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#issuecomment-1214993836
@alamb PTAL😊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on a diff in pull request #3146: Support number of histogram bins in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#discussion_r945645035
##########
datafusion/physical-expr/src/aggregate/approx_percentile_cont.rs:
##########
@@ -113,6 +116,64 @@ impl ApproxPercentileCont {
}
}
+fn validate_input_percentile_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<f64> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let percentile = match lit {
+ ScalarValue::Float32(Some(q)) => *q as f64,
+ ScalarValue::Float64(Some(q)) => *q as f64,
+ got => return Err(DataFusionError::NotImplemented(format!(
+ "Percentile value for 'APPROX_PERCENTILE_CONT' must be Float32 or Float64 literal (got data type {})",
+ got.get_datatype()
+ )))
+ };
+
+ // Ensure the percentile is between 0 and 1.
+ if !(0.0..=1.0).contains(&percentile) {
+ return Err(DataFusionError::Plan(format!(
+ "Percentile value must be between 0.0 and 1.0 inclusive, {} is invalid",
+ percentile
+ )));
+ }
+ Ok(percentile)
+}
+
+fn validate_input_max_size_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<usize> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let max_size = match lit {
+ ScalarValue::UInt8(Some(q)) => *q as usize,
Review Comment:
For now i think ew can not input unsigned int from sql 🤔
So need these Uint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ursabot commented on pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#issuecomment-1217736637
Benchmark runs are scheduled for baseline = 2aa0a98d6d95cc2641216ee3ccfe954e2b2eec57 and contender = 929eb6d860fb60ba994b24397ad3c3eb7d839cdf. 929eb6d860fb60ba994b24397ad3c3eb7d839cdf is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d3f516e073c74d4a886f32b78c19d364...db1b73af40784adaa340704da8b194bb/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/4ce5b76884be47a4b035f0c99ed641b6...0e6cdc6f69ae4e8889987954f9d1754c/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/6b8267dd8ee34f96b962ac1566038682...f5115fa6707e4342a15887c60ad00f65/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f534795bfb1c49bd877ca9ad95844835...d74a5dfa98e04d97842f468eb026d10a/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
andygrove commented on PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#issuecomment-1215082829
We should update the documentation at https://github.com/apache/arrow-datafusion/blob/master/docs/source/user-guide/sql/aggregate_functions.md as part of this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on a diff in pull request #3146: Support number of histogram bins in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#discussion_r945645035
##########
datafusion/physical-expr/src/aggregate/approx_percentile_cont.rs:
##########
@@ -113,6 +116,64 @@ impl ApproxPercentileCont {
}
}
+fn validate_input_percentile_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<f64> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let percentile = match lit {
+ ScalarValue::Float32(Some(q)) => *q as f64,
+ ScalarValue::Float64(Some(q)) => *q as f64,
+ got => return Err(DataFusionError::NotImplemented(format!(
+ "Percentile value for 'APPROX_PERCENTILE_CONT' must be Float32 or Float64 literal (got data type {})",
+ got.get_datatype()
+ )))
+ };
+
+ // Ensure the percentile is between 0 and 1.
+ if !(0.0..=1.0).contains(&percentile) {
+ return Err(DataFusionError::Plan(format!(
+ "Percentile value must be between 0.0 and 1.0 inclusive, {} is invalid",
+ percentile
+ )));
+ }
+ Ok(percentile)
+}
+
+fn validate_input_max_size_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<usize> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let max_size = match lit {
+ ScalarValue::UInt8(Some(q)) => *q as usize,
Review Comment:
For now i think we can not input unsigned int from sql 🤔
So need these Int.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on a diff in pull request #3146: Support number of histogram bins in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#discussion_r945645035
##########
datafusion/physical-expr/src/aggregate/approx_percentile_cont.rs:
##########
@@ -113,6 +116,64 @@ impl ApproxPercentileCont {
}
}
+fn validate_input_percentile_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<f64> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let percentile = match lit {
+ ScalarValue::Float32(Some(q)) => *q as f64,
+ ScalarValue::Float64(Some(q)) => *q as f64,
+ got => return Err(DataFusionError::NotImplemented(format!(
+ "Percentile value for 'APPROX_PERCENTILE_CONT' must be Float32 or Float64 literal (got data type {})",
+ got.get_datatype()
+ )))
+ };
+
+ // Ensure the percentile is between 0 and 1.
+ if !(0.0..=1.0).contains(&percentile) {
+ return Err(DataFusionError::Plan(format!(
+ "Percentile value must be between 0.0 and 1.0 inclusive, {} is invalid",
+ percentile
+ )));
+ }
+ Ok(percentile)
+}
+
+fn validate_input_max_size_expr(expr: &Arc<dyn PhysicalExpr>) -> Result<usize> {
+ // Extract the desired percentile literal
+ let lit = expr
+ .as_any()
+ .downcast_ref::<Literal>()
+ .ok_or_else(|| {
+ DataFusionError::Internal(
+ "desired percentile argument must be float literal".to_string(),
+ )
+ })?
+ .value();
+ let max_size = match lit {
+ ScalarValue::UInt8(Some(q)) => *q as usize,
Review Comment:
For now i think ew can not input unsigned int from sql 🤔
So need these Int.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#issuecomment-1216160384
> We should update the documentation at https://github.com/apache/arrow-datafusion/blob/master/docs/source/user-guide/sql/aggregate_functions.md as part of this PR
Thanks your remind @andygrove add in [7d67a2f](https://github.com/apache/arrow-datafusion/pull/3146/commits/7d67a2fd4f4d266d78891f93eebdd4ec2db8fbd1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on a diff in pull request #3146: Support number of centroids in approx_percentile_cont
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #3146:
URL: https://github.com/apache/arrow-datafusion/pull/3146#discussion_r947665790
##########
datafusion/core/tests/sql/aggregates.rs:
##########
@@ -925,6 +925,54 @@ async fn csv_query_approx_percentile_cont_with_weight() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn csv_query_approx_percentile_cont_with_histogram_bins() -> Result<()> {
+ let ctx = SessionContext::new();
+ register_aggregate_csv(&ctx).await?;
+
+ // compare approx_percentile_cont and approx_percentile_cont_with_weight
+ let sql = "SELECT c1, approx_percentile_cont(c3, 0.95, 200) AS c3_p95 FROM aggregate_test_100 GROUP BY 1 ORDER BY 1";
+ let actual = execute_to_batches(&ctx, sql).await;
+ let expected = vec![
+ "+----+--------+",
Review Comment:
@alamb Thanks for your review 😊
this sql should have with the same result as
`SELECT c1, approx_percentile_cont(c3, 0.95) AS c3_p95 FROM aggregate_test_100 GROUP BY 1 ORDER BY 1`
in line 867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org