You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2024/05/21 11:07:37 UTC
[PR] tsaucer/run TPC-H examples in CI [datafusion-python]
timsaucer opened a new pull request, #711:
URL: https://github.com/apache/datafusion-python/pull/711
# Which issue does this PR close?
Closes #696
# Rationale for this change
This PR sets up a work flow to generate TPH-C 1Gb data set in CI, runs the 22 examples, and compares their results to the
known answer file. By adding this PR we improve the robustness of our test suite.
# What changes are included in this PR?
This PR adds the following changes:
- Adds CI workflow to generate TPH-C data set and answer file. The results are cached from run to run.
- Adds pytest file to run all 22 examples and compare them to the answer file
- Exposes substring operation that was missing. Sometime between 36.0.0 and 38.0.0 substr was performing the operation instead.
# Are there any user-facing changes?
`substring` function exposed in python.
# Additional context
This PR replaces https://github.com/apache/datafusion-python/pull/710 which contains a lot of intermediate testing steps. This MR should be cleaner to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] tsaucer/run TPC-H examples in CI [datafusion-python]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #711:
URL: https://github.com/apache/datafusion-python/pull/711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] tsaucer/run TPC-H examples in CI [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #711:
URL: https://github.com/apache/datafusion-python/pull/711#issuecomment-2122437247
@Michael-J-Ward It looks like we have a _potential_ regression between 37.1.0 and 38.0.0. Namely `substr` on 37.1.0 would accept a start and length, the parameters that should apply to `substring`. I was using `substr` incorrectly but it worked by accident on 37.1.0 because `substr` and `substring` were calling the same underlying function. In 38.0.0 this is updated and `substr` now fails if you pass it two parameters.
This PR includes exposing `substring` which was missing on the python side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org