You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "tustvold (via GitHub)" <gi...@apache.org> on 2023/03/02 17:37:04 UTC
[GitHub] [arrow-datafusion] tustvold opened a new issue, #5453: parquet-test-utils Circular Crate Dependency
tustvold opened a new issue, #5453:
URL: https://github.com/apache/arrow-datafusion/issues/5453
**Describe the bug**
`parquet-test-utils` depends on `datafusion` which in turn has a dev-dependency on `parquet-test-utils`. This is not ideal as it results in repeated recompilation, when running tests.
**To Reproduce**
Steps to reproduce the behavior:
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #5453: parquet-test-utils Circular Crate Dependency
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5453:
URL: https://github.com/apache/arrow-datafusion/issues/5453#issuecomment-1452594987
@tustvold the reason is that the code to generate test parquet files is shared between the parquet benchmark and the parquet test
https://github.com/apache/arrow-datafusion/blob/49473d68bee42208aec0f5ad5be6326c1ed11661/benchmarks/src/bin/parquet_filter_pushdown.rs#L25
https://github.com/apache/arrow-datafusion/blob/49473d68bee42208aec0f5ad5be6326c1ed11661/datafusion/core/tests/parquet/filter_pushdown.rs#L39
The only thing I can think to do is to make a new crate (`datafusion-tests` for example) to mirror `datafusion-examples` that both depend `parquet-test-utils`
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #5453: parquet-test-utils Circular Crate Dependency
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5453:
URL: https://github.com/apache/arrow-datafusion/issues/5453#issuecomment-1458618746
Proposed fix https://github.com/apache/arrow-datafusion/pull/5506
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #5453: parquet-test-utils Circular Crate Dependency
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5453:
URL: https://github.com/apache/arrow-datafusion/issues/5453#issuecomment-1452576514
Maybe I could lift the stuff in datafusion dev that depends on parquet-test-utils into its own test. Let me check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #5453: parquet-test-utils Circular Crate Dependency
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #5453: parquet-test-utils Circular Crate Dependency
URL: https://github.com/apache/arrow-datafusion/issues/5453
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold commented on issue #5453: parquet-test-utils Circular Crate Dependency
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5453:
URL: https://github.com/apache/arrow-datafusion/issues/5453#issuecomment-1452599986
> code to generate test parquet files is shared between the parquet benchmark and the parquet test
This might be a naive question but why does this code depend on datafusion, the original access-log-gen didn't? Could we decouple it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org