You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2025/09/10 20:34:29 UTC
[PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
timsaucer opened a new pull request, #17509:
URL: https://github.com/apache/datafusion/pull/17509
## Which issue does this PR close?
None
## Rationale for this change
Reduces the duplicate dependencies.
We currently depend on bzip2 in two different ways. In attempting to reduce this, I needed to update `async-compression` which caused two different libraries to link to the system lzma library. This is not allowed in rust.
This PR updates avro-rs, but we cannot merge this PR until that crate merges https://github.com/apache/avro-rs/pull/284 and we remove the crates.io patch this PR contains.
## What changes are included in this PR?
Update avro-rs and switch from the unmaintained xz2 crate to liblzma.
## Are these changes tested?
Unit tests.
## Are there any user-facing changes?
None. This is simply a dependency update to a more recent crate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #17509:
URL: https://github.com/apache/datafusion/pull/17509#issuecomment-3280880292
It looks like upstream is on a 2 month release cadence, so we'll see if that lands first or the `arrow-avro`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #17509:
URL: https://github.com/apache/datafusion/pull/17509#issuecomment-3276553558
btw there is a bunch of work upstream to make an `arrow-avro` crate which would remove the dependency on `avro`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "findepi (via GitHub)" <gi...@apache.org>.
findepi commented on PR #17509:
URL: https://github.com/apache/datafusion/pull/17509#issuecomment-3279408081
is this "fixes https://github.com/apache/datafusion/issues/15342" ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #17509:
URL: https://github.com/apache/datafusion/pull/17509#issuecomment-3280868876
Thank you @findepi . I should have searched the issues to find it reported.
The upstream PR is merged, so once it's next release comes out I can update this PR and mark it ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on PR #17509:
URL: https://github.com/apache/datafusion/pull/17509#issuecomment-3543109158
@N-Boutaib @gabotechs ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Switch from xz2 to liblzma to reduce duplicate dependencies [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer merged PR #17509:
URL: https://github.com/apache/datafusion/pull/17509
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org