You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "adamreeve (via GitHub)" <gi...@apache.org> on 2025/03/09 20:44:24 UTC

[I] Support Parquet key management tools [arrow-rs]

adamreeve opened a new issue, #7256:
URL: https://github.com/apache/arrow-rs/issues/7256

   Follow up task to https://github.com/apache/arrow-rs/pull/6637 and https://github.com/apache/arrow-rs/pull/7111, which add support for Parquet modular encryption.
   
   Parquet-java, C++ Parquet and PyArrow provide a higher level API on top of Parquet modular encryption to simplify integration with a key management system (KMS) and support encryption best-practices. For compatibility with these implementations and to simplify the use of Parquet encryption, arrow-rs should also support this API.
   
   There's a design document that provides more details on this at https://docs.google.com/document/d/1bEu903840yb95k9q2X-BlsYKuXoygE4VnMDl9xz_zhk/edit?tab=t.0#heading=h.aatf1oymdx11
   
   I have made some progress towards implementing this in a branch (https://github.com/adamreeve/arrow-rs/tree/kmt).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve closed issue #7256: Support Parquet key management tools
URL: https://github.com/apache/arrow-rs/issues/7256


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #7256:
URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2864294398

   `label_issue.py` automatically added labels {'parquet'} from #7286


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on issue #7256:
URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2773968545

   Yes this is not done yet. I should have a PR ready for it soon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on issue #7256:
URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2870541278

   We (G-Research) have now released a new [parquet-key-management crate](https://crates.io/crates/parquet-key-management) to implement this functionality.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "AudriusButkevicius (via GitHub)" <gi...@apache.org>.
AudriusButkevicius commented on issue #7256:
URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2773759627

   I guess this is not closed as needs to be weaved in for encryption?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Support Parquet key management tools [arrow-rs]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on issue #7256:
URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2819910204

   See https://github.com/apache/arrow-rs/pull/7387#issuecomment-2819908130, it was decided this should be a third party crate rather than part of arrow-rs for now, so I'm closing this as not planned. I'll update here with details once a crate is available.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org