You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/26 23:05:33 UTC

[GitHub] [arrow-ballista] avantgardnerio opened a new pull request, #457: Bg custom serde

avantgardnerio opened a new pull request, #457:
URL: https://github.com/apache/arrow-ballista/pull/457

   # Which issue does this PR close?
   
   Closes #456.
   
   Described in issue 
   
   # What changes are included in this PR?
   
   1. New `delta` feature that allows registration of delta tables
   
   # Are there any user-facing changes?
   
   They can use deltalake formatted tables.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Support Deltalake [arrow-ballista]

Posted by "rtyler (via GitHub)" <gi...@apache.org>.

rtyler commented on PR #457:
URL: https://github.com/apache/arrow-ballista/pull/457#issuecomment-1771305622

   @avantgardnerio Yeah I figured as such. As a delta-rs maintainer I can assure you we're having more regular releases :smile: 
   
   In some experimentation from last night it looks like I have to make sure that the scheduler and all executors are going to have the same credentials in environments. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Support Deltalake [arrow-ballista]

Posted by "rtyler (via GitHub)" <gi...@apache.org>.

rtyler commented on PR #457:
URL: https://github.com/apache/arrow-ballista/pull/457#issuecomment-1769867816

   :wave: I stumbled into this pull request and I'm curious what it takes to move this pul l request forward.
   
   Also I'm not terribly familiar, yet, with ballista configuration, how would things like AWS access keys be accessed or provided to the deltalake crate on the executors when they access delta tables.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Support Deltalake [arrow-ballista]

Posted by "avantgardnerio (via GitHub)" <gi...@apache.org>.

avantgardnerio commented on PR #457:
URL: https://github.com/apache/arrow-ballista/pull/457#issuecomment-1771177294

   > what it takes to move this pul l request forward
   
   It was working internally for my old company. It basically just "died on the vine", because it was waiting on a DeltaLake release I think. Everything should basically just work.
   
   > AWS access keys be accessed or provided to the deltalake
   
   The environment variables are defined in the `object-store` crate, and if you set them it magically just works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Support Deltalake [datafusion-ballista]

Posted by "Dandandan (via GitHub)" <gi...@apache.org>.

Dandandan closed pull request #457: Support Deltalake
URL: https://github.com/apache/datafusion-ballista/pull/457


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org