Re: Azure Blob IO for Apache Beam

[Stephen Sisk <si...@google.com.INVALID>]

If you're interested in what's involved in building a read transform for
Azure Storage Blob, folks here on the dev@ alias are happy to help. We
could help outline in a bit more detail what'd be involved in doing the
HDFS->WASB hookup that Davor mentioned in his stackoverflow answer.

S

On Fri, Dec 30, 2016 at 8:53 PM Davor Bonaci <da...@apache.org> wrote:

> Hi Saulo,
> I've responded to your Stack Overflow question, but I can give a few more
> comments here.
>
> Pei (cc'd) is working on this. See this JIRA issue [1], its sub-tasks, and
> several design documents [2, 3].
>
> It would be awesome to also have native Azure Storage Blob support in Beam
> that builds on top of current work -- we'd love that contribution!
>
> Thanks,
> Davor
>
> [1] https://issues.apache.org/jira/browse/BEAM-59
> [2]
>
> https://docs.google.com/document/d/11TdPyZ9_zmjokhNWM3Id-XJsVG3qel2lhdKTknmZ_7M
> [3]
>
> https://docs.google.com/document/d/1-7vo9nLRsEEzDGnb562PuL4q9mUiq_ZVpCAiyyJw8p8/edit#heading=h.p3gc3colc2cs
>
> On Fri, Dec 30, 2016 at 5:42 PM, Saulo Ricci <in...@gmail.com> wrote:
>
> > Hi,
> >
> > I had posted this question
> > <http://stackoverflow.com/questions/41386717/azure-blob-
> > support-in-apache-beam>
> > @
> > stack overflow. Basically I'm trying to run Apache Beam on a spark
> cluster
> > hosted in a MS Azure environment. It seems Apache Beam doesn't have
> support
> > to Azure Blobs IO, right? Should be an alternative solution the
> > implementation of an Azure Blob IO for this case?
> >
> > Best
> > Saulo
> > --
> > Saulo
> >
>