You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by te...@gmail.com, te...@gmail.com on 2019/04/11 15:32:36 UTC

Accessing CSV headers

Hi 

I am new to Apache Beam, and have been trying to access the CSV headers from a csv file I referenced as input.

Is there a way to collect the headers in a CSV using Apache Beam so that the following records can be processed and compared as columns and rows to another csv file?

Please advise.

Thanks in advance.

Re: Accessing CSV headers

Posted by te...@gmail.com, te...@gmail.com.
I am using the latest SDK (Apache Beam 2.11.0)

I have been trying to use a variety of code online, however I am aware that com.google.cloud.dataflow.sdk has deprecated but updating this library did not enable the code I found to be recognised.

Deprecated:
https://github.com/obradovicluka/dataflow-playground

My code:
https://github.com/tejanhu/PoCTool-Apache-Beam

I would appreciate you sharing a code snippet or any advice.

Thanks
On 2019/04/11 21:19:32, Pablo Estrada <pa...@google.com> wrote: 
> What SDK are you using?
> 
> There are FileIO transforms that should help you do this. We can help you
> with a snippet if you share the SDK you're using.
> 
> On Thu, Apr 11, 2019 at 2:16 PM tejanahmedhu@gmail.com <
> tejanahmedhu@gmail.com> wrote:
> 
> > Hi
> >
> > I am new to Apache Beam, and have been trying to access the CSV headers
> > from a csv file I referenced as input.
> >
> > Is there a way to collect the headers in a CSV using Apache Beam so that
> > the following records can be processed and compared as columns and rows to
> > another csv file?
> >
> > Please advise.
> >
> > Thanks in advance.
> >
>

Re: Accessing CSV headers

Posted by Pablo Estrada <pa...@google.com>.
What SDK are you using?

There are FileIO transforms that should help you do this. We can help you
with a snippet if you share the SDK you're using.

On Thu, Apr 11, 2019 at 2:16 PM tejanahmedhu@gmail.com <
tejanahmedhu@gmail.com> wrote:

> Hi
>
> I am new to Apache Beam, and have been trying to access the CSV headers
> from a csv file I referenced as input.
>
> Is there a way to collect the headers in a CSV using Apache Beam so that
> the following records can be processed and compared as columns and rows to
> another csv file?
>
> Please advise.
>
> Thanks in advance.
>