Mapper

[Anil Rai <an...@gmail.com>]

Hi,
I was wondering if there is an UI for XML mapping that is already available
or some is working on?
We have transformXML. But the problem is that we need to generate the XSLT
outside and then feed it in. It would be nice if we have a mapper that a
developer/business user can use to map the source to the target graphically.
Also today the DB connector out comes out in avero format. We are forced to
add additional transforms like avero to json or avero to xml to proceed. It
would be nice if we could configure the DB processor to output the file in
the format that we need/
Any thoughts?

Thanks
Anil

Re: Mapper

[Anil Rai <an...@gmail.com>]

Thank you Aldrin. Will take a look.

Regards
Anil


On Fri, Jun 2, 2017 at 4:44 PM, Aldrin Piri <al...@gmail.com> wrote:

> Hi Anil,
>
> The Quick Start [1], contributor's guide [2], and developer's guide [3]
> have a lot of good information.  We are of course happy to answer any
> questions across the mailing list should you hit any parts that are unclear
> or just have general questions.
>
> Looking forward to your contributions!
>
> [1] http://nifi.apache.org/quickstart.html
> [2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
> [3] http://nifi.apache.org/developer-guide.html
>
> On Fri, Jun 2, 2017 at 4:39 PM, Anil Rai <an...@gmail.com> wrote:
>
> > To clarify...to start contributing...is there a guide for naming
> > convention, github, unit testing and things like that......
> >
> >
> > On Fri, Jun 2, 2017 at 4:38 PM, Anil Rai <an...@gmail.com> wrote:
> >
> > > Thanks Mark for the detailed explanation and suggestions. Will take a
> > > look. Yes, I was referring to ExecuteSQL.
> > > To start contributing, is there any documentation or guide for
> starters?
> > >
> > > Cheers
> > > Anil
> > >
> > >
> > > On Fri, Jun 2, 2017 at 8:24 AM, Mark Payne <ma...@hotmail.com>
> wrote:
> > >
> > >> Hi Anil,
> > >>
> > >> Sorry that it took so long to respond. This email got lost in my
> inbox,
> > >> it seems.
> > >>
> > >> Regarding a mapper UI: there is not currently anything like that in
> > NiFi.
> > >> I do not
> > >> know of anyone building such a thing either. However, it would be very
> > >> helpful,
> > >> for sure, if it is something that you are interested in contributing.
> > >>
> > >> Regarding ExecuteSQL (I'm assuming that this is the processor that
> > you're
> > >> referring to),
> > >> I agree that this can be better. In the latest version, 1.2.0, we
> > >> introduced the concept of Record
> > >> Readers and Record Writers, and I think that this Processor needs to
> be
> > >> updated to make use of
> > >> those. This would allow you to choose JSON, Avro, etc. as the output
> > >> format. At the moment, there
> > >> is no Record Reader or Writer for XML but I think that we will need to
> > be
> > >> building one in the near
> > >> future (again, another great place for you to jump in if you are
> > >> interested in contributing). I have
> > >> seen someone make use of the scripted Record Writer in order to write
> > >> XML, as well, as Groovy
> > >> has a lot of built-in niceties for reading and writing XML, so that
> > would
> > >> be another option.
> > >>
> > >> Thanks!
> > >> -Mark
> > >>
> > >>
> > >> > On May 19, 2017, at 12:14 PM, Anil Rai <an...@gmail.com>
> wrote:
> > >> >
> > >> > Hi,
> > >> > I was wondering if there is an UI for XML mapping that is already
> > >> available
> > >> > or some is working on?
> > >> > We have transformXML. But the problem is that we need to generate
> the
> > >> XSLT
> > >> > outside and then feed it in. It would be nice if we have a mapper
> > that a
> > >> > developer/business user can use to map the source to the target
> > >> graphically.
> > >> > Also today the DB connector out comes out in avero format. We are
> > >> forced to
> > >> > add additional transforms like avero to json or avero to xml to
> > >> proceed. It
> > >> > would be nice if we could configure the DB processor to output the
> > file
> > >> in
> > >> > the format that we need/
> > >> > Any thoughts?
> > >> >
> > >> > Thanks
> > >> > Anil
> > >>
> > >>
> > >
> >
>

Re: Mapper

[Aldrin Piri <al...@gmail.com>]

Hi Anil,

The Quick Start [1], contributor's guide [2], and developer's guide [3]
have a lot of good information.  We are of course happy to answer any
questions across the mailing list should you hit any parts that are unclear
or just have general questions.

Looking forward to your contributions!

[1] http://nifi.apache.org/quickstart.html
[2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
[3] http://nifi.apache.org/developer-guide.html

On Fri, Jun 2, 2017 at 4:39 PM, Anil Rai <an...@gmail.com> wrote:

> To clarify...to start contributing...is there a guide for naming
> convention, github, unit testing and things like that......
>
>
> On Fri, Jun 2, 2017 at 4:38 PM, Anil Rai <an...@gmail.com> wrote:
>
> > Thanks Mark for the detailed explanation and suggestions. Will take a
> > look. Yes, I was referring to ExecuteSQL.
> > To start contributing, is there any documentation or guide for starters?
> >
> > Cheers
> > Anil
> >
> >
> > On Fri, Jun 2, 2017 at 8:24 AM, Mark Payne <ma...@hotmail.com> wrote:
> >
> >> Hi Anil,
> >>
> >> Sorry that it took so long to respond. This email got lost in my inbox,
> >> it seems.
> >>
> >> Regarding a mapper UI: there is not currently anything like that in
> NiFi.
> >> I do not
> >> know of anyone building such a thing either. However, it would be very
> >> helpful,
> >> for sure, if it is something that you are interested in contributing.
> >>
> >> Regarding ExecuteSQL (I'm assuming that this is the processor that
> you're
> >> referring to),
> >> I agree that this can be better. In the latest version, 1.2.0, we
> >> introduced the concept of Record
> >> Readers and Record Writers, and I think that this Processor needs to be
> >> updated to make use of
> >> those. This would allow you to choose JSON, Avro, etc. as the output
> >> format. At the moment, there
> >> is no Record Reader or Writer for XML but I think that we will need to
> be
> >> building one in the near
> >> future (again, another great place for you to jump in if you are
> >> interested in contributing). I have
> >> seen someone make use of the scripted Record Writer in order to write
> >> XML, as well, as Groovy
> >> has a lot of built-in niceties for reading and writing XML, so that
> would
> >> be another option.
> >>
> >> Thanks!
> >> -Mark
> >>
> >>
> >> > On May 19, 2017, at 12:14 PM, Anil Rai <an...@gmail.com> wrote:
> >> >
> >> > Hi,
> >> > I was wondering if there is an UI for XML mapping that is already
> >> available
> >> > or some is working on?
> >> > We have transformXML. But the problem is that we need to generate the
> >> XSLT
> >> > outside and then feed it in. It would be nice if we have a mapper
> that a
> >> > developer/business user can use to map the source to the target
> >> graphically.
> >> > Also today the DB connector out comes out in avero format. We are
> >> forced to
> >> > add additional transforms like avero to json or avero to xml to
> >> proceed. It
> >> > would be nice if we could configure the DB processor to output the
> file
> >> in
> >> > the format that we need/
> >> > Any thoughts?
> >> >
> >> > Thanks
> >> > Anil
> >>
> >>
> >
>

Re: Mapper

[Anil Rai <an...@gmail.com>]

To clarify...to start contributing...is there a guide for naming
convention, github, unit testing and things like that......


On Fri, Jun 2, 2017 at 4:38 PM, Anil Rai <an...@gmail.com> wrote:

> Thanks Mark for the detailed explanation and suggestions. Will take a
> look. Yes, I was referring to ExecuteSQL.
> To start contributing, is there any documentation or guide for starters?
>
> Cheers
> Anil
>
>
> On Fri, Jun 2, 2017 at 8:24 AM, Mark Payne <ma...@hotmail.com> wrote:
>
>> Hi Anil,
>>
>> Sorry that it took so long to respond. This email got lost in my inbox,
>> it seems.
>>
>> Regarding a mapper UI: there is not currently anything like that in NiFi.
>> I do not
>> know of anyone building such a thing either. However, it would be very
>> helpful,
>> for sure, if it is something that you are interested in contributing.
>>
>> Regarding ExecuteSQL (I'm assuming that this is the processor that you're
>> referring to),
>> I agree that this can be better. In the latest version, 1.2.0, we
>> introduced the concept of Record
>> Readers and Record Writers, and I think that this Processor needs to be
>> updated to make use of
>> those. This would allow you to choose JSON, Avro, etc. as the output
>> format. At the moment, there
>> is no Record Reader or Writer for XML but I think that we will need to be
>> building one in the near
>> future (again, another great place for you to jump in if you are
>> interested in contributing). I have
>> seen someone make use of the scripted Record Writer in order to write
>> XML, as well, as Groovy
>> has a lot of built-in niceties for reading and writing XML, so that would
>> be another option.
>>
>> Thanks!
>> -Mark
>>
>>
>> > On May 19, 2017, at 12:14 PM, Anil Rai <an...@gmail.com> wrote:
>> >
>> > Hi,
>> > I was wondering if there is an UI for XML mapping that is already
>> available
>> > or some is working on?
>> > We have transformXML. But the problem is that we need to generate the
>> XSLT
>> > outside and then feed it in. It would be nice if we have a mapper that a
>> > developer/business user can use to map the source to the target
>> graphically.
>> > Also today the DB connector out comes out in avero format. We are
>> forced to
>> > add additional transforms like avero to json or avero to xml to
>> proceed. It
>> > would be nice if we could configure the DB processor to output the file
>> in
>> > the format that we need/
>> > Any thoughts?
>> >
>> > Thanks
>> > Anil
>>
>>
>

Re: Mapper

[Anil Rai <an...@gmail.com>]

Thanks Mark for the detailed explanation and suggestions. Will take a look.
Yes, I was referring to ExecuteSQL.
To start contributing, is there any documentation or guide for starters?

Cheers
Anil


On Fri, Jun 2, 2017 at 8:24 AM, Mark Payne <ma...@hotmail.com> wrote:

> Hi Anil,
>
> Sorry that it took so long to respond. This email got lost in my inbox, it
> seems.
>
> Regarding a mapper UI: there is not currently anything like that in NiFi.
> I do not
> know of anyone building such a thing either. However, it would be very
> helpful,
> for sure, if it is something that you are interested in contributing.
>
> Regarding ExecuteSQL (I'm assuming that this is the processor that you're
> referring to),
> I agree that this can be better. In the latest version, 1.2.0, we
> introduced the concept of Record
> Readers and Record Writers, and I think that this Processor needs to be
> updated to make use of
> those. This would allow you to choose JSON, Avro, etc. as the output
> format. At the moment, there
> is no Record Reader or Writer for XML but I think that we will need to be
> building one in the near
> future (again, another great place for you to jump in if you are
> interested in contributing). I have
> seen someone make use of the scripted Record Writer in order to write XML,
> as well, as Groovy
> has a lot of built-in niceties for reading and writing XML, so that would
> be another option.
>
> Thanks!
> -Mark
>
>
> > On May 19, 2017, at 12:14 PM, Anil Rai <an...@gmail.com> wrote:
> >
> > Hi,
> > I was wondering if there is an UI for XML mapping that is already
> available
> > or some is working on?
> > We have transformXML. But the problem is that we need to generate the
> XSLT
> > outside and then feed it in. It would be nice if we have a mapper that a
> > developer/business user can use to map the source to the target
> graphically.
> > Also today the DB connector out comes out in avero format. We are forced
> to
> > add additional transforms like avero to json or avero to xml to proceed.
> It
> > would be nice if we could configure the DB processor to output the file
> in
> > the format that we need/
> > Any thoughts?
> >
> > Thanks
> > Anil
>
>

Re: Mapper

[Mark Payne <ma...@hotmail.com>]

Hi Anil,

Sorry that it took so long to respond. This email got lost in my inbox, it seems.

Regarding a mapper UI: there is not currently anything like that in NiFi. I do not
know of anyone building such a thing either. However, it would be very helpful,
for sure, if it is something that you are interested in contributing.

Regarding ExecuteSQL (I'm assuming that this is the processor that you're referring to),
I agree that this can be better. In the latest version, 1.2.0, we introduced the concept of Record
Readers and Record Writers, and I think that this Processor needs to be updated to make use of
those. This would allow you to choose JSON, Avro, etc. as the output format. At the moment, there
is no Record Reader or Writer for XML but I think that we will need to be building one in the near
future (again, another great place for you to jump in if you are interested in contributing). I have
seen someone make use of the scripted Record Writer in order to write XML, as well, as Groovy
has a lot of built-in niceties for reading and writing XML, so that would be another option.

Thanks!
-Mark


> On May 19, 2017, at 12:14 PM, Anil Rai <an...@gmail.com> wrote:
> 
> Hi,
> I was wondering if there is an UI for XML mapping that is already available
> or some is working on?
> We have transformXML. But the problem is that we need to generate the XSLT
> outside and then feed it in. It would be nice if we have a mapper that a
> developer/business user can use to map the source to the target graphically.
> Also today the DB connector out comes out in avero format. We are forced to
> add additional transforms like avero to json or avero to xml to proceed. It
> would be nice if we could configure the DB processor to output the file in
> the format that we need/
> Any thoughts?
> 
> Thanks
> Anil