Re: POJO to Record

[Ed J <ed...@protonmail.com.INVALID>]

Hi Matt,
Thanks for the tips.  Will look into those options this week.

Many of the fields are dynamic and our XML schema doesn't really qualify for XMLReader based on what I can glean from the docs, for example: "Records are expected in the second level of XML data, embedded in an enclosing root tag."

The other rationale for building an external custom parser to transform the XML is that it provides the flexibility to parse from other tools/apps in addition to NiFi.

-Ed

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, September 28, 2018 11:49 AM, Matt Burgess <ma...@apache.org> wrote:

> Moving users list to BCC and adding devs list, as this question is
> better suited for the devs.
>
> Ed,
>
> Does XMLReader not parse your XML? I guess you'd need to know the
> schema so if you don't (especially if it is dynamic) then you might be
> faced with a custom processor as you described.
>
> Take a look at DataTypeUtils, it has methods toRecord() and
> convertRecordFieldtoObject() which helps you convert from POJO
> <->Record. The JoltTransformRecord processor uses these, so you can
> have a look at that code for an example.
>
> Regards,
> Matt
>
> On Fri, Sep 28, 2018 at 11:28 AM Ed J edja@protonmail.com wrote:
>
> > Hi,
> > I'm ingesting some XML with an overly complicated structure and I've build a custom java DOM parser to wrangle it into a saner form. (Is that frowned upon - should I just use the built-in processors to wrangle it?)
> > So my question is, I've parsed the XML into a simple POJO, how do I get that pojo into the next processor as a 'Record'. My custom nifi processor's onTrigger looks like this:
> > MyParser parser = new MyParser();
> > MyPojo pojo = parser.parse(flowFileContents);
> > // TODO: convert pojo to record
> > Record myrecord = pojo.toRecord(); // how to do this?
> > session.transfer(myrecord, REL_SUCCESS)
> > I'm guessing I could convert the pojo to JSON, write that out to the next stage, and then use a JsonRecordReader to convert the JSON into a record. However, if I can go straight from pojo to record, that seems more efficient - just not sure how to go about it. Thanks.