Re: GetMongo - Pass-on Initial FlowFile?

[Ryan Hendrickson <ry...@gmail.com>]

I was able to accomplish this by:
                 /---success---> *ReplaceText* (Prepend JSON Key)
---success-->  \
                /
                                     \
*GetMongo*
                                -------> *Merge Content* (Combine on
Correlation Attribute Name, Binary Concat)
                \
                                    /
                 \---original---> *ReplaceText*  (Prepend JSON Key)
---success--> /

Not the cleanest, but it gets it done.

Ryan

On Fri, Nov 30, 2018 at 3:43 PM Ryan Hendrickson <
ryan.andrew.hendrickson@gmail.com> wrote:

> I'm using the InferAvroSchema to generate the schema, such that I don't
> need to know about the JSON structure in advance.
>
> Another question.. Can the LookupRecord lookup based on the Mongo _id
> field?
>
> Using the GetMongo processor, I had the following as my query:  { _id:
> ObjectId("${uuid}") }
>
> Is there an equivalent way to do that with LookupRecord and the
> MongoDBLookupService?
>
> Thanks,
> Ryan
>
> On Fri, Nov 30, 2018 at 11:19 AM Ryan Hendrickson <
> ryan.andrew.hendrickson@gmail.com> wrote:
>
>> My incoming FlowFIle is a valid JSON Object.  The key names could be
>> anything, they're not defined.
>>
>> I need to add a top level object to every JSON Object, based on the
>> result from MongoDB.
>>
>> Is that possible with the JsonTreeReader/Schema, or do I need to know
>> what the fields are?
>>
>> Input:
>> { "key1" : "value", "key2" : value" }
>>
>> Output:
>> { "key1" : "value", "key2" : value", "mongorecord" : { "mongokey1" :
>> "mongovalue", "monogoKey2" : "mongovalue" } }
>>
>> Thanks,
>> Ryan
>>
>>
>> On Fri, Nov 30, 2018 at 10:54 AM Mike Thomsen <mi...@gmail.com>
>> wrote:
>>
>>> For the record side of things, you just need to create a schema that
>>> includes your existing JSON fields and a new branch that will have the
>>> enriched fields in it.
>>>
>>> On Fri, Nov 30, 2018 at 10:39 AM Ryan Hendrickson <
>>> ryan.andrew.hendrickson@gmail.com> wrote:
>>>
>>>> Hi Otto and Mike,
>>>>    The LookupRecord does look fruitful, although I don't have a defined
>>>> schema for the JsonTreeReader.  Is there a way to just keep it generic?   I
>>>> know I have valid JSON already, I just want to add the result of the
>>>> MongoQuery to a specific Json Path in the FlowFile.
>>>>
>>>>    The LookupAttribute also seems like it could be another avenue, but
>>>> it doesn't have the MongoDBLookupService in the list of Compatible
>>>> Controller Services.
>>>>
>>>> Ryan
>>>>
>>>> On Fri, Nov 30, 2018 at 10:24 AM Mike Thomsen <mi...@gmail.com>
>>>> wrote:
>>>>
>>>>> LookupAttribute + the MongoDBLookupService should be able to do that.
>>>>>
>>>>> On Thu, Nov 29, 2018 at 8:05 PM Otto Fowler <ot...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Sounds like you want to look at enrichment with the LookupRecord
>>>>>> processors and Mongo.
>>>>>>
>>>>>> https://community.hortonworks.com/articles/146198/data-flow-enrichment-with-nifi-part-3-lookuprecord.html
>>>>>>
>>>>>>
>>>>>> On November 29, 2018 at 17:12:38, Ryan Hendrickson (
>>>>>> ryan.andrew.hendrickson@gmail.com) wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>    I'm curious if the GetMongo processor can allow you can pass in a
>>>>>> FlowFile Document and enrich the original FlowFile document with the
>>>>>> results from the Query Result - or more generally store the result as a
>>>>>> NiFi Attribute, instead of replacing the content in the FlowFile.
>>>>>>
>>>>>>    I really want to "enrich" my results with data from Mongo, not
>>>>>> replace my FlowFile with the database record that matches.
>>>>>>
>>>>>>    I had thought I had figured out a way to do this in the past, but
>>>>>> can't seem to replicate it, so I figured I'd ask the community.
>>>>>>
>>>>>> Thanks,
>>>>>> Ryan
>>>>>>
>>>>>>