Re: LookupRecord using Attribute

["Thomas P." <th...@googlemail.com>]

finally I understood how it works!  ;)
I ended up adding 'authority' using JoltTransformation to use /authority
and now everything works fine
thanks Mike & Pierre for your input.


On 25 May 2018 at 15:37, Mike Thomsen <mi...@gmail.com> wrote:

> Based on your schema, when authority or code are not available in the
> flowfile content the record is invalid.
>
> On Fri, May 25, 2018 at 8:30 AM Thomas P. <th...@googlemail.com>
> wrote:
>
>> in the json example from my first post the field "authority" is set and
>> in this case I could use /authority
>> but when this field is not available in the content?
>>
>> {
>>   "code": "A1234",
>>
>>   "descriptions": [
>>     {
>>       "language": "EN",
>>       "description": "Nifi rocks!"
>>     }
>>   ]
>> }
>>
>> On 25 May 2018 at 14:13, Mike Thomsen <mi...@gmail.com> wrote:
>>
>>> /${authority} will yield a record path of /MyAuthority, but there is no
>>> "MyAuthority" in your flowfile.
>>>
>>> /authority, however, will do a lookup on the record and should find the
>>> value MyAuthority and set a map of { authority => MyAuthority } for the
>>> lookup service.
>>>
>>> On Fri, May 25, 2018 at 8:02 AM Thomas P. <thomaspuschmann@googlemail.
>>> com> wrote:
>>>
>>>> authority is an existing attribute on the flowfile. In the content
>>>> itself this field is not available thats why I want to use the attributes
>>>> value for querying.
>>>> [image: nifi_attribute.png]
>>>> ​
>>>>
>>>> On 25 May 2018 at 13:40, Pierre Villard <pi...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Is 'authority' an existing attribute in the incoming flow files or is
>>>>> it only an information available in the content of the flow file ?
>>>>>
>>>>> Pierre
>>>>>
>>>>> 2018-05-25 13:07 GMT+02:00 Thomas P. <th...@googlemail.com>:
>>>>>
>>>>>> Hello community,
>>>>>>
>>>>>> I am trying to query a MongoDB using LookupRecord which works fine
>>>>>> when I read the values directly from the json flow file.
>>>>>> But when I try to query the MongoService using an attribute, I don't
>>>>>> find the expected document.
>>>>>>
>>>>>> Here is my nifi 1.6 setup:
>>>>>>
>>>>>> json flow file:
>>>>>> {
>>>>>>   "code": "A1234",
>>>>>>   "authority": "MyAuthority",
>>>>>>   "descriptions": [
>>>>>>     {
>>>>>>       "language": "EN",
>>>>>>       "description": "Nifi rocks!"
>>>>>>     }
>>>>>>   ]
>>>>>> }
>>>>>>
>>>>>> avro schema:
>>>>>> {
>>>>>>   "type" : "record",
>>>>>>   "name" : "MyClass",
>>>>>>   "namespace" : "com.test.avro",
>>>>>>   "fields" : [ {
>>>>>>     "name" : "code",
>>>>>>     "type" : "string"
>>>>>>   }, {
>>>>>>     "name" : "authority",
>>>>>>     "type" : "string"
>>>>>>   }, {
>>>>>>     "name" : "descriptions",
>>>>>>     "type" : {
>>>>>>       "type" : "array",
>>>>>>       "items" : {
>>>>>>         "type" : "record",
>>>>>>         "name" : "descriptions",
>>>>>>         "fields" : [ {
>>>>>>           "name" : "language",
>>>>>>           "type" : "string"
>>>>>>         }, {
>>>>>>           "name" : "description",
>>>>>>           "type" : "string"
>>>>>>         } ]
>>>>>>       }
>>>>>>     }
>>>>>>   } ]
>>>>>> }
>>>>>>
>>>>>> two documents in mongo:
>>>>>> {
>>>>>>     "_id" : ObjectId("5b07db234fad446628118dde"),
>>>>>>     "code" : "A1234",
>>>>>>     "authority" : "MyAuthority",
>>>>>>     "descriptions" : [
>>>>>>         {
>>>>>>             "language" : "EN",
>>>>>>             "description" : "Nifi rocks!"
>>>>>>         }
>>>>>>     ]
>>>>>> }
>>>>>>
>>>>>> {
>>>>>>     "_id" : ObjectId("5b07db374fad446628118e10"),
>>>>>>     "code" : "A1234",
>>>>>>     "authority" : "AnotherAuthority",
>>>>>>     "descriptions" : [
>>>>>>         {
>>>>>>             "language" : "EN",
>>>>>>             "description" : "Nifi still rocks!"
>>>>>>         }
>>>>>>     ]
>>>>>> }
>>>>>>
>>>>>>
>>>>>> [image: nifi_lookuprecord_success.png]
>>>>>> ​
>>>>>>
>>>>>>
>>>>>> Now I try to query MongoDB using a value from an attribute but it
>>>>>> never matches.
>>>>>> The attribute 'authority' is set via UpdateAttribute and contains
>>>>>> value "MyAuthority".
>>>>>>
>>>>>> [image: nifi_lookuprecord_unmatched.png]
>>>>>> ​
>>>>>> ​
>>>>>> How can I query my document using attribute?
>>>>>>
>>>>>>
>>>>>> Thomas
>>>>>> ​
>>>>>>
>>>>>
>>>>>
>>>>
>>