You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by sudeep mishra <su...@gmail.com> on 2016/02/04 14:00:23 UTC
Send parameters while using getmongo processor
Hi,
I have following schema of records in MongoDB.
{
"_id" : ObjectId("56b1958a1ebecc0724588c39"),
"ContractNumber" : "ABC87gdtr53",
"DocumentType" : "TestDoc",
"FlowNr" : 3,
"TimeStamp" : "03/02/2016 05:51:09:023"
}
How can I query for a particular contract by 'ContractNumber' using
getmongo processor? I want to dynamically pass the value of ContractNumber
and get back the results.
Thanks & Regards,
Sudeep
Re: Send parameters while using getmongo processor
Posted by sudeep mishra <su...@gmail.com>.
Thanks Bryan for pointing me to these resources.
On Thu, Feb 4, 2016 at 10:15 PM, Bryan Bende <bb...@gmail.com> wrote:
> A good example is probably the recently added FetchElasticSearch
> processor...
>
> a) See the DOC_ID property with expressionLanguageSupported(true)
>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java#L87
>
> b) In onTrigger when getting the value for docId - final String docId =
> context.getProperty(DOC_ID).evaluateAttributeExpressions(flowFile).getValue();
>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java#L152
>
> This would assume that if you set the DOC_ID property to expression
> language like ${my.id} that something before this processor was setting "
> my.id" on each FlowFIle.
>
> This would depend on your use case, but for testing you could setup a flow
> with..
> GenerateFlowFile (change the scheduling to every few seconds) ->
> UpdateAttribute (set your id attribute) -> FetchMongo
>
> Let us know if that doesn't help.
>
> On Thu, Feb 4, 2016 at 11:28 AM, sudeep mishra <su...@gmail.com>
> wrote:
>
>> Hi Bryan,
>>
>> I am trying to create a processor on the lines of getmongo processor to
>> suit my needs.
>>
>> Can you please guide me how can I
>>
>> a) specify a property to support expression language?
>> b) assign the content or attribute of a flow file as the query parameter?
>>
>> Regards,
>>
>> Sudeep
>>
>> On Thu, Feb 4, 2016 at 9:11 PM, sudeep mishra <su...@gmail.com>
>> wrote:
>>
>>> Thanks for the feedback Bryan.
>>>
>>> Yes I need a processor similar to what you described.
>>>
>>> On Thu, Feb 4, 2016 at 7:38 PM, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>>> Hi Sudeep,
>>>>
>>>> From looking at the GetMongo processor, I don't think this can be done
>>>> today. That processor is meant to be a source processor that extracts data
>>>> from Mongo using a fixed query.
>>>> It seems to me like we would need a FetchMongo processor with a Query
>>>> field that supported expression language, and you could set it to
>>>> ContractNumber = ${flow.flile.contract.number}
>>>> Then incoming FlowFiles would have "flow.file.contract.number" as an
>>>> attribute and it would fetch documents matching that.
>>>>
>>>> I don't know that much about MongoDB, but does that sound like what you
>>>> need?
>>>>
>>>> -Bryan
>>>>
>>>>
>>>> On Thu, Feb 4, 2016 at 8:00 AM, sudeep mishra <sudeepshekharm@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have following schema of records in MongoDB.
>>>>>
>>>>> {
>>>>> "_id" : ObjectId("56b1958a1ebecc0724588c39"),
>>>>> "ContractNumber" : "ABC87gdtr53",
>>>>> "DocumentType" : "TestDoc",
>>>>> "FlowNr" : 3,
>>>>> "TimeStamp" : "03/02/2016 05:51:09:023"
>>>>> }
>>>>>
>>>>> How can I query for a particular contract by 'ContractNumber' using
>>>>> getmongo processor? I want to dynamically pass the value of ContractNumber
>>>>> and get back the results.
>>>>>
>>>>>
>>>>> Thanks & Regards,
>>>>>
>>>>> Sudeep
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>>
>>> Sudeep
>>>
>>
>>
>>
>
--
Thanks & Regards,
Sudeep
Re: Send parameters while using getmongo processor
Posted by Bryan Bende <bb...@gmail.com>.
A good example is probably the recently added FetchElasticSearch
processor...
a) See the DOC_ID property with expressionLanguageSupported(true)
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java#L87
b) In onTrigger when getting the value for docId - final String docId =
context.getProperty(DOC_ID).evaluateAttributeExpressions(flowFile).getValue();
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearch.java#L152
This would assume that if you set the DOC_ID property to expression
language like ${my.id} that something before this processor was setting "
my.id" on each FlowFIle.
This would depend on your use case, but for testing you could setup a flow
with..
GenerateFlowFile (change the scheduling to every few seconds) ->
UpdateAttribute (set your id attribute) -> FetchMongo
Let us know if that doesn't help.
On Thu, Feb 4, 2016 at 11:28 AM, sudeep mishra <su...@gmail.com>
wrote:
> Hi Bryan,
>
> I am trying to create a processor on the lines of getmongo processor to
> suit my needs.
>
> Can you please guide me how can I
>
> a) specify a property to support expression language?
> b) assign the content or attribute of a flow file as the query parameter?
>
> Regards,
>
> Sudeep
>
> On Thu, Feb 4, 2016 at 9:11 PM, sudeep mishra <su...@gmail.com>
> wrote:
>
>> Thanks for the feedback Bryan.
>>
>> Yes I need a processor similar to what you described.
>>
>> On Thu, Feb 4, 2016 at 7:38 PM, Bryan Bende <bb...@gmail.com> wrote:
>>
>>> Hi Sudeep,
>>>
>>> From looking at the GetMongo processor, I don't think this can be done
>>> today. That processor is meant to be a source processor that extracts data
>>> from Mongo using a fixed query.
>>> It seems to me like we would need a FetchMongo processor with a Query
>>> field that supported expression language, and you could set it to
>>> ContractNumber = ${flow.flile.contract.number}
>>> Then incoming FlowFiles would have "flow.file.contract.number" as an
>>> attribute and it would fetch documents matching that.
>>>
>>> I don't know that much about MongoDB, but does that sound like what you
>>> need?
>>>
>>> -Bryan
>>>
>>>
>>> On Thu, Feb 4, 2016 at 8:00 AM, sudeep mishra <su...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have following schema of records in MongoDB.
>>>>
>>>> {
>>>> "_id" : ObjectId("56b1958a1ebecc0724588c39"),
>>>> "ContractNumber" : "ABC87gdtr53",
>>>> "DocumentType" : "TestDoc",
>>>> "FlowNr" : 3,
>>>> "TimeStamp" : "03/02/2016 05:51:09:023"
>>>> }
>>>>
>>>> How can I query for a particular contract by 'ContractNumber' using
>>>> getmongo processor? I want to dynamically pass the value of ContractNumber
>>>> and get back the results.
>>>>
>>>>
>>>> Thanks & Regards,
>>>>
>>>> Sudeep
>>>>
>>>
>>>
>>
>>
>> --
>> Thanks & Regards,
>>
>> Sudeep
>>
>
>
>
Re: Send parameters while using getmongo processor
Posted by sudeep mishra <su...@gmail.com>.
Hi Bryan,
I am trying to create a processor on the lines of getmongo processor to
suit my needs.
Can you please guide me how can I
a) specify a property to support expression language?
b) assign the content or attribute of a flow file as the query parameter?
Regards,
Sudeep
On Thu, Feb 4, 2016 at 9:11 PM, sudeep mishra <su...@gmail.com>
wrote:
> Thanks for the feedback Bryan.
>
> Yes I need a processor similar to what you described.
>
> On Thu, Feb 4, 2016 at 7:38 PM, Bryan Bende <bb...@gmail.com> wrote:
>
>> Hi Sudeep,
>>
>> From looking at the GetMongo processor, I don't think this can be done
>> today. That processor is meant to be a source processor that extracts data
>> from Mongo using a fixed query.
>> It seems to me like we would need a FetchMongo processor with a Query
>> field that supported expression language, and you could set it to
>> ContractNumber = ${flow.flile.contract.number}
>> Then incoming FlowFiles would have "flow.file.contract.number" as an
>> attribute and it would fetch documents matching that.
>>
>> I don't know that much about MongoDB, but does that sound like what you
>> need?
>>
>> -Bryan
>>
>>
>> On Thu, Feb 4, 2016 at 8:00 AM, sudeep mishra <su...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have following schema of records in MongoDB.
>>>
>>> {
>>> "_id" : ObjectId("56b1958a1ebecc0724588c39"),
>>> "ContractNumber" : "ABC87gdtr53",
>>> "DocumentType" : "TestDoc",
>>> "FlowNr" : 3,
>>> "TimeStamp" : "03/02/2016 05:51:09:023"
>>> }
>>>
>>> How can I query for a particular contract by 'ContractNumber' using
>>> getmongo processor? I want to dynamically pass the value of ContractNumber
>>> and get back the results.
>>>
>>>
>>> Thanks & Regards,
>>>
>>> Sudeep
>>>
>>
>>
>
>
> --
> Thanks & Regards,
>
> Sudeep
>
Re: Send parameters while using getmongo processor
Posted by sudeep mishra <su...@gmail.com>.
Thanks for the feedback Bryan.
Yes I need a processor similar to what you described.
On Thu, Feb 4, 2016 at 7:38 PM, Bryan Bende <bb...@gmail.com> wrote:
> Hi Sudeep,
>
> From looking at the GetMongo processor, I don't think this can be done
> today. That processor is meant to be a source processor that extracts data
> from Mongo using a fixed query.
> It seems to me like we would need a FetchMongo processor with a Query
> field that supported expression language, and you could set it to
> ContractNumber = ${flow.flile.contract.number}
> Then incoming FlowFiles would have "flow.file.contract.number" as an
> attribute and it would fetch documents matching that.
>
> I don't know that much about MongoDB, but does that sound like what you
> need?
>
> -Bryan
>
>
> On Thu, Feb 4, 2016 at 8:00 AM, sudeep mishra <su...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I have following schema of records in MongoDB.
>>
>> {
>> "_id" : ObjectId("56b1958a1ebecc0724588c39"),
>> "ContractNumber" : "ABC87gdtr53",
>> "DocumentType" : "TestDoc",
>> "FlowNr" : 3,
>> "TimeStamp" : "03/02/2016 05:51:09:023"
>> }
>>
>> How can I query for a particular contract by 'ContractNumber' using
>> getmongo processor? I want to dynamically pass the value of ContractNumber
>> and get back the results.
>>
>>
>> Thanks & Regards,
>>
>> Sudeep
>>
>
>
--
Thanks & Regards,
Sudeep
Re: Send parameters while using getmongo processor
Posted by Bryan Bende <bb...@gmail.com>.
Hi Sudeep,
>From looking at the GetMongo processor, I don't think this can be done
today. That processor is meant to be a source processor that extracts data
from Mongo using a fixed query.
It seems to me like we would need a FetchMongo processor with a Query field
that supported expression language, and you could set it to ContractNumber
= ${flow.flile.contract.number}
Then incoming FlowFiles would have "flow.file.contract.number" as an
attribute and it would fetch documents matching that.
I don't know that much about MongoDB, but does that sound like what you
need?
-Bryan
On Thu, Feb 4, 2016 at 8:00 AM, sudeep mishra <su...@gmail.com>
wrote:
> Hi,
>
> I have following schema of records in MongoDB.
>
> {
> "_id" : ObjectId("56b1958a1ebecc0724588c39"),
> "ContractNumber" : "ABC87gdtr53",
> "DocumentType" : "TestDoc",
> "FlowNr" : 3,
> "TimeStamp" : "03/02/2016 05:51:09:023"
> }
>
> How can I query for a particular contract by 'ContractNumber' using
> getmongo processor? I want to dynamically pass the value of ContractNumber
> and get back the results.
>
>
> Thanks & Regards,
>
> Sudeep
>