You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Yves HAMEL <yh...@macif.fr> on 2018/03/09 14:44:19 UTC
PutMongo processor : specifying nested field and multiple field on Update
Query Key properties
Hello everyone,
I am using PutMongo processor to update documents in a mongoDB collection.
I would like to specify nested field and multiple key in the update Key.
For exemple :
- my mongodb documents are {"nom":"HAMEL", "prenom":"Yves",
"ids":{"idSoc":"1234", "idInterne":"788"}}
- I would'd like to set update query key to something like {"ids.idSoc" ,
"nom"}
- this would means that I update document with ids.idSoc and nom equals to
my mew document.
Those function are available with the native mongodb
db.collection.updateOne() method.
Il would like to know if this feture will be implemented in PutMongo
processor ?
How can I request the community to implement this feature ?
If necessary I could propose the code to the community.
Thanks
Best regards
(Embedded image moved to file: pic11173.jpg)
Yves HAMEL
Direction Digital & Systèmes d’Information Groupe
LM_DATA
MACIF - 2 et 4, rue Pied de Fond - 79037 Niort cedex 9
Tél. : +33 (0)5 49 09 36 06
Email : mon.email@macif.fr / Pré Doyen 2 – bureau 999
www.macif.fr - Appli présente sur Google Play Store & Apple Store
(Embedded image moved to file: pic10466.jpg)
Re: PutMongo processor : specifying nested field and multiple field
on Update Query Key properties
Posted by Mike Thomsen <mi...@gmail.com>.
Pierre,
I should have a PR tonight or tomorrow that should provide a reasonable
work around for Yves.
Thanks,
Mike
On Fri, Mar 16, 2018 at 10:08 AM, Mike Thomsen <mi...@gmail.com>
wrote:
> Yves/Pierre,
>
> I agree with Pierre. I think starting in 1.7 we're going to need to
> rethink this and possibly do some breaking changes to really make this work
> right. My $0.02 is that PutMongo should allow updates to come in the
> following use cases:
>
> 1. Read update keys using attributes prefixed with mongo.update and take
> the whole flowfile body as the content to use for the body of the update.
> Ex:
> mongo.update.username = Mike
> mongo.update.department = Engineering
> flowfile body = { "$set": { "email": "test@test.com" }}
>
> This method would not support nested fields unless I can find a good
> way to do that with the Java API.
>
> 2. Read a query from a particular field of the body using a "query field"
> attribute.
>
> 3. Read a query as a JSON string from an attribute named
> mongo.update.query.
>
> For PutMongoRecord, updates should be avoided because Avro doesn't support
> $ as the starting character of a field name (not sure if it can be part of
> the name period). So PutMongoRecord really should be used for bulk inserts.
>
>
>
> On Fri, Mar 16, 2018 at 8:27 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> Hi Yves,
>>
>> Sorry for the late answer. I believe you're correct: looking at the code,
>> it does not seem possible to update a document based on multiple IDs.
>> Feel free to create a JIRA here https://issues.apache.org/jira
>> /browse/NIFI
>> If you're up to provide the necessary code in a PR for that, that would
>> be awesome.
>>
>> Let us know if you need some help.
>>
>> Thanks
>>
>>
>> 2018-03-09 15:44 GMT+01:00 Yves HAMEL <yh...@macif.fr>:
>>
>>>
>>> Hello everyone,
>>> I am using PutMongo processor to update documents in a mongoDB
>>> collection.
>>> I would like to specify nested field and multiple key in the update Key.
>>>
>>> For exemple :
>>> - my mongodb documents are {"nom":"HAMEL", "prenom":"Yves",
>>> "ids":{"idSoc":"1234", "idInterne":"788"}}
>>> - I would'd like to set update query key to something like {"ids.idSoc" ,
>>> "nom"}
>>> - this would means that I update document with ids.idSoc and nom equals
>>> to
>>> my mew document.
>>>
>>> Those function are available with the native mongodb
>>> db.collection.updateOne() method.
>>>
>>> Il would like to know if this feture will be implemented in PutMongo
>>> processor ?
>>> How can I request the community to implement this feature ?
>>> If necessary I could propose the code to the community.
>>>
>>> Thanks
>>> Best regards
>>>
>>>
>>> (Embedded image moved to file: pic11173.jpg)
>>>
>>> Yves HAMEL
>>> Direction Digital & Systèmes d’Information Groupe
>>> LM_DATA
>>>
>>> MACIF - 2 et 4, rue Pied de Fond
>>> <https://maps.google.com/?q=4,+rue+Pied+de+Fond&entry=gmail&source=g> -
>>> 79037 Niort cedex 9
>>> Tél. : +33 (0)5 49 09 36 06
>>> Email : mon.email@macif.fr / Pré Doyen 2 – bureau 999
>>>
>>> www.macif.fr - Appli présente sur Google Play Store & Apple Store
>>> (Embedded image moved to file: pic10466.jpg)
>>
>>
>>
>
Re: PutMongo processor : specifying nested field and multiple field
on Update Query Key properties
Posted by Mike Thomsen <mi...@gmail.com>.
Yves/Pierre,
I agree with Pierre. I think starting in 1.7 we're going to need to rethink
this and possibly do some breaking changes to really make this work right.
My $0.02 is that PutMongo should allow updates to come in the following use
cases:
1. Read update keys using attributes prefixed with mongo.update and take
the whole flowfile body as the content to use for the body of the update.
Ex:
mongo.update.username = Mike
mongo.update.department = Engineering
flowfile body = { "$set": { "email": "test@test.com" }}
This method would not support nested fields unless I can find a good way
to do that with the Java API.
2. Read a query from a particular field of the body using a "query field"
attribute.
3. Read a query as a JSON string from an attribute named mongo.update.query.
For PutMongoRecord, updates should be avoided because Avro doesn't support
$ as the starting character of a field name (not sure if it can be part of
the name period). So PutMongoRecord really should be used for bulk inserts.
On Fri, Mar 16, 2018 at 8:27 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Hi Yves,
>
> Sorry for the late answer. I believe you're correct: looking at the code,
> it does not seem possible to update a document based on multiple IDs.
> Feel free to create a JIRA here https://issues.apache.org/jira/browse/NIFI
> If you're up to provide the necessary code in a PR for that, that would be
> awesome.
>
> Let us know if you need some help.
>
> Thanks
>
>
> 2018-03-09 15:44 GMT+01:00 Yves HAMEL <yh...@macif.fr>:
>
>>
>> Hello everyone,
>> I am using PutMongo processor to update documents in a mongoDB collection.
>> I would like to specify nested field and multiple key in the update Key.
>>
>> For exemple :
>> - my mongodb documents are {"nom":"HAMEL", "prenom":"Yves",
>> "ids":{"idSoc":"1234", "idInterne":"788"}}
>> - I would'd like to set update query key to something like {"ids.idSoc" ,
>> "nom"}
>> - this would means that I update document with ids.idSoc and nom equals to
>> my mew document.
>>
>> Those function are available with the native mongodb
>> db.collection.updateOne() method.
>>
>> Il would like to know if this feture will be implemented in PutMongo
>> processor ?
>> How can I request the community to implement this feature ?
>> If necessary I could propose the code to the community.
>>
>> Thanks
>> Best regards
>>
>>
>> (Embedded image moved to file: pic11173.jpg)
>>
>> Yves HAMEL
>> Direction Digital & Systèmes d’Information Groupe
>> LM_DATA
>>
>> MACIF - 2 et 4, rue Pied de Fond
>> <https://maps.google.com/?q=4,+rue+Pied+de+Fond&entry=gmail&source=g> -
>> 79037 Niort cedex 9
>> Tél. : +33 (0)5 49 09 36 06
>> Email : mon.email@macif.fr / Pré Doyen 2 – bureau 999
>>
>> www.macif.fr - Appli présente sur Google Play Store & Apple Store
>> (Embedded image moved to file: pic10466.jpg)
>
>
>
Re: PutMongo processor : specifying nested field and multiple field
on Update Query Key properties
Posted by Pierre Villard <pi...@gmail.com>.
Hi Yves,
Sorry for the late answer. I believe you're correct: looking at the code,
it does not seem possible to update a document based on multiple IDs.
Feel free to create a JIRA here https://issues.apache.org/jira/browse/NIFI
If you're up to provide the necessary code in a PR for that, that would be
awesome.
Let us know if you need some help.
Thanks
2018-03-09 15:44 GMT+01:00 Yves HAMEL <yh...@macif.fr>:
>
> Hello everyone,
> I am using PutMongo processor to update documents in a mongoDB collection.
> I would like to specify nested field and multiple key in the update Key.
>
> For exemple :
> - my mongodb documents are {"nom":"HAMEL", "prenom":"Yves",
> "ids":{"idSoc":"1234", "idInterne":"788"}}
> - I would'd like to set update query key to something like {"ids.idSoc" ,
> "nom"}
> - this would means that I update document with ids.idSoc and nom equals to
> my mew document.
>
> Those function are available with the native mongodb
> db.collection.updateOne() method.
>
> Il would like to know if this feture will be implemented in PutMongo
> processor ?
> How can I request the community to implement this feature ?
> If necessary I could propose the code to the community.
>
> Thanks
> Best regards
>
>
> (Embedded image moved to file: pic11173.jpg)
>
> Yves HAMEL
> Direction Digital & Systèmes d’Information Groupe
> LM_DATA
>
> MACIF - 2 et 4, rue Pied de Fond - 79037 Niort cedex 9
> Tél. : +33 (0)5 49 09 36 06
> Email : mon.email@macif.fr / Pré Doyen 2 – bureau 999
>
> www.macif.fr - Appli présente sur Google Play Store & Apple Store
> (Embedded image moved to file: pic10466.jpg)