You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by mohammed shambakey <sh...@gmail.com> on 2017/05/25 14:04:18 UTC
using rest api to enable transmission in remote process group
Hi
I'm trying to call nifi rest api from Python "request" module (
http://docs.python-requests.org/en/master/) to enable transmission in a
remote process group (I think this is done by changing the "transmitting"
element in the required input port to a boolean "true").
So, I get the JSON file of the remote process group using this Python
"requests" module commad:
rpg=requests.get("http://localhost:8085/nifi-api/remote-process-groups/<id
of remote process group>").json()
The output json file is attached
I modified the "transmitting" element of the the input port to boolean
"True", then, I tried to pass the modified JSON file to the following
command
rpg_new=requests.put("http://localhost:8085/nifi-api/remote-process-groups/<id
of remote process group>", json=rpg).json()
But I receive the following error:
"
Traceback (most recent call last):
File "rpg_nifi_create.py", line 116, in <module>
rpg_update=requests.put(url_rpg_update+rpg['id'],json=rpg,headers=token_header).json()
File
"/home/shambakey1/miniconda2/lib/python2.7/site-packages/requests/models.py",
line 826, in json
return complexjson.loads(self.text, **kwargs)
File "/home/shambakey1/miniconda2/lib/python2.7/json/__init__.py", line
339, in loads
return _default_decoder.decode(s)
File "/home/shambakey1/miniconda2/lib/python2.7/json/decoder.py", line
367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 2 - line 1 column 63 (char 1 - 62)
"
I tried to truncate the passed JSON file such that it contains information
only about "revision" and the modified "inputPorts" list, but still the
same problem.
I can the JSON file (before and after modification) using jsonview, so I
think there is nothing wrong the JSON structure (e.g., no independent
dictionaries). I don't know what is the problem?
Regards
--
Mohammed
Re: using rest api to enable transmission in remote process group
Posted by Matt Gilman <ma...@gmail.com>.
Once the RPG is created it starts communicating with the target instance.
This process is done asynchronously so you will need to wait for that to
complete. The different options are really just the condition that
identifies when the connection can be created. An alternative would be to
loop and GET the RPG with each iteration and break out of your loop once
the desired port is available. At that point, your request to create the
connection should work on the first attempt.
Thanks
Matt
On Fri, May 26, 2017 at 9:08 AM, mohammed shambakey <sh...@gmail.com>
wrote:
> Thanks Matt
>
> I used the "/remote-processor-groups/<group id>/input-ports/<port id>"
> with a JSON file that has only "revision", and under
> "remoteProcessGroupPort", it has "id", "groupId" and "transmitting", and it
> worked. (Currently, I'm using NIFI 1.2.0 and the docs says: "This
> endpoint is subject to change as NiFi and it's REST API evolve").
>
> One more thing please, I had to make a loop to make a connection between a
> "processor" and a created "remote process group". In each iteration, I
> check the status of the rest api to find if the connection is created
> successfully or not, otherwise, the loop sleeps for some time then begins
> the next iteration. I just wonder if there is something I should do in NIFI
> configuration to the need for this loop?
>
> Regards
>
> On Thu, May 25, 2017 at 11:06 AM, Matt Gilman <ma...@gmail.com>
> wrote:
>
>> Mohammed,
>>
>> I would guess that the request failed and the response body contains an
>> error message and you're attempting to parse that as JSON. I'd suggest
>> inspecting the response codes from the request before attempting to
>> interpret the response body. You can also look at your logs/nifi-user.log
>> for any errors from your request.
>>
>> Thanks
>>
>> Matt
>>
>> On Thu, May 25, 2017 at 10:04 AM, mohammed shambakey <
>> shambakey1@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I'm trying to call nifi rest api from Python "request" module (
>>> http://docs.python-requests.org/en/master/) to enable transmission in a
>>> remote process group (I think this is done by changing the "transmitting"
>>> element in the required input port to a boolean "true").
>>>
>>> So, I get the JSON file of the remote process group using this Python
>>> "requests" module commad:
>>>
>>> rpg=requests.get("http://localhost:8085/nifi-api/remote-process-groups/<id
>>> of remote process group>").json()
>>> The output json file is attached
>>>
>>> I modified the "transmitting" element of the the input port to boolean
>>> "True", then, I tried to pass the modified JSON file to the following
>>> command
>>>
>>> rpg_new=requests.put("http://localhost:8085/nifi-api/remote-
>>> process-groups/<id of remote process group>", json=rpg).json()
>>>
>>> But I receive the following error:
>>> "
>>> Traceback (most recent call last):
>>> File "rpg_nifi_create.py", line 116, in <module>
>>> rpg_update=requests.put(url_rpg_update+rpg['id'],json=rpg,he
>>> aders=token_header).json()
>>> File "/home/shambakey1/miniconda2/lib/python2.7/site-packages/requests/models.py",
>>> line 826, in json
>>> return complexjson.loads(self.text, **kwargs)
>>> File "/home/shambakey1/miniconda2/lib/python2.7/json/__init__.py",
>>> line 339, in loads
>>> return _default_decoder.decode(s)
>>> File "/home/shambakey1/miniconda2/lib/python2.7/json/decoder.py",
>>> line 367, in decode
>>> raise ValueError(errmsg("Extra data", s, end, len(s)))
>>> ValueError: Extra data: line 1 column 2 - line 1 column 63 (char 1 - 62)
>>> "
>>>
>>> I tried to truncate the passed JSON file such that it contains
>>> information only about "revision" and the modified "inputPorts" list, but
>>> still the same problem.
>>>
>>> I can the JSON file (before and after modification) using jsonview, so I
>>> think there is nothing wrong the JSON structure (e.g., no independent
>>> dictionaries). I don't know what is the problem?
>>>
>>> Regards
>>>
>>> --
>>> Mohammed
>>>
>>
>>
>
>
> --
> Mohammed
>
Re: using rest api to enable transmission in remote process group
Posted by mohammed shambakey <sh...@gmail.com>.
Thanks Matt
I used the "/remote-processor-groups/<group id>/input-ports/<port id>" with
a JSON file that has only "revision", and under "remoteProcessGroupPort",
it has "id", "groupId" and "transmitting", and it worked. (Currently, I'm
using NIFI 1.2.0 and the docs says: "This endpoint is subject to change as
NiFi and it's REST API evolve").
One more thing please, I had to make a loop to make a connection between a
"processor" and a created "remote process group". In each iteration, I
check the status of the rest api to find if the connection is created
successfully or not, otherwise, the loop sleeps for some time then begins
the next iteration. I just wonder if there is something I should do in NIFI
configuration to the need for this loop?
Regards
On Thu, May 25, 2017 at 11:06 AM, Matt Gilman <ma...@gmail.com>
wrote:
> Mohammed,
>
> I would guess that the request failed and the response body contains an
> error message and you're attempting to parse that as JSON. I'd suggest
> inspecting the response codes from the request before attempting to
> interpret the response body. You can also look at your logs/nifi-user.log
> for any errors from your request.
>
> Thanks
>
> Matt
>
> On Thu, May 25, 2017 at 10:04 AM, mohammed shambakey <shambakey1@gmail.com
> > wrote:
>
>> Hi
>>
>> I'm trying to call nifi rest api from Python "request" module (
>> http://docs.python-requests.org/en/master/) to enable transmission in a
>> remote process group (I think this is done by changing the "transmitting"
>> element in the required input port to a boolean "true").
>>
>> So, I get the JSON file of the remote process group using this Python
>> "requests" module commad:
>>
>> rpg=requests.get("http://localhost:8085/nifi-api/remote-process-groups/<id
>> of remote process group>").json()
>> The output json file is attached
>>
>> I modified the "transmitting" element of the the input port to boolean
>> "True", then, I tried to pass the modified JSON file to the following
>> command
>>
>> rpg_new=requests.put("http://localhost:8085/nifi-api/remote-
>> process-groups/<id of remote process group>", json=rpg).json()
>>
>> But I receive the following error:
>> "
>> Traceback (most recent call last):
>> File "rpg_nifi_create.py", line 116, in <module>
>> rpg_update=requests.put(url_rpg_update+rpg['id'],json=rpg,he
>> aders=token_header).json()
>> File "/home/shambakey1/miniconda2/lib/python2.7/site-packages/requests/models.py",
>> line 826, in json
>> return complexjson.loads(self.text, **kwargs)
>> File "/home/shambakey1/miniconda2/lib/python2.7/json/__init__.py",
>> line 339, in loads
>> return _default_decoder.decode(s)
>> File "/home/shambakey1/miniconda2/lib/python2.7/json/decoder.py", line
>> 367, in decode
>> raise ValueError(errmsg("Extra data", s, end, len(s)))
>> ValueError: Extra data: line 1 column 2 - line 1 column 63 (char 1 - 62)
>> "
>>
>> I tried to truncate the passed JSON file such that it contains
>> information only about "revision" and the modified "inputPorts" list, but
>> still the same problem.
>>
>> I can the JSON file (before and after modification) using jsonview, so I
>> think there is nothing wrong the JSON structure (e.g., no independent
>> dictionaries). I don't know what is the problem?
>>
>> Regards
>>
>> --
>> Mohammed
>>
>
>
--
Mohammed
Re: using rest api to enable transmission in remote process group
Posted by Matt Gilman <ma...@gmail.com>.
Mohammed,
I would guess that the request failed and the response body contains an
error message and you're attempting to parse that as JSON. I'd suggest
inspecting the response codes from the request before attempting to
interpret the response body. You can also look at your logs/nifi-user.log
for any errors from your request.
Thanks
Matt
On Thu, May 25, 2017 at 10:04 AM, mohammed shambakey <sh...@gmail.com>
wrote:
> Hi
>
> I'm trying to call nifi rest api from Python "request" module (
> http://docs.python-requests.org/en/master/) to enable transmission in a
> remote process group (I think this is done by changing the "transmitting"
> element in the required input port to a boolean "true").
>
> So, I get the JSON file of the remote process group using this Python
> "requests" module commad:
>
> rpg=requests.get("http://localhost:8085/nifi-api/remote-process-groups/<id
> of remote process group>").json()
> The output json file is attached
>
> I modified the "transmitting" element of the the input port to boolean
> "True", then, I tried to pass the modified JSON file to the following
> command
>
> rpg_new=requests.put("http://localhost:8085/nifi-api/
> remote-process-groups/<id of remote process group>", json=rpg).json()
>
> But I receive the following error:
> "
> Traceback (most recent call last):
> File "rpg_nifi_create.py", line 116, in <module>
> rpg_update=requests.put(url_rpg_update+rpg['id'],json=rpg,
> headers=token_header).json()
> File "/home/shambakey1/miniconda2/lib/python2.7/site-packages/requests/models.py",
> line 826, in json
> return complexjson.loads(self.text, **kwargs)
> File "/home/shambakey1/miniconda2/lib/python2.7/json/__init__.py", line
> 339, in loads
> return _default_decoder.decode(s)
> File "/home/shambakey1/miniconda2/lib/python2.7/json/decoder.py", line
> 367, in decode
> raise ValueError(errmsg("Extra data", s, end, len(s)))
> ValueError: Extra data: line 1 column 2 - line 1 column 63 (char 1 - 62)
> "
>
> I tried to truncate the passed JSON file such that it contains information
> only about "revision" and the modified "inputPorts" list, but still the
> same problem.
>
> I can the JSON file (before and after modification) using jsonview, so I
> think there is nothing wrong the JSON structure (e.g., no independent
> dictionaries). I don't know what is the problem?
>
> Regards
>
> --
> Mohammed
>