You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "Ravi Papisetti (rpapiset)" <rp...@cisco.com> on 2017/07/13 21:20:53 UTC

Failed to create dbcp connection pool - using nifi API

NiFi Version 1.3

Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.

Below is snippet of code updating properties of connection pool service

String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;

HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);

entityJsonString has all the configuration set along with password.

Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:

Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")

I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.

Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.

Appreciate any help.

Thanks,

Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by "Ravi Papisetti (rpapiset)" <rp...@cisco.com>.
Thank you, Arpit for your help.

Thanks,
Ravi Papisetti
From: Arpit Gupta <ar...@hortonworks.com>
Reply-To: "users@nifi.apache.org" <us...@nifi.apache.org>
Date: Thursday, 13 July 2017 at 5:22 PM
To: "users@nifi.apache.org" <us...@nifi.apache.org>
Subject: Re: Failed to create dbcp connection pool - using nifi API

You want to set the id of the new controller service to the same value as the id of connectionPoolIn as well as the revision. Thus it will end up updating the correct controller service.

You will most likely need to create a new ControllerServiceDTO and set its id and state and then set the component of the new controller service entity to the aforementioned controller service dto.


--
Arpit

On Jul 13, 2017, at 3:16 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Thank Arpit. You are right. I sent password property as well while updating the state and it worked.

On approach you suggested: does it create new controller service (with new id) or updates the existing controller service with new state?

Thanks,
Ravi Papisetti

From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 5:08 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Following call would return the controller service entity with masked password.

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();


Instead of sending the above entity create a new controller service dto which just has the id, revision and state set and use this dto to create a new controller service entity and pass that to the put request to set the required state.

--
Arpit

On Jul 13, 2017, at 3:02 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Controller service is instantiated from template. After instantiation, controller service config is updated to overwrite jdbcurl, username and password etc.

Below is snippet of code to enable controller service:
        Optional<ControllerServiceEntity> oriConnPoolEntity = getConnPoolById(apiBaseUrl, clientId, connectionPoolId);//returns ControllerServiceEntity using connection pool identifier

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();

connectionPoolIn.getComponent().setState(state);
        ObjectMapper mapper = new ObjectMapper();
        mapper.setSerializationInclusion(Include.NON_NULL);
        String entityJsonString = null;
        try {
            entityJsonString = mapper.writeValueAsString(connectionPoolIn);
            HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
…

Do you think putRequest call here is going to update whole config (including masked password?) while updating the state?

Thanks,
Ravi Papisetti


From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 4:51 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;
HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
entityJsonString has all the configuration set along with password.
Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:
Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")
I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.
Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.
Appreciate any help.
Thanks,
Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by Arpit Gupta <ar...@hortonworks.com>.
You want to set the id of the new controller service to the same value as the id of connectionPoolIn as well as the revision. Thus it will end up updating the correct controller service.

You will most likely need to create a new ControllerServiceDTO and set its id and state and then set the component of the new controller service entity to the aforementioned controller service dto.


--
Arpit

On Jul 13, 2017, at 3:16 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Thank Arpit. You are right. I sent password property as well while updating the state and it worked.

On approach you suggested: does it create new controller service (with new id) or updates the existing controller service with new state?

Thanks,
Ravi Papisetti

From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 5:08 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Following call would return the controller service entity with masked password.

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();


Instead of sending the above entity create a new controller service dto which just has the id, revision and state set and use this dto to create a new controller service entity and pass that to the put request to set the required state.

--
Arpit

On Jul 13, 2017, at 3:02 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Controller service is instantiated from template. After instantiation, controller service config is updated to overwrite jdbcurl, username and password etc.

Below is snippet of code to enable controller service:
        Optional<ControllerServiceEntity> oriConnPoolEntity = getConnPoolById(apiBaseUrl, clientId, connectionPoolId);//returns ControllerServiceEntity using connection pool identifier

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();

connectionPoolIn.getComponent().setState(state);
        ObjectMapper mapper = new ObjectMapper();
        mapper.setSerializationInclusion(Include.NON_NULL);
        String entityJsonString = null;
        try {
            entityJsonString = mapper.writeValueAsString(connectionPoolIn);
            HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
…

Do you think putRequest call here is going to update whole config (including masked password?) while updating the state?

Thanks,
Ravi Papisetti


From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 4:51 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;
HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
entityJsonString has all the configuration set along with password.
Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:
Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")
I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.
Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.
Appreciate any help.
Thanks,
Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by "Ravi Papisetti (rpapiset)" <rp...@cisco.com>.
Thank Arpit. You are right. I sent password property as well while updating the state and it worked.

On approach you suggested: does it create new controller service (with new id) or updates the existing controller service with new state?

Thanks,
Ravi Papisetti

From: Arpit Gupta <ar...@hortonworks.com>
Reply-To: "users@nifi.apache.org" <us...@nifi.apache.org>
Date: Thursday, 13 July 2017 at 5:08 PM
To: "users@nifi.apache.org" <us...@nifi.apache.org>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Following call would return the controller service entity with masked password.

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();


Instead of sending the above entity create a new controller service dto which just has the id, revision and state set and use this dto to create a new controller service entity and pass that to the put request to set the required state.

--
Arpit

On Jul 13, 2017, at 3:02 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Controller service is instantiated from template. After instantiation, controller service config is updated to overwrite jdbcurl, username and password etc.

Below is snippet of code to enable controller service:
        Optional<ControllerServiceEntity> oriConnPoolEntity = getConnPoolById(apiBaseUrl, clientId, connectionPoolId);//returns ControllerServiceEntity using connection pool identifier

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();

connectionPoolIn.getComponent().setState(state);
        ObjectMapper mapper = new ObjectMapper();
        mapper.setSerializationInclusion(Include.NON_NULL);
        String entityJsonString = null;
        try {
            entityJsonString = mapper.writeValueAsString(connectionPoolIn);
            HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
…

Do you think putRequest call here is going to update whole config (including masked password?) while updating the state?

Thanks,
Ravi Papisetti


From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 4:51 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;
HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
entityJsonString has all the configuration set along with password.
Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:
Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")
I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.
Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.
Appreciate any help.
Thanks,
Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by Arpit Gupta <ar...@hortonworks.com>.
Following call would return the controller service entity with masked password.

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();


Instead of sending the above entity create a new controller service dto which just has the id, revision and state set and use this dto to create a new controller service entity and pass that to the put request to set the required state.

--
Arpit

On Jul 13, 2017, at 3:02 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

Controller service is instantiated from template. After instantiation, controller service config is updated to overwrite jdbcurl, username and password etc.

Below is snippet of code to enable controller service:
        Optional<ControllerServiceEntity> oriConnPoolEntity = getConnPoolById(apiBaseUrl, clientId, connectionPoolId);//returns ControllerServiceEntity using connection pool identifier

ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();

connectionPoolIn.getComponent().setState(state);
        ObjectMapper mapper = new ObjectMapper();
        mapper.setSerializationInclusion(Include.NON_NULL);
        String entityJsonString = null;
        try {
            entityJsonString = mapper.writeValueAsString(connectionPoolIn);
            HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
…

Do you think putRequest call here is going to update whole config (including masked password?) while updating the state?

Thanks,
Ravi Papisetti


From: Arpit Gupta <ar...@hortonworks.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Thursday, 13 July 2017 at 4:51 PM
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;
HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
entityJsonString has all the configuration set along with password.
Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:
Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")
I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.
Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.
Appreciate any help.
Thanks,
Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by "Ravi Papisetti (rpapiset)" <rp...@cisco.com>.
Controller service is instantiated from template. After instantiation, controller service config is updated to overwrite jdbcurl, username and password etc.

Below is snippet of code to enable controller service:

        Optional<ControllerServiceEntity> oriConnPoolEntity = getConnPoolById(apiBaseUrl, clientId, connectionPoolId);//returns ControllerServiceEntity using connection pool identifier


ControllerServiceEntity connectionPoolIn = oriConnPoolEntity.get();



connectionPoolIn.getComponent().setState(state);

        ObjectMapper mapper = new ObjectMapper();

        mapper.setSerializationInclusion(Include.NON_NULL);

        String entityJsonString = null;

        try {

            entityJsonString = mapper.writeValueAsString(connectionPoolIn);

            HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);

…



Do you think putRequest call here is going to update whole config (including masked password?) while updating the state?



Thanks,

Ravi Papisetti


From: Arpit Gupta <ar...@hortonworks.com>
Reply-To: "users@nifi.apache.org" <us...@nifi.apache.org>
Date: Thursday, 13 July 2017 at 4:51 PM
To: "users@nifi.apache.org" <us...@nifi.apache.org>
Subject: Re: Failed to create dbcp connection pool - using nifi API

Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:

NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;
HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);
entityJsonString has all the configuration set along with password.
Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:
Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")
I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.
Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.
Appreciate any help.
Thanks,
Ravi Papisetti

Re: Failed to create dbcp connection pool - using nifi API

Posted by Arpit Gupta <ar...@hortonworks.com>.
Hi Ravi


How are you issuing the call to enable the controller service?

One thing that could be happening is that the call to enable the controller service is also sending the password. Response returned when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong.

--
Arpit

On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset) <rp...@cisco.com>> wrote:


NiFi Version 1.3

Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client.

Below is snippet of code updating properties of connection pool service

String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;

HttpResponse httpResponse = HttpUtil.putRequest(apiUrl, entityJsonString);

entityJsonString has all the configuration set along with password.

Here I can see controller entity service with password set. However when QueryDatabaseTable (or any other sql processor) is started, it fails with Processor exception:

Cannot create PoolableConnectionFactory (FATAL: password authentication failed for user "xxxxx")

I logged into web UI, updated connection pool service manually and it works fine. Same password is used via api and in UI.

Wondering why the password set using API didn't work, but works when the same password set using NiFi Web UI.

Appreciate any help.

Thanks,

Ravi Papisetti