[jira] [Updated] (AMBARI-21760) POST config via Ambari REST API doesn't get the password mask revolved back to the real pwd

["Di Li (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-21760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Di Li updated AMBARI-21760:
---------------------------
    Description: 
we are hitting a kind of weird issue in the BigSQL upgrade process.  We do the following:
* BigSQL backup takes a backup of the bigsql configurations using Ambari REST API.  This gives use password fields which are masked like SECRET:bigsql-users-env:1:bigsql_user_password
* BigSQL upgrade updates the bigsql configurations using the Ambari REST API by passing in the backup (including the masked fields)

The end result is that the configuration stored in the DB for the password is SECRET:bigsql-users-env:1:bigsql_user_password.  Once BIGSQL is added back to the cluster and we attempt to start it, that masked password value is included in the command json rather than the unmasked password.

So my questions are:

1) Is there a way of properly updating the configurations when they contain these masked passwords such that the passwords in the DB will actually be resolved?
2) Is this just an Ambari defect, in that it should unmask the password when it includes the value in the command json?  As a note, kerberization of the cluster actually attempts to determine the actual password value by using the org.apache.ambari.server.utils.SecretReference (edited)

[4:58] 
@here Please note this is a critical defect for us.  It results in several problems, first we update the bigsql user OS password using the masked password value which means we can't log in using the normal password.  Second, we are hitting DB consistency warnings from the old BigSQL configurations which got disassociated from BigSQL when it was removed from the cluster.  If we clean up these old configurations and then enable kerberos post migration, the kerberization will fail because it can't resolve the masked password reference (in other words the SecretReference class throws an exception about missing a config version).

Please note the issue we hit wasn't during blueprint installation. it was during bigsql upgrade, here is the workflow

1. bigsql backup before Ambari upgrade - bigsql will use ambari rest api to read all bigsql configurations. the rest api returns "SECRET:bigsql-users-env:1:bigsql_user_password" for the password field ( instead of the real pwd, this is expected , by design)

2. Ambari upgrade then EU
3. Bigsql upgrade- bigsql will use ambari server rest api to HTTP PUT the data exported in step 1 back to the database. bigsql does not modify the mask. in other words, "SECRET:bigsql-users-env:1:bigsql_user_passwords" got stored back in to the db.
4. bigsql perform some ambari custom command, where it expects the password to be resolved in the command-[task_id].json file.

_Step 4 is where the issue happened. The mask remained in the command json file, instead of the real pwd._

The weird part is that our QA hit it on migration (Ambari at 2.2/2.4.0 level), but didn't hit it on HDP 2.5.3/BigSQL 4.2.5 to HDP 2.6.2/BigSQL 5.0.1 upgrade.

So I am wondering if this has anything to do with starting with a cluser with Ambari 2.4.x ( thus me asked you for an HDP Ambari build)

> POST config via Ambari REST API doesn't get the password mask revolved back to the real pwd
> -------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-21760
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21760
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-sever
>    Affects Versions: 2.5.2
>            Reporter: Di Li
>             Fix For: 2.5.2
>
>
> we are hitting a kind of weird issue in the BigSQL upgrade process.  We do the following:
> * BigSQL backup takes a backup of the bigsql configurations using Ambari REST API.  This gives use password fields which are masked like SECRET:bigsql-users-env:1:bigsql_user_password
> * BigSQL upgrade updates the bigsql configurations using the Ambari REST API by passing in the backup (including the masked fields)
> The end result is that the configuration stored in the DB for the password is SECRET:bigsql-users-env:1:bigsql_user_password.  Once BIGSQL is added back to the cluster and we attempt to start it, that masked password value is included in the command json rather than the unmasked password.
> So my questions are:
> 1) Is there a way of properly updating the configurations when they contain these masked passwords such that the passwords in the DB will actually be resolved?
> 2) Is this just an Ambari defect, in that it should unmask the password when it includes the value in the command json?  As a note, kerberization of the cluster actually attempts to determine the actual password value by using the org.apache.ambari.server.utils.SecretReference (edited)
> [4:58] 
> @here Please note this is a critical defect for us.  It results in several problems, first we update the bigsql user OS password using the masked password value which means we can't log in using the normal password.  Second, we are hitting DB consistency warnings from the old BigSQL configurations which got disassociated from BigSQL when it was removed from the cluster.  If we clean up these old configurations and then enable kerberos post migration, the kerberization will fail because it can't resolve the masked password reference (in other words the SecretReference class throws an exception about missing a config version).
> Please note the issue we hit wasn't during blueprint installation. it was during bigsql upgrade, here is the workflow
> 1. bigsql backup before Ambari upgrade - bigsql will use ambari rest api to read all bigsql configurations. the rest api returns "SECRET:bigsql-users-env:1:bigsql_user_password" for the password field ( instead of the real pwd, this is expected , by design)
> 2. Ambari upgrade then EU
> 3. Bigsql upgrade- bigsql will use ambari server rest api to HTTP PUT the data exported in step 1 back to the database. bigsql does not modify the mask. in other words, "SECRET:bigsql-users-env:1:bigsql_user_passwords" got stored back in to the db.
> 4. bigsql perform some ambari custom command, where it expects the password to be resolved in the command-[task_id].json file.
> _Step 4 is where the issue happened. The mask remained in the command json file, instead of the real pwd._
> The weird part is that our QA hit it on migration (Ambari at 2.2/2.4.0 level), but didn't hit it on HDP 2.5.3/BigSQL 4.2.5 to HDP 2.6.2/BigSQL 5.0.1 upgrade.
> So I am wondering if this has anything to do with starting with a cluser with Ambari 2.4.x ( thus me asked you for an HDP Ambari build)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)