You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2014/08/20 21:24:07 UTC
Review Request 24904: Allow Customers To Define The Sqoop User
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24904/
-----------------------------------------------------------
Review request for Ambari and Andrew Onischuk.
Bugs: AMBARI-6955
https://issues.apache.org/jira/browse/AMBARI-6955
Repository: ambari
Description
-------
When installing Sqoop, there is no configuration section, and thus no "Misc" tab to specify the Sqoop user.
Post-install, the Sqoop user is not exposed via
api/v1/clusters/c1/configurations?type=global&tag=version1
And it is hardcoded as
sqoop_user = "sqoop"
in params.py
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py 723cd3d
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 2e8c8cd
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/package/scripts/params.py 4384171
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py f769b3f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py 8da05ed
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/params.py 6c307d3
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/sqoop.py 007c031
ambari-server/src/test/python/stacks/1.3.2/configs/default.hbasedecom.json cc1154d
ambari-server/src/test/python/stacks/1.3.2/configs/default.json 96a7682
ambari-server/src/test/python/stacks/1.3.2/configs/default.non_gmetad_host.json aab868f
ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 75c53fc
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 34f7b7e
ambari-server/src/test/python/stacks/2.0.6/configs/default.json bae564f
ambari-server/src/test/python/stacks/2.0.6/configs/default.non_gmetad_host.json 319863a
ambari-server/src/test/python/stacks/2.0.6/configs/ha_default.json 7adf696
ambari-server/src/test/python/stacks/2.0.6/configs/ha_secured.json 6c61278
ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 88de472
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 926a6c6
ambari-server/src/test/python/stacks/2.1/configs/default.json ddfe340
ambari-server/src/test/python/stacks/2.1/configs/secured.json 335d66c
ambari-web/app/data/HDP2/site_properties.js 169840f
ambari-web/app/data/site_properties.js 5fa4b37
Diff: https://reviews.apache.org/r/24904/diff/
Testing
-------
----------------------------------------------------------------------
Ran 221 tests in 0.546s
OK
----------------------------------------------------------------------
Total run:593
Total errors:0
Total failures:0
OK
Thanks,
Dmitro Lisnichenko
Re: Review Request 24904: Allow Customers To Define The Sqoop User
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24904/#review51173
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On Aug. 21, 2014, 1:28 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24904/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2014, 1:28 p.m.)
>
>
> Review request for Ambari and Andrew Onischuk.
>
>
> Bugs: AMBARI-6955
> https://issues.apache.org/jira/browse/AMBARI-6955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When installing Sqoop, there is no configuration section, and thus no "Misc" tab to specify the Sqoop user.
> Post-install, the Sqoop user is not exposed via
> api/v1/clusters/c1/configurations?type=global&tag=version1
> And it is hardcoded as
> sqoop_user = "sqoop"
> in params.py
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py 723cd3d
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 2e8c8cd
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/configuration/sqoop-env.xml 3731f8b
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/package/scripts/params.py 4384171
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py f769b3f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py 8da05ed
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/configuration/sqoop-env.xml 3731f8b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/params.py 6c307d3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/sqoop.py 007c031
> ambari-server/src/test/python/stacks/1.3.2/configs/default.hbasedecom.json cc1154d
> ambari-server/src/test/python/stacks/1.3.2/configs/default.json 96a7682
> ambari-server/src/test/python/stacks/1.3.2/configs/default.non_gmetad_host.json aab868f
> ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 75c53fc
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 34f7b7e
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json bae564f
> ambari-server/src/test/python/stacks/2.0.6/configs/default.non_gmetad_host.json 319863a
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_default.json 7adf696
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_secured.json 6c61278
> ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 88de472
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 926a6c6
> ambari-server/src/test/python/stacks/2.1/configs/default.json ddfe340
> ambari-server/src/test/python/stacks/2.1/configs/secured.json 335d66c
> ambari-web/app/data/HDP2/site_properties.js 169840f
> ambari-web/app/data/site_properties.js 5fa4b37
>
> Diff: https://reviews.apache.org/r/24904/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Ran 221 tests in 0.546s
>
> OK
> ----------------------------------------------------------------------
> Total run:593
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 24904: Allow Customers To Define The Sqoop User
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24904/
-----------------------------------------------------------
(Updated Aug. 21, 2014, 1:28 p.m.)
Review request for Ambari and Andrew Onischuk.
Bugs: AMBARI-6955
https://issues.apache.org/jira/browse/AMBARI-6955
Repository: ambari
Description
-------
When installing Sqoop, there is no configuration section, and thus no "Misc" tab to specify the Sqoop user.
Post-install, the Sqoop user is not exposed via
api/v1/clusters/c1/configurations?type=global&tag=version1
And it is hardcoded as
sqoop_user = "sqoop"
in params.py
Diffs (updated)
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py 723cd3d
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 2e8c8cd
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/configuration/sqoop-env.xml 3731f8b
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/package/scripts/params.py 4384171
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py f769b3f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py 8da05ed
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/configuration/sqoop-env.xml 3731f8b
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/params.py 6c307d3
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/sqoop.py 007c031
ambari-server/src/test/python/stacks/1.3.2/configs/default.hbasedecom.json cc1154d
ambari-server/src/test/python/stacks/1.3.2/configs/default.json 96a7682
ambari-server/src/test/python/stacks/1.3.2/configs/default.non_gmetad_host.json aab868f
ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 75c53fc
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 34f7b7e
ambari-server/src/test/python/stacks/2.0.6/configs/default.json bae564f
ambari-server/src/test/python/stacks/2.0.6/configs/default.non_gmetad_host.json 319863a
ambari-server/src/test/python/stacks/2.0.6/configs/ha_default.json 7adf696
ambari-server/src/test/python/stacks/2.0.6/configs/ha_secured.json 6c61278
ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 88de472
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 926a6c6
ambari-server/src/test/python/stacks/2.1/configs/default.json ddfe340
ambari-server/src/test/python/stacks/2.1/configs/secured.json 335d66c
ambari-web/app/data/HDP2/site_properties.js 169840f
ambari-web/app/data/site_properties.js 5fa4b37
Diff: https://reviews.apache.org/r/24904/diff/
Testing
-------
----------------------------------------------------------------------
Ran 221 tests in 0.546s
OK
----------------------------------------------------------------------
Total run:593
Total errors:0
Total failures:0
OK
Thanks,
Dmitro Lisnichenko
Re: Review Request 24904: Allow Customers To Define The Sqoop User
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24904/#review51168
-----------------------------------------------------------
ambari-web/app/data/HDP2/site_properties.js
<https://reviews.apache.org/r/24904/#comment89200>
The same here
ambari-web/app/data/site_properties.js
<https://reviews.apache.org/r/24904/#comment89199>
We can not hardcode defaultValue and description here. In this case this should be taken from xml's this is a bit less hardcoding on UI which is better.
- Andrew Onischuk
On Aug. 20, 2014, 7:24 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24904/
> -----------------------------------------------------------
>
> (Updated Aug. 20, 2014, 7:24 p.m.)
>
>
> Review request for Ambari and Andrew Onischuk.
>
>
> Bugs: AMBARI-6955
> https://issues.apache.org/jira/browse/AMBARI-6955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When installing Sqoop, there is no configuration section, and thus no "Misc" tab to specify the Sqoop user.
> Post-install, the Sqoop user is not exposed via
> api/v1/clusters/c1/configurations?type=global&tag=version1
> And it is hardcoded as
> sqoop_user = "sqoop"
> in params.py
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py 723cd3d
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 2e8c8cd
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/package/scripts/params.py 4384171
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py f769b3f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py 8da05ed
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/params.py 6c307d3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/sqoop.py 007c031
> ambari-server/src/test/python/stacks/1.3.2/configs/default.hbasedecom.json cc1154d
> ambari-server/src/test/python/stacks/1.3.2/configs/default.json 96a7682
> ambari-server/src/test/python/stacks/1.3.2/configs/default.non_gmetad_host.json aab868f
> ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 75c53fc
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 34f7b7e
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json bae564f
> ambari-server/src/test/python/stacks/2.0.6/configs/default.non_gmetad_host.json 319863a
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_default.json 7adf696
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_secured.json 6c61278
> ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 88de472
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 926a6c6
> ambari-server/src/test/python/stacks/2.1/configs/default.json ddfe340
> ambari-server/src/test/python/stacks/2.1/configs/secured.json 335d66c
> ambari-web/app/data/HDP2/site_properties.js 169840f
> ambari-web/app/data/site_properties.js 5fa4b37
>
> Diff: https://reviews.apache.org/r/24904/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Ran 221 tests in 0.546s
>
> OK
> ----------------------------------------------------------------------
> Total run:593
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 24904: Allow Customers To Define The Sqoop User
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24904/#review51135
-----------------------------------------------------------
Talked on this Dmitriy internally. Posting the review results here:
1.
Try this:
+if 'sqoop-env' in config['configurations'] and 'sqoop_user' in config['configurations']['sqoop-env']:
+ sqoop_user = config['configurations']['sqoop-env']['sqoop_user']
+else:
+ sqoop_user = ""
+
->
+ sqoop_user = config['configurations']['sqoop-env']['sqoop_user']
If there is no sqoop-env it will steal work since we have lazy failing.
2. Thi is not safe. Since we can remove / rename sqoop_user property, as we've done before.
has_sqoop_client = not len(sqoop_user) == 0
->
has_sqoop_client = 'sqoop-env' in config['configurations']
3. Also you haven't add sqoop_user to sqoop-env.xml
Which can break sqoop on blueprints.
- Andrew Onischuk
On Aug. 20, 2014, 7:24 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24904/
> -----------------------------------------------------------
>
> (Updated Aug. 20, 2014, 7:24 p.m.)
>
>
> Review request for Ambari and Andrew Onischuk.
>
>
> Bugs: AMBARI-6955
> https://issues.apache.org/jira/browse/AMBARI-6955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When installing Sqoop, there is no configuration section, and thus no "Misc" tab to specify the Sqoop user.
> Post-install, the Sqoop user is not exposed via
> api/v1/clusters/c1/configurations?type=global&tag=version1
> And it is hardcoded as
> sqoop_user = "sqoop"
> in params.py
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py 723cd3d
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 2e8c8cd
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/SQOOP/package/scripts/params.py 4384171
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py f769b3f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py 8da05ed
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/params.py 6c307d3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/SQOOP/package/scripts/sqoop.py 007c031
> ambari-server/src/test/python/stacks/1.3.2/configs/default.hbasedecom.json cc1154d
> ambari-server/src/test/python/stacks/1.3.2/configs/default.json 96a7682
> ambari-server/src/test/python/stacks/1.3.2/configs/default.non_gmetad_host.json aab868f
> ambari-server/src/test/python/stacks/1.3.2/configs/secured.json 75c53fc
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 34f7b7e
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json bae564f
> ambari-server/src/test/python/stacks/2.0.6/configs/default.non_gmetad_host.json 319863a
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_default.json 7adf696
> ambari-server/src/test/python/stacks/2.0.6/configs/ha_secured.json 6c61278
> ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 88de472
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 926a6c6
> ambari-server/src/test/python/stacks/2.1/configs/default.json ddfe340
> ambari-server/src/test/python/stacks/2.1/configs/secured.json 335d66c
> ambari-web/app/data/HDP2/site_properties.js 169840f
> ambari-web/app/data/site_properties.js 5fa4b37
>
> Diff: https://reviews.apache.org/r/24904/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Ran 221 tests in 0.546s
>
> OK
> ----------------------------------------------------------------------
> Total run:593
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>