You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/01/13 04:09:22 UTC
[GitHub] [ozone] symious opened a new pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
symious opened a new pull request #1788:
URL: https://github.com/apache/ozone/pull/1788
…er setup
## What changes were proposed in this pull request?
- Add the option of "--security" to the tool of "genconf".
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2212
## How was this patch tested?
Manually tested. The usage of the tool is now as follows.
```
Usage: ozone genconf [-hV] [--security] [--verbose] [-conf=<configurationPath>]
[-D=<String=String>]... <path>
Tool to generate template ozone-site.xml
<path> Directory path where ozone-site file should be generated.
--security Generate security config.
--verbose More verbose output. Show the stack trace of the errors.
-conf=<configurationPath>
-D, --set=<String=String>
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on a change in pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on a change in pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#discussion_r620085937
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2143,57 +2187,65 @@
</property>
<property>
<name>ozone.recon.http.auth.kerberos.keytab</name>
- <value/>
- <tag>RECON, SECURITY</tag>
+ <value>/path/to/HTTP.keytab</value>
+ <tag>RECON, SECURITY, KERBEROS</tag>
<description>
The keytab file for HTTP Kerberos authentication in Recon.
</description>
</property>
<property>
<name>ozone.recon.http.auth.kerberos.principal</name>
- <value/>
- <tag>RECON</tag>
+ <value>HTTP/_HOST@REALM</value>
+ <tag>RECON, KERBEROS</tag>
Review comment:
@dineshchitlangia Thanks for the review, SECURITY tag added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-775203955
Thanks the patch @symious. It looks a simple change for me.
My only fear is that I am not sure if the generated config file is enough to start Ozone in secure environment.
For unsecure environment all the required attributes are filled with some reasonable default (like using /tmp for metadata).
I don't know what is a good approach, but the overall goal is to make the configuration easier for the users.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia commented on a change in pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on a change in pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#discussion_r575751760
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2143,57 +2187,65 @@
</property>
<property>
<name>ozone.recon.http.auth.kerberos.keytab</name>
- <value/>
- <tag>RECON, SECURITY</tag>
+ <value>/path/to/HTTP.keytab</value>
+ <tag>RECON, SECURITY, KERBEROS</tag>
<description>
The keytab file for HTTP Kerberos authentication in Recon.
</description>
</property>
<property>
<name>ozone.recon.http.auth.kerberos.principal</name>
- <value/>
- <tag>RECON</tag>
+ <value>HTTP/_HOST@REALM</value>
+ <tag>RECON, KERBEROS</tag>
Review comment:
Add tag SECURITY in line with other similar configs.
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -1699,31 +1740,34 @@
</property>
<property>
<name>ozone.om.kerberos.keytab.file</name>
- <value></value>
- <tag> OZONE, SECURITY</tag>
+ <value/>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description> The keytab file used by OzoneManager daemon to login as its
service principal. The principal name is configured with
ozone.om.kerberos.principal.
</description>
</property>
<property>
<name>ozone.om.kerberos.principal</name>
- <value></value>
- <tag> OZONE, SECURITY</tag>
+ <value/>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description>The OzoneManager service principal. Ex om/_HOST@REALM.COM</description>
</property>
<property>
<name>ozone.om.http.auth.kerberos.principal</name>
- <value>HTTP/_HOST@EXAMPLE.COM</value>
+ <value>HTTP/_HOST@REALM</value>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description>
- OzoneManager http server kerberos principal.
+ Ozone Manager http server service principal if SPNEGO is enabled for om http server.
</description>
</property>
<property>
<name>ozone.om.http.auth.kerberos.keytab</name>
- <value>/etc/security/keytabs/HTTP.keytab</value>
+ <value>/path/to/HTTP.keytab</value>
Review comment:
I wouldn't recommend changing the default value of the path to keytab on all such occurrences.
From what I have seen, most admins keep the keytabs at /etc/security/keytabs.
If we avoid changing this, then only a very few subset of users would have to change this in the template before using.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious edited a comment on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious edited a comment on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-759225997
@amaliujia Sure. The screen shot is as follows.
<img width="583" alt="Screenshot 2021-01-13 at 2 00 19 PM" src="https://user-images.githubusercontent.com/14933944/104412724-d36d7f00-55a7-11eb-8fe0-b334e6dde5d4.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on a change in pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on a change in pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#discussion_r575753318
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -1699,31 +1740,34 @@
</property>
<property>
<name>ozone.om.kerberos.keytab.file</name>
- <value></value>
- <tag> OZONE, SECURITY</tag>
+ <value/>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description> The keytab file used by OzoneManager daemon to login as its
service principal. The principal name is configured with
ozone.om.kerberos.principal.
</description>
</property>
<property>
<name>ozone.om.kerberos.principal</name>
- <value></value>
- <tag> OZONE, SECURITY</tag>
+ <value/>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description>The OzoneManager service principal. Ex om/_HOST@REALM.COM</description>
</property>
<property>
<name>ozone.om.http.auth.kerberos.principal</name>
- <value>HTTP/_HOST@EXAMPLE.COM</value>
+ <value>HTTP/_HOST@REALM</value>
+ <tag>OZONE, SECURITY, KERBEROS</tag>
<description>
- OzoneManager http server kerberos principal.
+ Ozone Manager http server service principal if SPNEGO is enabled for om http server.
</description>
</property>
<property>
<name>ozone.om.http.auth.kerberos.keytab</name>
- <value>/etc/security/keytabs/HTTP.keytab</value>
+ <value>/path/to/HTTP.keytab</value>
Review comment:
@dineshchitlangia Thanks for the review, the keytabs directory has been changed to "/etc/security/keytabs".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-776591727
@elek @adoroszlai Thanks for the reply. I think it will be good to wait for @dineshchitlangia 's reply. Meanwhile, I can try to build a secure cluster and fill in the required configs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia commented on a change in pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on a change in pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#discussion_r575754494
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -2143,57 +2187,65 @@
</property>
<property>
<name>ozone.recon.http.auth.kerberos.keytab</name>
- <value/>
- <tag>RECON, SECURITY</tag>
+ <value>/path/to/HTTP.keytab</value>
+ <tag>RECON, SECURITY, KERBEROS</tag>
<description>
The keytab file for HTTP Kerberos authentication in Recon.
</description>
</property>
<property>
<name>ozone.recon.http.auth.kerberos.principal</name>
- <value/>
- <tag>RECON</tag>
+ <value>HTTP/_HOST@REALM</value>
+ <tag>RECON, KERBEROS</tag>
Review comment:
@symious any thoughts about this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-778609267
@dineshchitlangia Thanks for the review. Added a new commit with the following update:
1. Added the unit test for security config generation, and update the command option description.
2. After checking the configurations with the tag of "SECURITY", I found a lot of unrelated or fundamental configs that users shouldn't use usually, such as`hdds.x509.signature.algorithm`. And then I walk through the document of Ozone Security, I found that all configs are related to kerberos, so my idea is to add a new tag named as "KERBEROS" to track the common used security configs.
3. After checking the docker file of ozonesecure, I found that the principal and keytab are coordinated with the KDC server, which means the KDC server and the keytab file has to be generated and prepared before we start ozone cluster, so the configs must be updated manually unless we set up a KDC server and update the keytab files for the users, which I think is quite strange.
In docker, flokkr/issuer helps to initial KDC and update the keytab files, so for testing, it would be quite convenient for the user to just use docker. And for `genconf --security`, I think it's mainly for the users who has Kerberos set up already and want to fill the required security configs template for a productive secure cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia edited a comment on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-759219634
@symious is it possible to attach a screenshot for the output of this command flag?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-775224577
> Thanks the patch @symious. It looks a simple change for me.
>
> My only fear is that I am not sure if the generated config file is enough to start Ozone in secure environment.
>
> For unsecure environment all the required attributes are filled with some reasonable default (like using /tmp for metadata).
>
> I don't know what is a good approach, but the overall goal is to make the configuration easier for the users.
@elek Thanks for the reply. IMHO, secure config is for advanced users, that they do have the ability and requirement to build a secure cluster. The default config may also mislead the user somehow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-776560278
> My only fear is that I am not sure if the generated config file is enough to start Ozone in secure environment.
Thanks @elek for sharing your thoughts. I also had the same doubt: most of the security-related configs are empty by default. That's why I pinged @dineshchitlangia, the reporter of this task, for review, in the hope that we can clarify the intended scope.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-759219634
@symious is it possible to attach a screenshot for the output of this command flag.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-826623845
@elek Thanks for the review, I just found the other comment @dineshchitlangia mentioned above, let me add the "SECURITY" tag, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-831149552
Merging it now. Thanks the update @symious and the review @dineshchitlangia and @adoroszlai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] symious commented on pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
symious commented on pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#issuecomment-759225997
Sure. The screen shot is as follows.
<img width="583" alt="Screenshot 2021-01-13 at 2 00 19 PM" src="https://user-images.githubusercontent.com/14933944/104412724-d36d7f00-55a7-11eb-8fe0-b334e6dde5d4.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia commented on a change in pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on a change in pull request #1788:
URL: https://github.com/apache/ozone/pull/1788#discussion_r573877934
##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genconf/GenerateOzoneRequiredConfigurations.java
##########
@@ -61,6 +62,9 @@
description = "Directory path where ozone-site file should be generated.")
private String path;
+ @Option(names = "--security", description = "Generate security config.")
Review comment:
```suggestion
@Option(names = "--security", description = "Generates security config template, update template with proper values before use.")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek merged pull request #1788: HDDS-2212. Genconf tool should generate config files for secure clust…
Posted by GitBox <gi...@apache.org>.
elek merged pull request #1788:
URL: https://github.com/apache/ozone/pull/1788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org