You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hawq.apache.org by janebeckman <gi...@git.apache.org> on 2017/10/10 21:43:30 UTC

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

GitHub user janebeckman opened a pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131

    Document extending KDC ticket interval

    Extending the ticket interval is needed for Kerberos Active Directory support. Also added back guc that was previously documented in PHD

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/janebeckman/incubator-hawq-docs feature/kerberos_update

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-hawq-docs/pull/131.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #131
    
----
commit 4892b35d82fecc331512c92442a84a61cfa9768d
Author: Jane Beckman <jb...@pivotal.io>
Date:   2017-10-06T23:05:08Z

    Updates based on preliminary AD config findings

commit 0af9ac32087248f9666363bca88863c48b425e7e
Author: Jane Beckman <jb...@pivotal.io>
Date:   2017-10-07T00:01:55Z

    Minor change to kdc.conf process

commit 517be285352d8bdc6bd37a661e8d0b2b945d7fa9
Author: Jane Beckman <jb...@pivotal.io>
Date:   2017-10-10T21:18:56Z

    Updates to section on changing ticket interval

----


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143874810

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

Also, call out `server_ticket_renewal_interval` as a HAWQ parameter here.

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143875135

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

+### <a id="change_ticket"></a>Changing the Ticket Renewal Interval
+
+The lifetime of the Kerberos ticket may need to be changed. The ticket lifetime is the minimum of the following values:
+
+* `max_life` in `kdc.conf` on the KDC servers.
+* `ticket_lifetime` in `krb5.conf` on the client
+* maxlife for the user principal
+* maxlife for the service principal in krbtgt\[REALM\]
+* maxlife for the AFS service principal "afs/[realm_in_lower_case]"
+* the requested lifetime in the ticket request
+
+**Note:** The kdc.conf file supplements krb5.conf for programs using KDC. The kdc.conf file contains defaults used when issuing Kerberos tickets, as well as KDC configuration information.
+
+On starting HAWQ, the Resource Manager initializes the kerberos ticket to expire after 12 hours. On KDC servers, this interval can be even longer. (Your specific configuration may differ from these standards, so set the ticket to renew before your system ticket lifetime.) Reset the `server_ticket_renew_interval` to renew prior to the default value and restart the cluster to have the new value take effect.
+
+You will perform different procedures if you use Ambari for cluster management or manage your cluster from the command line.
+
+ 1. If you manage your cluster using Ambari:
+
+ 1. Login to the Ambari UI from a supported web browser.
+
+ 2. Navigate to the **HAWQ** service, **Configs > Advanced** tab and expand the **Custom hawq-site** drop down.
+
+ 3. Set the value of `server_ticket_renew_interval` to renew prior to the default interval. For example set `server_ticket_renew_interval` to 18000000. This will renew the ticket every five hours.
+
+ 4. **Save** this configuration change and then select the now orange **Restart > Restart All Affected** menu button to restart your HAWQ cluster.
+
+ 5. Exit the Ambari UI.
+
+ 2. If you manage your cluster from the command line:
+
+ 1. On all KDC servers, open the KDC configuration file `/etc/krb5kdc/kdc.conf` and change `max_life` to a value less than the renewal interval.
--- End diff --

Verify with dev that this is something we want to document for all.

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by radarwave <gi...@git.apache.org>.

Github user radarwave commented on a diff in the pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143965363
  
    --- Diff: markdown/clientaccess/kerberos.html.md.erb ---
    @@ -697,3 +753,172 @@ Follow these steps to install and configure a Kerberos KDC server on a Red Hat E
         root@kdc-server$ /sbin/chkconfig krb5kdc on
         root@kdc-server$ /sbin/chkconfig kadmin on
         ```
    +    
    +### <a id="task_setup_kdc_ad"></a>Install and Configure KDC Server on Active Directory
    +
    +Follow these steps to install and configure a Kerberos KDC server on a Windows host with Active Directory. 
    +
    +1. Log into the Windows server as a user with administrator permissions. 
    +
    +2. From the **Start** menu, select **Control Panel** \> **Adminisrative Tools** \> **Active Directory Users and Computers**. 
    +
    +   Note: If there is no entry for **Active Directory Users and Computers**, Active Directory service may not have been correctly installed. 
    +
    +3. Go to the DATALOCAL entry in the directory tree of the **Active Directory Users and Computers** window and  right click on **Managed Service Accounts**. 
    +
    +4.  Select **New** \> **User**.
    +
    +5.  A **New Object - User** popup window will appear. In the **First name:** window, enter `gpadmin` and also enter `gpadnub` as the **User logon name**.
    +
    +6. Click **Next**. Create and confirm a password. Enable the checkbox for **Password never expires**. Click **Next**, then **Finish**. 
    +
    +7. Log into Windows as Administrator. Depending on your Windows environment, either open a Windows power shell or click **Start** and open a command prompt session and select **Run as Administrator** \> **Yes** to open an administrator window.
    +
    +8. Add a Service Principal Name (SPN) for the account just created:
    +
    +   ``` shell
    +   PS C:\Users\Administrator> setspn -A postgres/hdp3.example.com gpadmin
    +   ```
    +   
    +9. Generate a keytab file by using the `ktpass` command. This command takes the form:
    +
    +   ``` shell
    +   ktpass -princ postgres/<FQDN of HDB master> -pass <password> -mapuser <Managed Service Account name> -crypto ALL -ptype KRB5_NT_PRINCIPAL -out <keytab filename> -kvno 0 
    +   ```
    +   For example, to create `hdp3.keytab` for gpadmin on hdp3.mydomain.com@DATA.LOCAL:
    +   
    +   ``` shell
    +   PS C:\Users\Administrator> ktpass -princ postgres/hdp3.mydomain.com@DATA.LOCAL -pass abcd1234 -mapuser gpadmin -crypto ALL -ptype KRB5_NT_PRINCIPAL -out hdp3.keytab -kvno 0
    +   
    +   Targeting domain controller: WIN-TIH2EBEERUK.DATA.LOCAL
    +   
    +   Using legacy password setting method
    +   Key created.
    +   Key created.
    +   Key created.
    +   Key created.
    +   Key created.
    +   Output keytab to hdp3.keytab:
    +   Keytab version: 0x502
    +   keysize 61 postgres/hdp3.example.com@DATA.LOCAL ptype 1 (KRB5_NT_PRINCIPAL) vno 0 etype 0x1 (DES-CBC-CRC) keylength 8 (0xa8d0405789a8469e)
    +keysize 61 postgres/hdp3.example.com@DATA.LOCAL ptype 1 (KRB5_NT_PRINCIPAL) vno 0 etype 0x3 (DES-CBC-MD5) keylength 8 (0xa8d0405789a8469e)
    +keysize 69 postgres/hdp3.example.com@DATA.LOCAL ptype 1 (KRB5_NT_PRINCIPAL) vno 0 etype 0x17 (RC4-HMAC) keylength 16 (0x161cff084477fe596a5db81874498a24)
    +keysize 85 postgres/hdp3.example.com@DATA.LOCAL ptype 1 (KRB5_NT_PRINCIPAL) vno 0 etype 0x12 (AES256-SHA1) keylength 32 (0x20648bd82de77bf66a1dcac4b3050bc308f2cc38f4a13e814ad5bd30e67ef388)
    +keysize 69 postgres/hdp3.example.com@DATA.LOCAL ptype 1 (KRB5_NT_PRINCIPAL) vno 0 etype 0x11 (AES128-SHA1) keylength 16 (0xa026bb25aa495af5334cd4f185d33071):
    +   ```
    +   
    +10. Transfer the generated keytab file to the HDB master. For example:
    +
    +    ``` shell
    +    root@kdc-server$ scp /etc/ad_keytab/keytabs/hawq.service.keytab <master>:/etc/ad_keytab/keytabs/hawq.service.keytab
    +    ```
    +Change the ownership of the keytab file to `gpadmin:gpadmin` and the mode to `600`. 
    +
    +
    +    ``` shell
    +    root@kdc-server$ ssh <master> chown gpadmin:gpadmin /etc/ad_keytab/keytabs/hawq.service.keytab
    +    root@kdc-server$ ssh <master> chmod 600 /etc/ad_keytab/keytabs/hawq.service.keytab
    +    ```
    +    
    +***Can you use automated kerberos setup on Ambari?**
    +
    +11. Log into HAWQ through the administrator window and install the Kerberos packages on the HAWQ master. 
    +
    +12. Change the ticket renewal interval.
    +
    +13. Edit the `.etc/krb5.conf` configuration file to define the Kerberos realm for the cluster. 
    +
    +   **Can you set up with PS C:\Users\Administrator> ? What if you use Ambari? This page has commands for Powershell: 
    +   https://hortonworks.com/blog/enabling-kerberos-hdp-active-directory-integration/**
    --- End diff --
    
    We can remove reference of this link as it's kind of out of date and do not have too much valuable for our settings. 


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143874681

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

12 hours **by default**.

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by janebeckman <gi...@git.apache.org>.

Github user janebeckman commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143884263

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

Verify if it should be done for all KDC servers?

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143874973

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

Let's indicate the units up front.

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143873572
  
    --- Diff: markdown/clientaccess/kerberos.html.md.erb ---
    @@ -476,7 +476,8 @@ Perform the following steps to configure Kerberos authentication for specific HA
     
     ### <a id="hawq_kerb_dbaccess"></a>Authenticating User Access to HAWQ 
     
    -When Kerberos user authentication is enabled for HAWQ, users must request a ticket from the Kerberos KDC server before connecting to HAWQ. You must request the ticket for a principal matching the requested database user name. When granted, the ticket expires after a set period of time, after which you will need to request another ticket.
    +When Kerberos user authentication is enabled for HAWQ, users must request a ticket from the Kerberos KDC server before connecting to HAWQ. You must request the ticket for a principal matching the requested database user name. When granted, the ticket expires after a set period of time, after which you must either request another ticket or have the ticket renewed. The default expiration time is 12 hours, set in the parameter `server_ticket_renew_interval`. To avoid KDC access issues, the ticket should be renewed before it expires.
    --- End diff --
    
    Should probably add that the actual expiration default/configuration is kerberos software-dependent.


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143874518
  
    --- Diff: markdown/clientaccess/kerberos.html.md.erb ---
    @@ -515,6 +516,53 @@ Valid starting     Expires            Service principal
     
     After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.
     
    +### <a id="change_ticket"></a>Changing the Ticket Renewal Interval
    +
    +The lifetime of the Kerberos ticket may need to be changed. The ticket lifetime is the minimum of the following values:
    +
    +* `max_life` in `kdc.conf` on the KDC servers.
    --- End diff --
    
    Not sure if we need to provide this level of detail. It might be better to just leave this as kerberos implementation-specific.


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-hawq-docs/pull/131


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by radarwave <gi...@git.apache.org>.

Github user radarwave commented on a diff in the pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143955400
  
    --- Diff: markdown/clientaccess/kerberos.html.md.erb ---
    @@ -515,6 +516,53 @@ Valid starting     Expires            Service principal
     
     After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.
     
    +### <a id="change_ticket"></a>Changing the Ticket Renewal Interval
    +
    +The lifetime of the Kerberos ticket may need to be changed. The ticket lifetime is the minimum of the following values:
    +
    +* `max_life` in `kdc.conf` on the KDC servers.
    --- End diff --
    
    This is one of the issue user hits, and possible a common issue for integrate HAWQ with AD KDC, so we'd better keep it.
    
    But I think we don't need to talk details of how to setup '/etc/krb5kdc/kdc.conf', just doc how to set 'server_ticket_renew_interval' for HAWQ and explain why this change is needed should be enough.


---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by dyozie <gi...@git.apache.org>.

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143875192

--- Diff: markdown/clientaccess/kerberos.html.md.erb ---
@@ -515,6 +516,53 @@ Valid starting Expires Service principal

After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.

Again, mention units up front.

---

[GitHub] incubator-hawq-docs pull request #131: Document extending KDC ticket interva...

Posted by janebeckman <gi...@git.apache.org>.

Github user janebeckman commented on a diff in the pull request:

    https://github.com/apache/incubator-hawq-docs/pull/131#discussion_r143882368
  
    --- Diff: markdown/clientaccess/kerberos.html.md.erb ---
    @@ -515,6 +516,53 @@ Valid starting     Expires            Service principal
     
     After generating a ticket, you can connect to a HAWQ database as a kerberos-authenticated user using `psql` or other client programs.
     
    +### <a id="change_ticket"></a>Changing the Ticket Renewal Interval
    +
    +The lifetime of the Kerberos ticket may need to be changed. The ticket lifetime is the minimum of the following values:
    +
    +* `max_life` in `kdc.conf` on the KDC servers.
    --- End diff --
    
    Wen Lin wanted a section on the ticket lifetime and something like this.


---