You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Lav Jain <lj...@pivotal.io> on 2016/03/23 23:23:02 UTC
Review Request 45252: PXF service checks fails on secured cluster
without YARN
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/
-----------------------------------------------------------
Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
Bugs: AMBARI-15542
https://issues.apache.org/jira/browse/AMBARI-15542
Repository: ambari
Description
-------
PXF service checks fails on secured cluster
```
2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
2016-03-11 23:37:35,588 - Testing PXF HDFS read
2016-03-11 23:37:35,675 - Error code: 500
2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
```
Pxf log says
```
SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
```
Diffs
-----
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
Diff: https://reviews.apache.org/r/45252/diff/
Testing
-------
Tested manually with and without YARN, with and without security.
Thanks,
Lav Jain
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Goutam Tadi <go...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125417
-----------------------------------------------------------
Ship it!
Ship It!
- Goutam Tadi
On March 23, 2016, 10:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by bhuvnesh chaudhary <bc...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125472
-----------------------------------------------------------
Ship it!
Ship It!
- bhuvnesh chaudhary
On March 25, 2016, 8:50 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 25, 2016, 8:50 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Lav Jain <lj...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/
-----------------------------------------------------------
(Updated March 25, 2016, 8:50 p.m.)
Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
Bugs: AMBARI-15542
https://issues.apache.org/jira/browse/AMBARI-15542
Repository: ambari
Description
-------
PXF service checks fails on secured cluster
```
2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
2016-03-11 23:37:35,588 - Testing PXF HDFS read
2016-03-11 23:37:35,675 - Error code: 500
2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
```
Pxf log says
```
SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
```
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
Diff: https://reviews.apache.org/r/45252/diff/
Testing
-------
Tested manually with and without YARN, with and without security.
Thanks,
Lav Jain
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Lav Jain <lj...@pivotal.io>.
> On March 24, 2016, 1:18 a.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py, line 154
> > <https://reviews.apache.org/r/45252/diff/1/?file=1312544#file1312544line154>
> >
> > Does YARN have to be functional (green status), or just installed so that libraries are available?
YARN libraries are already present. It has to be configured (to populate yarn-site.xml).
- Lav
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125170
-----------------------------------------------------------
On March 23, 2016, 10:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Matt <mi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125170
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py (line 154)
<https://reviews.apache.org/r/45252/#comment187958>
Does YARN have to be functional (green status), or just installed so that libraries are available?
- Matt
On March 23, 2016, 3:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 3:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Matt <mi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125388
-----------------------------------------------------------
Ship it!
Ship It!
- Matt
On March 23, 2016, 3:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 3:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by Lav Jain <lj...@pivotal.io>.
> On March 24, 2016, 1:40 a.m., bhuvnesh chaudhary wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py, line 156
> > <https://reviews.apache.org/r/45252/diff/1/?file=1312544#file1312544line156>
> >
> > is it a test or pre-requisite to execute test ?
The test fails unless YARN is there. We are pre-catching the exception and giving a more meaningful message.
- Lav
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125175
-----------------------------------------------------------
On March 23, 2016, 10:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45252: PXF service checks fails on secured cluster
without YARN
Posted by bhuvnesh chaudhary <bc...@pivotal.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125175
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py (line 156)
<https://reviews.apache.org/r/45252/#comment187960>
is it a test or pre-requisite to execute test ?
- bhuvnesh chaudhary
On March 23, 2016, 10:23 p.m., Lav Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True, 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs': 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local': '/usr/bin/kinit', 'principal_name': 'hdfs-phd@AMBARI.APACHE.ORG', 'user': 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test: PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py eed2d18
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> -------
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>