You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by "Lionel, Liu" <bh...@163.com> on 2019/01/09 11:28:45 UTC
RE: Griffin - how to configure authentication for
ElasticSearchentity/http request
Hi Prachi and Selvaraj,
For the authentication of http requests like ES and livy, some other users also had such problems, we’re planning to fix them.
For the ES authentication request, it would affect in both write and read phase.
There’s a PR to solve it in write phase, not merged yet, but maybe it can help you.
https://github.com/apache/griffin/pull/463
Thanks
Lionel, Liu
From: Prachi Kishore Hunnargikar
Sent: 2019年1月9日 16:22
To: dev@griffin.apache.org
Cc: Selvaraj K
Subject: RE: Griffin - how to configure authentication for ElasticSearchentity/http request
Hello Griffin community!
Has anyone tried configuring authentication for ElasticSearch entity/http request? We are testing Griffin on Azure HDInsight Spark cluster.
Problem
Solution
Code affected
Griffin application won’t start or even if started all jobs go in “DEAD” state
401 post to Livy error: Credentials missing or Authentication is not basic
It was failing at this line in
String resultStr = restTemplate.getForObject(uri, String.class);
JobServiceImpl.java
We added the interceptors like below and it started working:
final List<ClientHttpRequestInterceptor> interceptors = new ArrayList<ClientHttpRequestInterceptor>();
interceptors.add( new BasicAuthInterceptor( "xxx", "xxx" ) );
restTemplate.setInterceptors( interceptors );
service\src\main\java\org\apache\
griffin\core\job\JobServiceImpl.java
It was failing in SparkSubmitJob.java:
result = restTemplate.postForObject(livyUri,springEntity,String.class);
We added proper headers and it started working:
String auth = "xxx" + ":" + "xxx";
byte [] authentication = auth.getBytes();
byte[] base64Authentication = Base64Utils.encode(authentication);
String baseCredential = new String(base64Authentication);
headers.add(HttpHeaders.AUTHORIZATION, "Basic " + baseCredential);
headers.setAccept(Collections.singletonList(MediaType.TEXT_PLAIN));
service\src\main\java\org\apache\
griffin\core\job\SparkSubmitJob.java
Jobs run successfully but no metrics /graph is displayed upon clicking either “DQ Metrics, My Dashboard or Health”
ERROR 11173 --- [nio-8090-exec-3] o.a.g.c.m.MetricServiceImpl : Failed to get metric values named TestJob. null
It is failing at: MetricStoreImpl.java at:
HttpEntity entity = getHttpEntityForSearch(metricName, from, size,
tmst);
OR:
Response response = client.performRequest("GET", urlGet,
Collections.emptyMap(), entity);
Can anyone help in how to solve this issue?
service\src\main\java\org\apache\
griffin\core\metric\MetricStoreImpl.java
Thanks and Regards
Prachi Hunnargikar & Selvaraj K