You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by "Lionel, Liu" <bh...@163.com> on 2019/01/09 11:28:45 UTC

RE: Griffin - how to configure authentication for ElasticSearchentity/http request

Hi Prachi and Selvaraj,

For the authentication of http requests like ES and livy, some other users also had such problems, we’re planning to fix them.
For the ES authentication request, it would affect in both write and read phase. 
There’s a PR to solve it in write phase, not merged yet, but maybe it can help you.
https://github.com/apache/griffin/pull/463


Thanks
Lionel, Liu

From: Prachi Kishore Hunnargikar
Sent: 2019年1月9日 16:22
To: dev@griffin.apache.org
Cc: Selvaraj K
Subject: RE: Griffin - how to configure authentication for ElasticSearchentity/http request

Hello Griffin community!



Has anyone tried configuring authentication for ElasticSearch entity/http request? We are testing Griffin on Azure HDInsight Spark cluster.



Problem


Solution


Code affected


Griffin application won’t start or even if started all jobs go in “DEAD” state


401 post to Livy error: Credentials missing or Authentication is not basic


It was failing at this line in
String resultStr = restTemplate.getForObject(uri, String.class);

JobServiceImpl.java



We added the interceptors like below and it started working:
final List<ClientHttpRequestInterceptor> interceptors = new ArrayList<ClientHttpRequestInterceptor>();
interceptors.add( new BasicAuthInterceptor( "xxx", "xxx" ) );
restTemplate.setInterceptors( interceptors );




service\src\main\java\org\apache\

griffin\core\job\JobServiceImpl.java

It was failing in SparkSubmitJob.java:
result = restTemplate.postForObject(livyUri,springEntity,String.class);

We added proper headers and it started working:

String auth = "xxx" + ":" + "xxx";
byte [] authentication = auth.getBytes();
byte[] base64Authentication = Base64Utils.encode(authentication);
String baseCredential = new String(base64Authentication);
headers.add(HttpHeaders.AUTHORIZATION, "Basic " + baseCredential);
headers.setAccept(Collections.singletonList(MediaType.TEXT_PLAIN));



service\src\main\java\org\apache\

griffin\core\job\SparkSubmitJob.java


Jobs run successfully but no metrics /graph is displayed upon clicking either “DQ Metrics, My Dashboard or Health”


ERROR 11173 --- [nio-8090-exec-3] o.a.g.c.m.MetricServiceImpl              : Failed to get metric values named TestJob. null

It is failing at: MetricStoreImpl.java at:
HttpEntity entity = getHttpEntityForSearch(metricName, from, size,
        tmst);

OR:

    Response response = client.performRequest("GET", urlGet,
            Collections.emptyMap(), entity);

Can anyone help in how to solve this issue?


service\src\main\java\org\apache\

griffin\core\metric\MetricStoreImpl.java




Thanks and Regards

Prachi Hunnargikar & Selvaraj K