[jira] [Resolved] (GRIFFIN-290) Fix bug for submitting job to livy

["William Guo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GRIFFIN-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

William Guo resolved GRIFFIN-290.
---------------------------------
    Fix Version/s: 0.6.0
       Resolution: Fixed

Issue resolved by pull request 534
[https://github.com/apache/griffin/pull/534]

> Fix bug for submitting job to livy
> ----------------------------------
>
>                 Key: GRIFFIN-290
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-290
>             Project: Griffin
>          Issue Type: Task
>            Reporter: wan kun
>            Priority: Major
>             Fix For: 0.6.0
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> When griffin submit multiple DQ jobs to livy, the http parameter `name` is always griffin. 
>  So livy will reject them.
> job request :
>  
> {quote}[owner: null, request: [proxyUser: None, file: hdfs://nameservice-standby/user/kun.wan/measure-0.6.0-SNAPSHOT.jar, 
>  args: {
>  "spark" :
> Unknown macro: \{ "log.level" }
> ,
>  "sinks" : [
> Unknown macro: \{ .... }
> ],
>  "griffin.checkpoint" : [ ]
>  },{
>  "measure.type" : "griffin",
>  "id" : 5202,
>  "name" : "spu_null_check",
>  "owner" : "test",
>  "description" : "check null value for store and category",
>  "deleted" : false,
>  "timestamp" : 1568195100000,
>  "dq.type" : "PROFILING",
>  "sinks" : [ "ELASTICSEARCH", "HDFS" ],
>  "process.type" : "BATCH",
>  "rule.description" :
> ,
>  "data.sources" : [
> Unknown macro: \{ .... }
> ],
>  "evaluate.rule" :
> ,
>  "measure.type" : "griffin"
>  },raw,raw, driverMemory: 1g, executorMemory: 6g, executorCores: 2, numExecutors: 6, queue: root.users.kun_dot_wan, {color:#ff0000}*name: griffin*{color}]]
> {quote}
>  
> livy Response : 
> {quote}400 Bad Request
>  [Date:"Thu, 12 Sep 2019 10:00:00 GMT", Content-Type:"application/json;charset=utf-8", Content-Length:"47", Server:"Jetty(9.3.24.v20180605)"]
> {"msg":"Duplicate session name: Some(griffin)"}
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)