You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by "Lionel Liu (JIRA)" <ji...@apache.org> on 2018/09/05 01:51:00 UTC

[jira] [Commented] (GRIFFIN-189) Griffin - Livy error

    [ https://issues.apache.org/jira/browse/GRIFFIN-189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603814#comment-16603814 ] 

Lionel Liu commented on GRIFFIN-189:
------------------------------------

Hi, it seems like the submission of spark job failed by livy server error.
 # What environment are you working on? The docker we provided or the environment you've deployed yourself?
 # Have you configured "livy.uri" as the correct url like "http://<ip>:8998/batches"?
 # Does the livy server work as normal? You can have a simple test of it.
 # Can you submit the livy test from the griffin server machine?

> Griffin - Livy error
> --------------------
>
>                 Key: GRIFFIN-189
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-189
>             Project: Griffin (Incubating)
>          Issue Type: Bug
>    Affects Versions: 0.2.0-incubating
>            Reporter: Cory Woytasik
>            Priority: Major
>              Labels: beginner, newbie, usability
>
> We are trying to get griffin set up and after creating measure and jobs and letting them run we have noticed the results are not available via the DQ metrics link or metric link from the job itself.  We have noticed when the job gets submitted the following spark context and error message are generated.  We assume we must have a setting in one of the directories set incorrectly.  Thoughts?
>  
> INFO 20972 --- [ryBean_Worker-2] o.a.g.c.j.SparkSubmitJob                 : {
>   "measure.type" : "griffin",
>   "id" : 13,
>   "name" : "LineageAccuracy",
>   "owner" : "test",
>   "description" : "AccuracyTest",
>   "organization" : null,
>   "deleted" : false,
>   "timestamp" : 1535998320000,
>   "dq.type" : "accuracy",
>   "process.type" : "batch",
>   "data.sources" : [ {
>     "id" : 16,
>     "name" : "source",
>     "connectors" : [ {
>       "id" : 17,
>       "name" : "source1535741016027",
>       "type" : "HIVE",
>       "version" : "1.2",
>       "predicates" : [ ],
>       "data.unit" : "1day",
>       "config" : {
>         "database" : "default",
>         "table.name" : "lineage"
>       }
>     } ]
>   }, {
>     "id" : 18,
>     "name" : "target",
>     "connectors" : [ {
>       "id" : 19,
>       "name" : "target1535741022277",
>       "type" : "HIVE",
>       "version" : "1.2",
>       "predicates" : [ ],
>       "data.unit" : "1day",
>       "config" : {
>         "database" : "default",
>         "table.name" : "lineageload"
>       }
>     } ]
>   } ],
>   "evaluate.rule" : {
>     "id" : 14,
>     "rules" : [ {
>       "id" : 15,
>       "rule" : "source.asset=target.asset AND source.element=target.element AND source.elementtype=target.elementtype AND source.object=target.object AND source.objecttype=target.objecttype AND source.objectfield=target.objectfield AND source.sourceelement=target.sourceelement AND source.sourceobject=target.sourceobject AND source.sourcefield=target.sourcefield AND source.sourcefieldname=target.sourcefieldname AND source.transformationtext=target.transformationtext AND source.displayindicator=target.displayindicator",
>       "name" : "accuracy",
>       "dsl.type" : "griffin-dsl",
>       "dq.type" : "accuracy"
>     } ]
>   },
>   "measure.type" : "griffin"
> }
> {color:#FF0000}2018-09-04 13:12:00.752 ERROR 20972 --- [ryBean_Worker-2] o.a.g.c.j.SparkSubmitJob                 : Post to livy error. 500 Internal Server Error{color}
> [EL Fine]: sql: 2018-09-04 13:12:00.754--ClientSession(787879814)--Connection(1389579691)--UPDATE JOBINSTANCEBEAN SET predicate_job_deleted = ?, STATE = ? WHERE (ID = ?)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)