You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by lk_hadoop <lk...@163.com> on 2018/09/13 05:28:48 UTC

LineageLogger can't getEdges

hi,all:
    I want to change LineageLogger code to output more log info . 
    I create a new mvn project and cp LineageLogger code to my own class named MyLineageLogger. Then I add my code to output more log info ,last I build a new jar. but when I config it to <hive.exec.post.hooks> ,the log file didn't contain any edges info :  "edges":[],"vertices":[]  

   I've tried to use hive's own LineageLogger with the same sql , and all is ok.
   and if I config both hive's own LineageLogger  and my MyLineageLogger  to the <hive.exec.post.hooks> , both LineageLogger  and MyLineageLogger  can out put full lineage log.

   I note that ,if I only config MyLineageLogger  , LinkedHashMap<String, ObjectPair<SelectOperator,org.apache.hadoop.hive.ql.metadata.Table>> finalSelOps = index.getFinalSelectOps();
   will return 0
   
can anybody give me some clue , why this happen ? thanks...


2018-09-13


lk_hadoop

Re: Re: LineageLogger can't getEdges

Posted by lk_hadoop <lk...@163.com>.
is this the reason ? 

code in Optimizer.java
if (postExecHooks.contains("org.apache.hadoop.hive.ql.hooks.PostExecutePrinter")
        || postExecHooks.contains("org.apache.hadoop.hive.ql.hooks.LineageLogger")) {
      transformations.add(new Generator());
    }



2018-09-13 

lk_hadoop 



发件人："lk_hadoop"<lk...@163.com>
发送时间：2018-09-13 16:53
主题：Re: LineageLogger can't getEdges
收件人："user"<us...@hive.apache.org>
抄送：

I write a log on org.apache.hadoop.hive.ql.Driver  :  LOG.info("Driver:1866 : " + hookContext.getIndex().getFinalSelectOps().size());



if I config LineageLogger to <hive.exec.post.hooks> the log will out put 1 or it will out put 0 , why ??


2018-09-13 

lk_hadoop 



发件人："lk_hadoop"<lk...@163.com>
发送时间：2018-09-13 13:28
主题：LineageLogger can't getEdges
收件人："user"<us...@hive.apache.org>
抄送：

hi,all:
    I want to change LineageLogger code to output more log info . 
    I create a new mvn project and cp LineageLogger code to my own class named MyLineageLogger. Then I add my code to output more log info ,last I build a new jar. but when I config it to <hive.exec.post.hooks> ,the log file didn't contain any edges info :  "edges":[],"vertices":[]  

   I've tried to use hive's own LineageLogger with the same sql , and all is ok.
   and if I config both hive's own LineageLogger  and my MyLineageLogger  to the <hive.exec.post.hooks> , both LineageLogger  and MyLineageLogger  can out put full lineage log.

   I note that ,if I only config MyLineageLogger  , LinkedHashMap<String, ObjectPair<SelectOperator,org.apache.hadoop.hive.ql.metadata.Table>> finalSelOps = index.getFinalSelectOps();
   will return 0
   
can anybody give me some clue , why this happen ? thanks...


2018-09-13


lk_hadoop

Re: LineageLogger can't getEdges

Posted by lk_hadoop <lk...@163.com>.
I write a log on org.apache.hadoop.hive.ql.Driver  :  LOG.info("Driver:1866 : " + hookContext.getIndex().getFinalSelectOps().size());



if I config LineageLogger to <hive.exec.post.hooks> the log will out put 1 or it will out put 0 , why ??


2018-09-13 

lk_hadoop 



发件人："lk_hadoop"<lk...@163.com>
发送时间：2018-09-13 13:28
主题：LineageLogger can't getEdges
收件人："user"<us...@hive.apache.org>
抄送：

hi,all:
    I want to change LineageLogger code to output more log info . 
    I create a new mvn project and cp LineageLogger code to my own class named MyLineageLogger. Then I add my code to output more log info ,last I build a new jar. but when I config it to <hive.exec.post.hooks> ,the log file didn't contain any edges info :  "edges":[],"vertices":[]  

   I've tried to use hive's own LineageLogger with the same sql , and all is ok.
   and if I config both hive's own LineageLogger  and my MyLineageLogger  to the <hive.exec.post.hooks> , both LineageLogger  and MyLineageLogger  can out put full lineage log.

   I note that ,if I only config MyLineageLogger  , LinkedHashMap<String, ObjectPair<SelectOperator,org.apache.hadoop.hive.ql.metadata.Table>> finalSelOps = index.getFinalSelectOps();
   will return 0
   
can anybody give me some clue , why this happen ? thanks...


2018-09-13


lk_hadoop