You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Andrea Santurbano <sa...@gmail.com> on 2019/10/06 20:08:52 UTC

KSQL interpreter

Hi guys,
I'm building a KQL interpreter. KSQL is the streaming SQL engine that
enables real-time data processing against Apache Kafka topics.
I created an interpreter that leverages the InterpreterOutputStream class
in order to stream the result from the backend to the frontend, but I get
the following representation in the frontend:

https://imgur.com/a/J7unk25

What am I doing wrong?

Following the Interpreter implementation:

https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java

Thanks a lot

Andrea

Re: KSQL interpreter

Posted by Jeff Zhang <zj...@gmail.com>.
Awesome, @Alex & @Andrea, let me know if you need any help on this.

Alex Ott <al...@gmail.com> 于2019年10月12日周六 上午1:06写道:

> If you have time next week, let's discuss how we could merge things
> together...
>
> On Fri, Oct 11, 2019 at 6:34 PM Alex Ott <al...@gmail.com> wrote:
>
> > Hello Andrea
> >
> > I've started KSQL interpreter for a long time - it's linked to
> > corresponding JIRA (ZEPPELIN-3548):
> > https://github.com/alexott/zeppelin/tree/ZEPPELIN-3548-KSQL
> >
> > although it doesn't support OutputStream yet
> >
> > Andrea Santurbano  at "Sun, 6 Oct 2019 22:08:52 +0200" wrote:
> >  AS> Hi guys,
> >  AS> I'm building a KQL interpreter. KSQL is the streaming SQL engine
> that
> >  AS> enables real-time data processing against Apache Kafka topics.
> >  AS> I created an interpreter that leverages the InterpreterOutputStream
> > class
> >  AS> in order to stream the result from the backend to the frontend, but
> I
> > get
> >  AS> the following representation in the frontend:
> >
> >  AS> https://imgur.com/a/J7unk25
> >
> >  AS> What am I doing wrong?
> >
> >  AS> Following the Interpreter implementation:
> >
> >  AS>
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> >
> >  AS> Thanks a lot
> >
> >  AS> Andrea
> >
> >
> > --
> > With best wishes,                    Alex Ott
> > http://alexott.net/
> > Twitter: alexott_en (English), alexott (Russian)
> >
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>


-- 
Best Regards

Jeff Zhang

Re: KSQL interpreter

Posted by Alex Ott <al...@gmail.com>.
If you have time next week, let's discuss how we could merge things
together...

On Fri, Oct 11, 2019 at 6:34 PM Alex Ott <al...@gmail.com> wrote:

> Hello Andrea
>
> I've started KSQL interpreter for a long time - it's linked to
> corresponding JIRA (ZEPPELIN-3548):
> https://github.com/alexott/zeppelin/tree/ZEPPELIN-3548-KSQL
>
> although it doesn't support OutputStream yet
>
> Andrea Santurbano  at "Sun, 6 Oct 2019 22:08:52 +0200" wrote:
>  AS> Hi guys,
>  AS> I'm building a KQL interpreter. KSQL is the streaming SQL engine that
>  AS> enables real-time data processing against Apache Kafka topics.
>  AS> I created an interpreter that leverages the InterpreterOutputStream
> class
>  AS> in order to stream the result from the backend to the frontend, but I
> get
>  AS> the following representation in the frontend:
>
>  AS> https://imgur.com/a/J7unk25
>
>  AS> What am I doing wrong?
>
>  AS> Following the Interpreter implementation:
>
>  AS>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>
>  AS> Thanks a lot
>
>  AS> Andrea
>
>
> --
> With best wishes,                    Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>


-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)

Re: KSQL interpreter

Posted by Alex Ott <al...@gmail.com>.
Hello Andrea

I've started KSQL interpreter for a long time - it's linked to
corresponding JIRA (ZEPPELIN-3548):
https://github.com/alexott/zeppelin/tree/ZEPPELIN-3548-KSQL

although it doesn't support OutputStream yet

Andrea Santurbano  at "Sun, 6 Oct 2019 22:08:52 +0200" wrote:
 AS> Hi guys,
 AS> I'm building a KQL interpreter. KSQL is the streaming SQL engine that
 AS> enables real-time data processing against Apache Kafka topics.
 AS> I created an interpreter that leverages the InterpreterOutputStream class
 AS> in order to stream the result from the backend to the frontend, but I get
 AS> the following representation in the frontend:

 AS> https://imgur.com/a/J7unk25

 AS> What am I doing wrong?

 AS> Following the Interpreter implementation:

 AS> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java

 AS> Thanks a lot

 AS> Andrea


-- 
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
I had the time to check into the code.
I fixed the bug please look into this:
https://github.com/apache/zeppelin/pull/3477

Il giorno mar 8 ott 2019 alle ore 15:25 Jeff Zhang <zj...@gmail.com> ha
scritto:

> Thanks for reporting this issue.
>
> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午6:55写道:
>
> > It's a bug in the visualization (I guess), also the example provided
> here:
> >
> > https://github.com/apache/zeppelin/pull/2809
> >
> > is not working anymore
> >
> > I opened this issue:
> >
> > https://issues.apache.org/jira/browse/ZEPPELIN-4367
> >
> > If someone can mentor me on how to fix this, I can do it.
> > Thanks a lot
> > Andrea
> >
> > Il giorno mar 8 ott 2019 alle ore 12:06 Andrea Santurbano <
> > santand@gmail.com>
> > ha scritto:
> >
> > > Following the Interpreter log (seems to be good):
> > >
> > >
> >
> https://gist.github.com/conker84/aad9b9a220e781b30d5aab1ddc74a4d7#file-zeppelin-interpreter-debug-log
> > >
> > > Il giorno mar 8 ott 2019 alle ore 11:38 Andrea Santurbano <
> > > santand@gmail.com> ha scritto:
> > >
> > >> Seems to be different, please check here:
> > >>
> > >> https://gist.github.com/conker84/4a54df4c73dab310cdc9284f15e90075
> > >>
> > >> Can you mentor me about what I'm wrong?
> > >>
> > >>
> > >>
> > >> Il giorno mar 8 ott 2019 alle ore 11:13 Jeff Zhang <zj...@gmail.com>
> > ha
> > >> scritto:
> > >>
> > >>> The the following line is suspicious to me.
> > >>>
> > >>>
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120
> > >>>
> > >>> Could you add the following line to log4j.properties so that you can
> > see
> > >>> more logging info.
> > >>>      log4j.logger.org.apache.zeppelin.interpreter=DEBUG
> > >>>
> > >>> Then you should be able to see logging like following in the
> > interpreter
> > >>> log. That means interpreter generate output and sent it to zeppelin
> > >>> server.
> > >>> If you don't see this kind of logging, then it should be your
> > interpreter
> > >>> issue.
> > >>>
> > >>> DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
> > >>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> > >>> resourcePoolGetAll from ZeppelinServer
> > >>> DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
> > >>>
> > >>> DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index
> > 7:
> > >>> DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8
> > >>>
> > >>> DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
> > >>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> > >>> resourcePoolGetAll from ZeppelinServer
> > >>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
> > >>>
> > >>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index
> > 8:
> > >>> DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
> > >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9
> > >>>
> > >>>
> > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:
> > >>>
> > >>> > Did you check the animated gif that I provided in the first email?
> > >>> Looks
> > >>> > like a problem in the visualization.
> > >>> >
> > >>> > Btw I expect the same behaviour as here:
> > >>> >
> > >>> > https://github.com/apache/zeppelin/pull/2809
> > >>> > and
> > >>> > https://github.com/apache/zeppelin/pull/2873
> > >>> >
> > >>> >
> > >>> >
> > >>> > Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <
> zjffdu@gmail.com
> > >
> > >>> ha
> > >>> > scritto:
> > >>> >
> > >>> > > I encounter some issues when integrating flink streaming, not
> sure
> > >>> the
> > >>> > same
> > >>> > > as you.
> > >>> > > Just curious to know what is your expected output if it is
> > streaming
> > >>> job
> > >>> > > and output is generated continuously.
> > >>> > >
> > >>> > >
> > >>> > > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
> > >>> > >
> > >>> > > > No clue at all?
> > >>> > > >
> > >>> > > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> > >>> > > > santand@gmail.com>
> > >>> > > > ha scritto:
> > >>> > > >
> > >>> > > > > Could it be a bug in the front-end? Or Am I missing
> something?
> > >>> > > > >
> > >>> > > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> > >>> > > > > santand@gmail.com> ha scritto:
> > >>> > > > >
> > >>> > > > >> More or less yes (it transforms sql statements into
> > KafkaStreams
> > >>> > > jobs).
> > >>> > > > >> But I simply call a rest endpoint that has as response
> > >>> > > > Transfer-Encoding:
> > >>> > > > >> chunked and the data comes in as you can check with the
> linked
> > >>> > image.
> > >>> > > > >> The only thing it's that I have blank rows (but if I debug
> the
> > >>> code
> > >>> > > the
> > >>> > > > >> data is in there) and when I stop the job every blank row
> gets
> > >>> the
> > >>> > > > correct
> > >>> > > > >> value.
> > >>> > > > >>
> > >>> > > > >>
> > >>> > > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
> > >>> > zjffdu@gmail.com>
> > >>> > > > ha
> > >>> > > > >> scritto:
> > >>> > > > >>
> > >>> > > > >>> I don't know much about KSQL, is each running a streaming
> job
> > >>> which
> > >>> > > > never
> > >>> > > > >>> stops ?
> > >>> > > > >>>
> > >>> > > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一
> > 上午4:09写道:
> > >>> > > > >>>
> > >>> > > > >>> > Hi guys,
> > >>> > > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL
> > >>> engine
> > >>> > > that
> > >>> > > > >>> > enables real-time data processing against Apache Kafka
> > >>> topics.
> > >>> > > > >>> > I created an interpreter that leverages the
> > >>> > InterpreterOutputStream
> > >>> > > > >>> class
> > >>> > > > >>> > in order to stream the result from the backend to the
> > >>> frontend,
> > >>> > > but I
> > >>> > > > >>> get
> > >>> > > > >>> > the following representation in the frontend:
> > >>> > > > >>> >
> > >>> > > > >>> > https://imgur.com/a/J7unk25
> > >>> > > > >>> >
> > >>> > > > >>> > What am I doing wrong?
> > >>> > > > >>> >
> > >>> > > > >>> > Following the Interpreter implementation:
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> > >>> > > > >>> >
> > >>> > > > >>> > Thanks a lot
> > >>> > > > >>> >
> > >>> > > > >>> > Andrea
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > > >>>
> > >>> > > > >>> --
> > >>> > > > >>> Best Regards
> > >>> > > > >>>
> > >>> > > > >>> Jeff Zhang
> > >>> > > > >>>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > > Best Regards
> > >>> > >
> > >>> > > Jeff Zhang
> > >>> > >
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>> Best Regards
> > >>>
> > >>> Jeff Zhang
> > >>>
> > >>
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: KSQL interpreter

Posted by Jeff Zhang <zj...@gmail.com>.
Thanks for reporting this issue.

Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午6:55写道:

> It's a bug in the visualization (I guess), also the example provided here:
>
> https://github.com/apache/zeppelin/pull/2809
>
> is not working anymore
>
> I opened this issue:
>
> https://issues.apache.org/jira/browse/ZEPPELIN-4367
>
> If someone can mentor me on how to fix this, I can do it.
> Thanks a lot
> Andrea
>
> Il giorno mar 8 ott 2019 alle ore 12:06 Andrea Santurbano <
> santand@gmail.com>
> ha scritto:
>
> > Following the Interpreter log (seems to be good):
> >
> >
> https://gist.github.com/conker84/aad9b9a220e781b30d5aab1ddc74a4d7#file-zeppelin-interpreter-debug-log
> >
> > Il giorno mar 8 ott 2019 alle ore 11:38 Andrea Santurbano <
> > santand@gmail.com> ha scritto:
> >
> >> Seems to be different, please check here:
> >>
> >> https://gist.github.com/conker84/4a54df4c73dab310cdc9284f15e90075
> >>
> >> Can you mentor me about what I'm wrong?
> >>
> >>
> >>
> >> Il giorno mar 8 ott 2019 alle ore 11:13 Jeff Zhang <zj...@gmail.com>
> ha
> >> scritto:
> >>
> >>> The the following line is suspicious to me.
> >>>
> >>>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120
> >>>
> >>> Could you add the following line to log4j.properties so that you can
> see
> >>> more logging info.
> >>>      log4j.logger.org.apache.zeppelin.interpreter=DEBUG
> >>>
> >>> Then you should be able to see logging like following in the
> interpreter
> >>> log. That means interpreter generate output and sent it to zeppelin
> >>> server.
> >>> If you don't see this kind of logging, then it should be your
> interpreter
> >>> issue.
> >>>
> >>> DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
> >>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> >>> resourcePoolGetAll from ZeppelinServer
> >>> DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
> >>>
> >>> DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index
> 7:
> >>> DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8
> >>>
> >>> DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
> >>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> >>> resourcePoolGetAll from ZeppelinServer
> >>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
> >>>
> >>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index
> 8:
> >>> DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
> >>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9
> >>>
> >>>
> >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:
> >>>
> >>> > Did you check the animated gif that I provided in the first email?
> >>> Looks
> >>> > like a problem in the visualization.
> >>> >
> >>> > Btw I expect the same behaviour as here:
> >>> >
> >>> > https://github.com/apache/zeppelin/pull/2809
> >>> > and
> >>> > https://github.com/apache/zeppelin/pull/2873
> >>> >
> >>> >
> >>> >
> >>> > Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zjffdu@gmail.com
> >
> >>> ha
> >>> > scritto:
> >>> >
> >>> > > I encounter some issues when integrating flink streaming, not sure
> >>> the
> >>> > same
> >>> > > as you.
> >>> > > Just curious to know what is your expected output if it is
> streaming
> >>> job
> >>> > > and output is generated continuously.
> >>> > >
> >>> > >
> >>> > > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
> >>> > >
> >>> > > > No clue at all?
> >>> > > >
> >>> > > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> >>> > > > santand@gmail.com>
> >>> > > > ha scritto:
> >>> > > >
> >>> > > > > Could it be a bug in the front-end? Or Am I missing something?
> >>> > > > >
> >>> > > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> >>> > > > > santand@gmail.com> ha scritto:
> >>> > > > >
> >>> > > > >> More or less yes (it transforms sql statements into
> KafkaStreams
> >>> > > jobs).
> >>> > > > >> But I simply call a rest endpoint that has as response
> >>> > > > Transfer-Encoding:
> >>> > > > >> chunked and the data comes in as you can check with the linked
> >>> > image.
> >>> > > > >> The only thing it's that I have blank rows (but if I debug the
> >>> code
> >>> > > the
> >>> > > > >> data is in there) and when I stop the job every blank row gets
> >>> the
> >>> > > > correct
> >>> > > > >> value.
> >>> > > > >>
> >>> > > > >>
> >>> > > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
> >>> > zjffdu@gmail.com>
> >>> > > > ha
> >>> > > > >> scritto:
> >>> > > > >>
> >>> > > > >>> I don't know much about KSQL, is each running a streaming job
> >>> which
> >>> > > > never
> >>> > > > >>> stops ?
> >>> > > > >>>
> >>> > > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一
> 上午4:09写道:
> >>> > > > >>>
> >>> > > > >>> > Hi guys,
> >>> > > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL
> >>> engine
> >>> > > that
> >>> > > > >>> > enables real-time data processing against Apache Kafka
> >>> topics.
> >>> > > > >>> > I created an interpreter that leverages the
> >>> > InterpreterOutputStream
> >>> > > > >>> class
> >>> > > > >>> > in order to stream the result from the backend to the
> >>> frontend,
> >>> > > but I
> >>> > > > >>> get
> >>> > > > >>> > the following representation in the frontend:
> >>> > > > >>> >
> >>> > > > >>> > https://imgur.com/a/J7unk25
> >>> > > > >>> >
> >>> > > > >>> > What am I doing wrong?
> >>> > > > >>> >
> >>> > > > >>> > Following the Interpreter implementation:
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> >>> > > > >>> >
> >>> > > > >>> > Thanks a lot
> >>> > > > >>> >
> >>> > > > >>> > Andrea
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > > >>>
> >>> > > > >>> --
> >>> > > > >>> Best Regards
> >>> > > > >>>
> >>> > > > >>> Jeff Zhang
> >>> > > > >>>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Best Regards
> >>> > >
> >>> > > Jeff Zhang
> >>> > >
> >>> >
> >>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>>
> >>
>


-- 
Best Regards

Jeff Zhang

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
It's a bug in the visualization (I guess), also the example provided here:

https://github.com/apache/zeppelin/pull/2809

is not working anymore

I opened this issue:

https://issues.apache.org/jira/browse/ZEPPELIN-4367

If someone can mentor me on how to fix this, I can do it.
Thanks a lot
Andrea

Il giorno mar 8 ott 2019 alle ore 12:06 Andrea Santurbano <sa...@gmail.com>
ha scritto:

> Following the Interpreter log (seems to be good):
>
> https://gist.github.com/conker84/aad9b9a220e781b30d5aab1ddc74a4d7#file-zeppelin-interpreter-debug-log
>
> Il giorno mar 8 ott 2019 alle ore 11:38 Andrea Santurbano <
> santand@gmail.com> ha scritto:
>
>> Seems to be different, please check here:
>>
>> https://gist.github.com/conker84/4a54df4c73dab310cdc9284f15e90075
>>
>> Can you mentor me about what I'm wrong?
>>
>>
>>
>> Il giorno mar 8 ott 2019 alle ore 11:13 Jeff Zhang <zj...@gmail.com> ha
>> scritto:
>>
>>> The the following line is suspicious to me.
>>>
>>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120
>>>
>>> Could you add the following line to log4j.properties so that you can see
>>> more logging info.
>>>      log4j.logger.org.apache.zeppelin.interpreter=DEBUG
>>>
>>> Then you should be able to see logging like following in the interpreter
>>> log. That means interpreter generate output and sent it to zeppelin
>>> server.
>>> If you don't see this kind of logging, then it should be your interpreter
>>> issue.
>>>
>>> DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
>>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
>>> resourcePoolGetAll from ZeppelinServer
>>> DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>>>
>>> DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 7:
>>> DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8
>>>
>>> DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
>>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
>>> resourcePoolGetAll from ZeppelinServer
>>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>>>
>>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 8:
>>> DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
>>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9
>>>
>>>
>>> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:
>>>
>>> > Did you check the animated gif that I provided in the first email?
>>> Looks
>>> > like a problem in the visualization.
>>> >
>>> > Btw I expect the same behaviour as here:
>>> >
>>> > https://github.com/apache/zeppelin/pull/2809
>>> > and
>>> > https://github.com/apache/zeppelin/pull/2873
>>> >
>>> >
>>> >
>>> > Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zj...@gmail.com>
>>> ha
>>> > scritto:
>>> >
>>> > > I encounter some issues when integrating flink streaming, not sure
>>> the
>>> > same
>>> > > as you.
>>> > > Just curious to know what is your expected output if it is streaming
>>> job
>>> > > and output is generated continuously.
>>> > >
>>> > >
>>> > > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
>>> > >
>>> > > > No clue at all?
>>> > > >
>>> > > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
>>> > > > santand@gmail.com>
>>> > > > ha scritto:
>>> > > >
>>> > > > > Could it be a bug in the front-end? Or Am I missing something?
>>> > > > >
>>> > > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
>>> > > > > santand@gmail.com> ha scritto:
>>> > > > >
>>> > > > >> More or less yes (it transforms sql statements into KafkaStreams
>>> > > jobs).
>>> > > > >> But I simply call a rest endpoint that has as response
>>> > > > Transfer-Encoding:
>>> > > > >> chunked and the data comes in as you can check with the linked
>>> > image.
>>> > > > >> The only thing it's that I have blank rows (but if I debug the
>>> code
>>> > > the
>>> > > > >> data is in there) and when I stop the job every blank row gets
>>> the
>>> > > > correct
>>> > > > >> value.
>>> > > > >>
>>> > > > >>
>>> > > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
>>> > zjffdu@gmail.com>
>>> > > > ha
>>> > > > >> scritto:
>>> > > > >>
>>> > > > >>> I don't know much about KSQL, is each running a streaming job
>>> which
>>> > > > never
>>> > > > >>> stops ?
>>> > > > >>>
>>> > > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
>>> > > > >>>
>>> > > > >>> > Hi guys,
>>> > > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL
>>> engine
>>> > > that
>>> > > > >>> > enables real-time data processing against Apache Kafka
>>> topics.
>>> > > > >>> > I created an interpreter that leverages the
>>> > InterpreterOutputStream
>>> > > > >>> class
>>> > > > >>> > in order to stream the result from the backend to the
>>> frontend,
>>> > > but I
>>> > > > >>> get
>>> > > > >>> > the following representation in the frontend:
>>> > > > >>> >
>>> > > > >>> > https://imgur.com/a/J7unk25
>>> > > > >>> >
>>> > > > >>> > What am I doing wrong?
>>> > > > >>> >
>>> > > > >>> > Following the Interpreter implementation:
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>>> > > > >>> >
>>> > > > >>> > Thanks a lot
>>> > > > >>> >
>>> > > > >>> > Andrea
>>> > > > >>> >
>>> > > > >>>
>>> > > > >>>
>>> > > > >>> --
>>> > > > >>> Best Regards
>>> > > > >>>
>>> > > > >>> Jeff Zhang
>>> > > > >>>
>>> > > > >>
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > > Best Regards
>>> > >
>>> > > Jeff Zhang
>>> > >
>>> >
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
Following the Interpreter log (seems to be good):
https://gist.github.com/conker84/aad9b9a220e781b30d5aab1ddc74a4d7#file-zeppelin-interpreter-debug-log

Il giorno mar 8 ott 2019 alle ore 11:38 Andrea Santurbano <sa...@gmail.com>
ha scritto:

> Seems to be different, please check here:
>
> https://gist.github.com/conker84/4a54df4c73dab310cdc9284f15e90075
>
> Can you mentor me about what I'm wrong?
>
>
>
> Il giorno mar 8 ott 2019 alle ore 11:13 Jeff Zhang <zj...@gmail.com> ha
> scritto:
>
>> The the following line is suspicious to me.
>>
>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120
>>
>> Could you add the following line to log4j.properties so that you can see
>> more logging info.
>>      log4j.logger.org.apache.zeppelin.interpreter=DEBUG
>>
>> Then you should be able to see logging like following in the interpreter
>> log. That means interpreter generate output and sent it to zeppelin
>> server.
>> If you don't see this kind of logging, then it should be your interpreter
>> issue.
>>
>> DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
>> resourcePoolGetAll from ZeppelinServer
>> DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>>
>> DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 7:
>> DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8
>>
>> DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
>> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
>> resourcePoolGetAll from ZeppelinServer
>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>>
>> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 8:
>> DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
>> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9
>>
>>
>> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:
>>
>> > Did you check the animated gif that I provided in the first email? Looks
>> > like a problem in the visualization.
>> >
>> > Btw I expect the same behaviour as here:
>> >
>> > https://github.com/apache/zeppelin/pull/2809
>> > and
>> > https://github.com/apache/zeppelin/pull/2873
>> >
>> >
>> >
>> > Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zj...@gmail.com>
>> ha
>> > scritto:
>> >
>> > > I encounter some issues when integrating flink streaming, not sure the
>> > same
>> > > as you.
>> > > Just curious to know what is your expected output if it is streaming
>> job
>> > > and output is generated continuously.
>> > >
>> > >
>> > > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
>> > >
>> > > > No clue at all?
>> > > >
>> > > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
>> > > > santand@gmail.com>
>> > > > ha scritto:
>> > > >
>> > > > > Could it be a bug in the front-end? Or Am I missing something?
>> > > > >
>> > > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
>> > > > > santand@gmail.com> ha scritto:
>> > > > >
>> > > > >> More or less yes (it transforms sql statements into KafkaStreams
>> > > jobs).
>> > > > >> But I simply call a rest endpoint that has as response
>> > > > Transfer-Encoding:
>> > > > >> chunked and the data comes in as you can check with the linked
>> > image.
>> > > > >> The only thing it's that I have blank rows (but if I debug the
>> code
>> > > the
>> > > > >> data is in there) and when I stop the job every blank row gets
>> the
>> > > > correct
>> > > > >> value.
>> > > > >>
>> > > > >>
>> > > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
>> > zjffdu@gmail.com>
>> > > > ha
>> > > > >> scritto:
>> > > > >>
>> > > > >>> I don't know much about KSQL, is each running a streaming job
>> which
>> > > > never
>> > > > >>> stops ?
>> > > > >>>
>> > > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
>> > > > >>>
>> > > > >>> > Hi guys,
>> > > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL
>> engine
>> > > that
>> > > > >>> > enables real-time data processing against Apache Kafka topics.
>> > > > >>> > I created an interpreter that leverages the
>> > InterpreterOutputStream
>> > > > >>> class
>> > > > >>> > in order to stream the result from the backend to the
>> frontend,
>> > > but I
>> > > > >>> get
>> > > > >>> > the following representation in the frontend:
>> > > > >>> >
>> > > > >>> > https://imgur.com/a/J7unk25
>> > > > >>> >
>> > > > >>> > What am I doing wrong?
>> > > > >>> >
>> > > > >>> > Following the Interpreter implementation:
>> > > > >>> >
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>> > > > >>> >
>> > > > >>> > Thanks a lot
>> > > > >>> >
>> > > > >>> > Andrea
>> > > > >>> >
>> > > > >>>
>> > > > >>>
>> > > > >>> --
>> > > > >>> Best Regards
>> > > > >>>
>> > > > >>> Jeff Zhang
>> > > > >>>
>> > > > >>
>> > > >
>> > >
>> > >
>> > > --
>> > > Best Regards
>> > >
>> > > Jeff Zhang
>> > >
>> >
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
Seems to be different, please check here:

https://gist.github.com/conker84/4a54df4c73dab310cdc9284f15e90075

Can you mentor me about what I'm wrong?



Il giorno mar 8 ott 2019 alle ore 11:13 Jeff Zhang <zj...@gmail.com> ha
scritto:

> The the following line is suspicious to me.
>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120
>
> Could you add the following line to log4j.properties so that you can see
> more logging info.
>      log4j.logger.org.apache.zeppelin.interpreter=DEBUG
>
> Then you should be able to see logging like following in the interpreter
> log. That means interpreter generate output and sent it to zeppelin server.
> If you don't see this kind of logging, then it should be your interpreter
> issue.
>
> DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> resourcePoolGetAll from ZeppelinServer
> DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>
> DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 7:
> DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8
>
> DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
> RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
> resourcePoolGetAll from ZeppelinServer
> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onAppend]:830) - Output Append:
>
> DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 8:
> DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
> RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9
>
>
> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:
>
> > Did you check the animated gif that I provided in the first email? Looks
> > like a problem in the visualization.
> >
> > Btw I expect the same behaviour as here:
> >
> > https://github.com/apache/zeppelin/pull/2809
> > and
> > https://github.com/apache/zeppelin/pull/2873
> >
> >
> >
> > Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zj...@gmail.com> ha
> > scritto:
> >
> > > I encounter some issues when integrating flink streaming, not sure the
> > same
> > > as you.
> > > Just curious to know what is your expected output if it is streaming
> job
> > > and output is generated continuously.
> > >
> > >
> > > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
> > >
> > > > No clue at all?
> > > >
> > > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> > > > santand@gmail.com>
> > > > ha scritto:
> > > >
> > > > > Could it be a bug in the front-end? Or Am I missing something?
> > > > >
> > > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> > > > > santand@gmail.com> ha scritto:
> > > > >
> > > > >> More or less yes (it transforms sql statements into KafkaStreams
> > > jobs).
> > > > >> But I simply call a rest endpoint that has as response
> > > > Transfer-Encoding:
> > > > >> chunked and the data comes in as you can check with the linked
> > image.
> > > > >> The only thing it's that I have blank rows (but if I debug the
> code
> > > the
> > > > >> data is in there) and when I stop the job every blank row gets the
> > > > correct
> > > > >> value.
> > > > >>
> > > > >>
> > > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
> > zjffdu@gmail.com>
> > > > ha
> > > > >> scritto:
> > > > >>
> > > > >>> I don't know much about KSQL, is each running a streaming job
> which
> > > > never
> > > > >>> stops ?
> > > > >>>
> > > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
> > > > >>>
> > > > >>> > Hi guys,
> > > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL
> engine
> > > that
> > > > >>> > enables real-time data processing against Apache Kafka topics.
> > > > >>> > I created an interpreter that leverages the
> > InterpreterOutputStream
> > > > >>> class
> > > > >>> > in order to stream the result from the backend to the frontend,
> > > but I
> > > > >>> get
> > > > >>> > the following representation in the frontend:
> > > > >>> >
> > > > >>> > https://imgur.com/a/J7unk25
> > > > >>> >
> > > > >>> > What am I doing wrong?
> > > > >>> >
> > > > >>> > Following the Interpreter implementation:
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> > > > >>> >
> > > > >>> > Thanks a lot
> > > > >>> >
> > > > >>> > Andrea
> > > > >>> >
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> Best Regards
> > > > >>>
> > > > >>> Jeff Zhang
> > > > >>>
> > > > >>
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > >
> > > Jeff Zhang
> > >
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: KSQL interpreter

Posted by Jeff Zhang <zj...@gmail.com>.
The the following line is suspicious to me.
https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java#L120

Could you add the following line to log4j.properties so that you can see
more logging info.
     log4j.logger.org.apache.zeppelin.interpreter=DEBUG

Then you should be able to see logging like following in the interpreter
log. That means interpreter generate output and sent it to zeppelin server.
If you don't see this kind of logging, then it should be your interpreter
issue.

DEBUG [2019-10-08 17:11:14,822] ({pool-1-thread-3}
RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
resourcePoolGetAll from ZeppelinServer
DEBUG [2019-10-08 17:11:15,813] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onAppend]:830) - Output Append:

DEBUG [2019-10-08 17:11:15,815] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 7:
DEBUG [2019-10-08 17:11:15,816] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onAppend]:830) - Output Append: 8

DEBUG [2019-10-08 17:11:15,826] ({pool-1-thread-3}
RemoteInterpreterServer.java[resourcePoolGetAll]:996) - Request
resourcePoolGetAll from ZeppelinServer
DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onAppend]:830) - Output Append:

DEBUG [2019-10-08 17:11:16,808] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onUpdate]:840) - Output Update for index 8:
DEBUG [2019-10-08 17:11:16,809] ({grpc-default-executor-0}
RemoteInterpreterServer.java[onAppend]:830) - Output Append: 9


Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:36写道:

> Did you check the animated gif that I provided in the first email? Looks
> like a problem in the visualization.
>
> Btw I expect the same behaviour as here:
>
> https://github.com/apache/zeppelin/pull/2809
> and
> https://github.com/apache/zeppelin/pull/2873
>
>
>
> Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zj...@gmail.com> ha
> scritto:
>
> > I encounter some issues when integrating flink streaming, not sure the
> same
> > as you.
> > Just curious to know what is your expected output if it is streaming job
> > and output is generated continuously.
> >
> >
> > Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
> >
> > > No clue at all?
> > >
> > > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> > > santand@gmail.com>
> > > ha scritto:
> > >
> > > > Could it be a bug in the front-end? Or Am I missing something?
> > > >
> > > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> > > > santand@gmail.com> ha scritto:
> > > >
> > > >> More or less yes (it transforms sql statements into KafkaStreams
> > jobs).
> > > >> But I simply call a rest endpoint that has as response
> > > Transfer-Encoding:
> > > >> chunked and the data comes in as you can check with the linked
> image.
> > > >> The only thing it's that I have blank rows (but if I debug the code
> > the
> > > >> data is in there) and when I stop the job every blank row gets the
> > > correct
> > > >> value.
> > > >>
> > > >>
> > > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <
> zjffdu@gmail.com>
> > > ha
> > > >> scritto:
> > > >>
> > > >>> I don't know much about KSQL, is each running a streaming job which
> > > never
> > > >>> stops ?
> > > >>>
> > > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
> > > >>>
> > > >>> > Hi guys,
> > > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL engine
> > that
> > > >>> > enables real-time data processing against Apache Kafka topics.
> > > >>> > I created an interpreter that leverages the
> InterpreterOutputStream
> > > >>> class
> > > >>> > in order to stream the result from the backend to the frontend,
> > but I
> > > >>> get
> > > >>> > the following representation in the frontend:
> > > >>> >
> > > >>> > https://imgur.com/a/J7unk25
> > > >>> >
> > > >>> > What am I doing wrong?
> > > >>> >
> > > >>> > Following the Interpreter implementation:
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> > > >>> >
> > > >>> > Thanks a lot
> > > >>> >
> > > >>> > Andrea
> > > >>> >
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Best Regards
> > > >>>
> > > >>> Jeff Zhang
> > > >>>
> > > >>
> > >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>


-- 
Best Regards

Jeff Zhang

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
Did you check the animated gif that I provided in the first email? Looks
like a problem in the visualization.

Btw I expect the same behaviour as here:

https://github.com/apache/zeppelin/pull/2809
and
https://github.com/apache/zeppelin/pull/2873



Il giorno mar 8 ott 2019 alle ore 10:04 Jeff Zhang <zj...@gmail.com> ha
scritto:

> I encounter some issues when integrating flink streaming, not sure the same
> as you.
> Just curious to know what is your expected output if it is streaming job
> and output is generated continuously.
>
>
> Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:
>
> > No clue at all?
> >
> > Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> > santand@gmail.com>
> > ha scritto:
> >
> > > Could it be a bug in the front-end? Or Am I missing something?
> > >
> > > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> > > santand@gmail.com> ha scritto:
> > >
> > >> More or less yes (it transforms sql statements into KafkaStreams
> jobs).
> > >> But I simply call a rest endpoint that has as response
> > Transfer-Encoding:
> > >> chunked and the data comes in as you can check with the linked image.
> > >> The only thing it's that I have blank rows (but if I debug the code
> the
> > >> data is in there) and when I stop the job every blank row gets the
> > correct
> > >> value.
> > >>
> > >>
> > >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <zj...@gmail.com>
> > ha
> > >> scritto:
> > >>
> > >>> I don't know much about KSQL, is each running a streaming job which
> > never
> > >>> stops ?
> > >>>
> > >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
> > >>>
> > >>> > Hi guys,
> > >>> > I'm building a KQL interpreter. KSQL is the streaming SQL engine
> that
> > >>> > enables real-time data processing against Apache Kafka topics.
> > >>> > I created an interpreter that leverages the InterpreterOutputStream
> > >>> class
> > >>> > in order to stream the result from the backend to the frontend,
> but I
> > >>> get
> > >>> > the following representation in the frontend:
> > >>> >
> > >>> > https://imgur.com/a/J7unk25
> > >>> >
> > >>> > What am I doing wrong?
> > >>> >
> > >>> > Following the Interpreter implementation:
> > >>> >
> > >>> >
> > >>> >
> > >>>
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> > >>> >
> > >>> > Thanks a lot
> > >>> >
> > >>> > Andrea
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>> Best Regards
> > >>>
> > >>> Jeff Zhang
> > >>>
> > >>
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: KSQL interpreter

Posted by Jeff Zhang <zj...@gmail.com>.
I encounter some issues when integrating flink streaming, not sure the same
as you.
Just curious to know what is your expected output if it is streaming job
and output is generated continuously.


Andrea Santurbano <sa...@gmail.com> 于2019年10月8日周二 下午4:01写道:

> No clue at all?
>
> Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <
> santand@gmail.com>
> ha scritto:
>
> > Could it be a bug in the front-end? Or Am I missing something?
> >
> > Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> > santand@gmail.com> ha scritto:
> >
> >> More or less yes (it transforms sql statements into KafkaStreams jobs).
> >> But I simply call a rest endpoint that has as response
> Transfer-Encoding:
> >> chunked and the data comes in as you can check with the linked image.
> >> The only thing it's that I have blank rows (but if I debug the code the
> >> data is in there) and when I stop the job every blank row gets the
> correct
> >> value.
> >>
> >>
> >> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <zj...@gmail.com>
> ha
> >> scritto:
> >>
> >>> I don't know much about KSQL, is each running a streaming job which
> never
> >>> stops ?
> >>>
> >>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
> >>>
> >>> > Hi guys,
> >>> > I'm building a KQL interpreter. KSQL is the streaming SQL engine that
> >>> > enables real-time data processing against Apache Kafka topics.
> >>> > I created an interpreter that leverages the InterpreterOutputStream
> >>> class
> >>> > in order to stream the result from the backend to the frontend, but I
> >>> get
> >>> > the following representation in the frontend:
> >>> >
> >>> > https://imgur.com/a/J7unk25
> >>> >
> >>> > What am I doing wrong?
> >>> >
> >>> > Following the Interpreter implementation:
> >>> >
> >>> >
> >>> >
> >>>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> >>> >
> >>> > Thanks a lot
> >>> >
> >>> > Andrea
> >>> >
> >>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>>
> >>
>


-- 
Best Regards

Jeff Zhang

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
No clue at all?

Il giorno lun 7 ott 2019 alle ore 14:14 Andrea Santurbano <sa...@gmail.com>
ha scritto:

> Could it be a bug in the front-end? Or Am I missing something?
>
> Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <
> santand@gmail.com> ha scritto:
>
>> More or less yes (it transforms sql statements into KafkaStreams jobs).
>> But I simply call a rest endpoint that has as response Transfer-Encoding:
>> chunked and the data comes in as you can check with the linked image.
>> The only thing it's that I have blank rows (but if I debug the code the
>> data is in there) and when I stop the job every blank row gets the correct
>> value.
>>
>>
>> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <zj...@gmail.com> ha
>> scritto:
>>
>>> I don't know much about KSQL, is each running a streaming job which never
>>> stops ?
>>>
>>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
>>>
>>> > Hi guys,
>>> > I'm building a KQL interpreter. KSQL is the streaming SQL engine that
>>> > enables real-time data processing against Apache Kafka topics.
>>> > I created an interpreter that leverages the InterpreterOutputStream
>>> class
>>> > in order to stream the result from the backend to the frontend, but I
>>> get
>>> > the following representation in the frontend:
>>> >
>>> > https://imgur.com/a/J7unk25
>>> >
>>> > What am I doing wrong?
>>> >
>>> > Following the Interpreter implementation:
>>> >
>>> >
>>> >
>>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>>> >
>>> > Thanks a lot
>>> >
>>> > Andrea
>>> >
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
Could it be a bug in the front-end? Or Am I missing something?

Il giorno lun 7 ott 2019 alle ore 09:39 Andrea Santurbano <sa...@gmail.com>
ha scritto:

> More or less yes (it transforms sql statements into KafkaStreams jobs).
> But I simply call a rest endpoint that has as response Transfer-Encoding:
> chunked and the data comes in as you can check with the linked image.
> The only thing it's that I have blank rows (but if I debug the code the
> data is in there) and when I stop the job every blank row gets the correct
> value.
>
>
> Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <zj...@gmail.com> ha
> scritto:
>
>> I don't know much about KSQL, is each running a streaming job which never
>> stops ?
>>
>> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
>>
>> > Hi guys,
>> > I'm building a KQL interpreter. KSQL is the streaming SQL engine that
>> > enables real-time data processing against Apache Kafka topics.
>> > I created an interpreter that leverages the InterpreterOutputStream
>> class
>> > in order to stream the result from the backend to the frontend, but I
>> get
>> > the following representation in the frontend:
>> >
>> > https://imgur.com/a/J7unk25
>> >
>> > What am I doing wrong?
>> >
>> > Following the Interpreter implementation:
>> >
>> >
>> >
>> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>> >
>> > Thanks a lot
>> >
>> > Andrea
>> >
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

Re: KSQL interpreter

Posted by Andrea Santurbano <sa...@gmail.com>.
More or less yes (it transforms sql statements into KafkaStreams jobs).
But I simply call a rest endpoint that has as response Transfer-Encoding:
chunked and the data comes in as you can check with the linked image.
The only thing it's that I have blank rows (but if I debug the code the
data is in there) and when I stop the job every blank row gets the correct
value.


Il giorno lun 7 ott 2019 alle ore 08:06 Jeff Zhang <zj...@gmail.com> ha
scritto:

> I don't know much about KSQL, is each running a streaming job which never
> stops ?
>
> Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:
>
> > Hi guys,
> > I'm building a KQL interpreter. KSQL is the streaming SQL engine that
> > enables real-time data processing against Apache Kafka topics.
> > I created an interpreter that leverages the InterpreterOutputStream class
> > in order to stream the result from the backend to the frontend, but I get
> > the following representation in the frontend:
> >
> > https://imgur.com/a/J7unk25
> >
> > What am I doing wrong?
> >
> > Following the Interpreter implementation:
> >
> >
> >
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
> >
> > Thanks a lot
> >
> > Andrea
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: KSQL interpreter

Posted by Jeff Zhang <zj...@gmail.com>.
I don't know much about KSQL, is each running a streaming job which never
stops ?

Andrea Santurbano <sa...@gmail.com> 于2019年10月7日周一 上午4:09写道:

> Hi guys,
> I'm building a KQL interpreter. KSQL is the streaming SQL engine that
> enables real-time data processing against Apache Kafka topics.
> I created an interpreter that leverages the InterpreterOutputStream class
> in order to stream the result from the backend to the frontend, but I get
> the following representation in the frontend:
>
> https://imgur.com/a/J7unk25
>
> What am I doing wrong?
>
> Following the Interpreter implementation:
>
>
> https://github.com/conker84/zeppelin/blob/kafka/kafka/src/main/java/org/apache/zeppelin/kafka/KafkaKSQLInterpreter.java
>
> Thanks a lot
>
> Andrea
>


-- 
Best Regards

Jeff Zhang