You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by nikhilsimha <r....@gmail.com> on 2018/02/14 21:35:36 UTC

How do I run SQL query on a dataStream that generates my custom type.

I have a stream of events of a custom type. I want to know how to convert
these events into Rows that can be queried. More specifically how do I
attach type information to the stream of rows that is generated?

I have the following code

```
    val execEnv = StreamExecutionEnvironment.getExecutionEnvironment
    val tableEnv: StreamTableEnvironment =
TableEnvironment.getTableEnvironment(execEnv)

    val eventStream : DataStream[Event] = ...
    val rowStream: DataStream[Row] = eventStream.map({e: Event => toRow(e)})
    tableEnv.registerDataStream("some_name", rowStream)

    tableEnv.sql("select bedrooms from some_name") 
```

This code compiles. But clearly wouldn't work because I didn't specify the
type or position of `bedrooms`. 


So what API do I use to specify the typeInformation of my rowStream?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: How do I run SQL query on a dataStream that generates my custom type.

Posted by Timo Walther <tw...@apache.org>.
Or even easier:

You can do specify the type after the map call:

eventStream.map({e: Event => toRow(e)})(Types.ROW_NAMED(...))

Regards,
Timo


Am 2/15/18 um 9:55 AM schrieb Timo Walther:
> Hi,
>
> In your case you don't have to convert to row if you don't want to. 
> The Table API will do automatic conversion once the stream of Event is 
> converted into a table. However, this only works if Event is a POJO.
>
> If you want to specify own type information your MapFunction can 
> implement the ResultTypeQueryable interface. In the getProducedType 
> method you can specify your row infomation. Like Types.ROW_NAMED(...).
>
> Hope this helps.
>
> Regards,
> Timo
>
> Am 2/14/18 um 10:35 PM schrieb nikhilsimha:
>> I have a stream of events of a custom type. I want to know how to 
>> convert
>> these events into Rows that can be queried. More specifically how do I
>> attach type information to the stream of rows that is generated?
>>
>> I have the following code
>>
>> ```
>>      val execEnv = StreamExecutionEnvironment.getExecutionEnvironment
>>      val tableEnv: StreamTableEnvironment =
>> TableEnvironment.getTableEnvironment(execEnv)
>>
>>      val eventStream : DataStream[Event] = ...
>>      val rowStream: DataStream[Row] = eventStream.map({e: Event => 
>> toRow(e)})
>>      tableEnv.registerDataStream("some_name", rowStream)
>>
>>      tableEnv.sql("select bedrooms from some_name")
>> ```
>>
>> This code compiles. But clearly wouldn't work because I didn't 
>> specify the
>> type or position of `bedrooms`.
>>
>>
>> So what API do I use to specify the typeInformation of my rowStream?
>>
>>
>>
>> -- 
>> Sent from: 
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>


Re: How do I run SQL query on a dataStream that generates my custom type.

Posted by Timo Walther <tw...@apache.org>.
Hi,

In your case you don't have to convert to row if you don't want to. The 
Table API will do automatic conversion once the stream of Event is 
converted into a table. However, this only works if Event is a POJO.

If you want to specify own type information your MapFunction can 
implement the ResultTypeQueryable interface. In the getProducedType 
method you can specify your row infomation. Like Types.ROW_NAMED(...).

Hope this helps.

Regards,
Timo

Am 2/14/18 um 10:35 PM schrieb nikhilsimha:
> I have a stream of events of a custom type. I want to know how to convert
> these events into Rows that can be queried. More specifically how do I
> attach type information to the stream of rows that is generated?
>
> I have the following code
>
> ```
>      val execEnv = StreamExecutionEnvironment.getExecutionEnvironment
>      val tableEnv: StreamTableEnvironment =
> TableEnvironment.getTableEnvironment(execEnv)
>
>      val eventStream : DataStream[Event] = ...
>      val rowStream: DataStream[Row] = eventStream.map({e: Event => toRow(e)})
>      tableEnv.registerDataStream("some_name", rowStream)
>
>      tableEnv.sql("select bedrooms from some_name")
> ```
>
> This code compiles. But clearly wouldn't work because I didn't specify the
> type or position of `bedrooms`.
>
>
> So what API do I use to specify the typeInformation of my rowStream?
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/