You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Anton Solovev <An...@epam.com> on 2016/11/02 12:53:36 UTC

[FLINK-3848] Add ProjectableTableSource

Hi folks,

Can you explain me what ProjectableTableSource is ?
What is it being invented for?

Best,
Anton

RE: [FLINK-3848] Add ProjectableTableSource

Posted by Anton Solovev <An...@epam.com>.
I have a question about type info, it looks like one in calcite mail you showed in jira,
but I catch CodeGenException when checking predefined table field types with output types from source

Flink cannot generate conversion because of row arity. We cannot reduce number of table fields, 
but we can create a new code generation 
or let nulls come from CsvInputFormt

Any ideas?

-----Original Message-----
From: Fabian Hueske [mailto:fhueske@gmail.com] 
Sent: Wednesday, November 2, 2016 5:11 PM
To: dev@flink.apache.org
Subject: Re: [FLINK-3848] Add ProjectableTableSource

Hi Anton,

a regular TableSource does not accept a predicate and return the whole table.
A ProjectableTableSource is able to evaluate a predicate while scanning.
TableSources that evaluate predicates while (or rather before) scanning can significantly reduce IO compared to a full scan.

Sources that could be access with a ProjectableTableSource would be a JDBC database which pushes a query to a JDBC connected database, a Parquet file, or an ORC file.

Best, Fabian






2016-11-02 13:53 GMT+01:00 Anton Solovev <An...@epam.com>:

> Hi folks,
>
> Can you explain me what ProjectableTableSource is ?
> What is it being invented for?
>
> Best,
> Anton
>
>

Re: [FLINK-3848] Add ProjectableTableSource

Posted by Fabian Hueske <fh...@gmail.com>.
Hi Anton,

a regular TableSource does not accept a predicate and return the whole
table.
A ProjectableTableSource is able to evaluate a predicate while scanning.
TableSources that evaluate predicates while (or rather before) scanning can
significantly reduce IO compared to a full scan.

Sources that could be access with a ProjectableTableSource would be a JDBC
database which pushes a query to a JDBC connected database, a Parquet file,
or an ORC file.

Best, Fabian






2016-11-02 13:53 GMT+01:00 Anton Solovev <An...@epam.com>:

> Hi folks,
>
> Can you explain me what ProjectableTableSource is ?
> What is it being invented for?
>
> Best,
> Anton
>
>