You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yu Watanabe <yu...@gmail.com> on 2019/11/20 11:56:18 UTC
Does apache flink support stream input from Postgresql ?
Hello .
I would like to ask question about possibility of stream read table rows
from PostgresQL using pyflink.
In use cases in online document, there is a example which is reading data
from transactional database.
https://flink.apache.org/img/usecases-datapipelines.png
<https://flink.apache.org/usecases.html>
Is it possible to stream read from PostgresQL using pyflink ?
Thanks,
Yu Watanabe
--
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
yu.w.tennis@gmail.com
[image: LinkedIn icon] <https://www.linkedin.com/in/yuwatanabe1> [image:
Twitter icon] <https://twitter.com/yuwtennis>
Re: Does apache flink support stream input from Postgresql ?
Posted by Zhenghua Gao <do...@gmail.com>.
The jdbc connector can read data from PostgreSQL for Table/SQL users.
For pyflink, cc @Hequn
*Best Regards,*
*Zhenghua Gao*
On Wed, Nov 20, 2019 at 7:56 PM Yu Watanabe <yu...@gmail.com> wrote:
> Hello .
>
> I would like to ask question about possibility of stream read table rows
> from PostgresQL using pyflink.
>
> In use cases in online document, there is a example which is reading data
> from transactional database.
>
> https://flink.apache.org/img/usecases-datapipelines.png
> <https://flink.apache.org/usecases.html>
>
> Is it possible to stream read from PostgresQL using pyflink ?
>
> Thanks,
> Yu Watanabe
>
> --
> Yu Watanabe
> Weekend Freelancer who loves to challenge building data platform
> yu.w.tennis@gmail.com
> [image: LinkedIn icon] <https://www.linkedin.com/in/yuwatanabe1> [image:
> Twitter icon] <https://twitter.com/yuwtennis>
>
Re: Does apache flink support stream input from Postgresql ?
Posted by Theo Diefenthal <th...@scoop-software.de>.
Hi,
You could check out Debezium [1] and get your database CDC into a kafka stream which can then be read nicely by flink.
Best regards
Theo
[1] https://debezium.io/documentation/reference/1.0/tutorial.html
Von: "Yu Watanabe" <yu...@gmail.com>
An: "user" <us...@flink.apache.org>
Gesendet: Mittwoch, 20. November 2019 12:56:18
Betreff: Does apache flink support stream input from Postgresql ?
Hello .
I would like to ask question about possibility of stream read table rows from PostgresQL using pyflink.
In use cases in online document, there is a example which is reading data from transactional database.
[ https://flink.apache.org/usecases.html | https://flink.apache.org/img/usecases-datapipelines.png ]
Is it possible to stream read from PostgresQL using pyflink ?
Thanks,
Yu Watanabe
--
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
[ mailto:yu.w.tennis@gmail.com | yu.w.tennis@gmail.com ]
[ https://www.linkedin.com/in/yuwatanabe1 ] [ https://twitter.com/yuwtennis ]
--
SCOOP Software GmbH - Gut Maarhausen - Eiler Straße 3 P - D-51107 Köln
Theo Diefenthal
T +49 221 801916-196 - F +49 221 801916-17 - M +49 160 90506575
theo.diefenthal@scoop-software.de - www.scoop-software.de
Sitz der Gesellschaft: Köln, Handelsregister: Köln,
Handelsregisternummer: HRB 36625
Geschäftsführung: Dr. Oleg Balovnev, Frank Heinen,
Martin Müller-Rohde, Dr. Wolfgang Reddig, Roland Scheel