You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Steve Niemitz <sn...@twitter.com> on 2022/01/12 21:54:11 UTC
Using CREATE EXTERNAL TABLE in a pipeline?
Is it possible to use CREATE EXTERNAL TABLE + a select statement in a
SqlTransform to act as a source for a pipeline? The functionality seems
really useful, but it seems like it only works from the JDBC context? Am I
missing anything?
Re: Using CREATE EXTERNAL TABLE in a pipeline?
Posted by Steve Niemitz <sn...@twitter.com>.
Oh that's awesome, I didn't notice the extra method on the result of
"query" and was trying to just add the CREATE statement in there. Thanks!
On Wed, Jan 12, 2022 at 5:00 PM Brian Hulette <bh...@google.com> wrote:
> Hi Steve,
>
> This functionality was added relatively recently, via
> SqlTransform(..).withDdlString(). There are some usage examples here:
> https://beam.apache.org/releases/javadoc/2.35.0/org/apache/beam/sdk/extensions/sql/SqlTransform.html
>
> Brian
>
> On Wed, Jan 12, 2022 at 1:54 PM Steve Niemitz <sn...@twitter.com>
> wrote:
>
>> Is it possible to use CREATE EXTERNAL TABLE + a select statement in a
>> SqlTransform to act as a source for a pipeline? The functionality seems
>> really useful, but it seems like it only works from the JDBC context? Am I
>> missing anything?
>>
>
Re: Using CREATE EXTERNAL TABLE in a pipeline?
Posted by Brian Hulette <bh...@google.com>.
Hi Steve,
This functionality was added relatively recently, via
SqlTransform(..).withDdlString(). There are some usage examples here:
https://beam.apache.org/releases/javadoc/2.35.0/org/apache/beam/sdk/extensions/sql/SqlTransform.html
Brian
On Wed, Jan 12, 2022 at 1:54 PM Steve Niemitz <sn...@twitter.com> wrote:
> Is it possible to use CREATE EXTERNAL TABLE + a select statement in a
> SqlTransform to act as a source for a pipeline? The functionality seems
> really useful, but it seems like it only works from the JDBC context? Am I
> missing anything?
>