You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/05/15 17:59:03 UTC
[jira] [Updated] (BEAM-9896) Add streaming for SnowflakeIO.Write to
Java SDK
[ https://issues.apache.org/jira/browse/BEAM-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-9896:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Resolved)
Hello! Due to a bug in our Jira configuration, this issue had status:Resolved but resolution:Unresolved.
I am bulk editing these issues to have resolution:Fixed
If a different resolution is appropriate, please change it. To do this, click the "Resolve" button (you can do this even for closed issues) and set the Resolution field to the right value.
> Add streaming for SnowflakeIO.Write to Java SDK
> -----------------------------------------------
>
> Key: BEAM-9896
> URL: https://issues.apache.org/jira/browse/BEAM-9896
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
> Affects Versions: 2.24.0
> Reporter: Dariusz Aniszewski
> Assignee: Kasia Kucharczyk
> Priority: P2
> Fix For: 2.24.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> SnowflakeIO has currently only batch processing. To provide streaming processing it is required to use [SnowPipe|https://docs.snowflake.com/en/user-guide/data-load-snowpipe.html] which enables streaming write to Snowflake. Currently it is impossible to provide streaming read.
> [Snowflake ingest library|https://github.com/snowflakedb/snowflake-ingest-java] for Java provides ingest methods which provides using SnowPipe and loading data by authorising with key-pair and providing staged files (ex. on GCS). It also provides method retrieving load history which is the only way to check if files could be loaded properly. Checking history can be only provided by actively requesting API because loading data is asynchronous.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)