You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jane Chan (Jira)" <ji...@apache.org> on 2023/04/27 09:13:00 UTC
[jira] [Updated] (FLINK-31956) Extend the CompiledPlan to read from/write to Flink's FileSystem

     [ https://issues.apache.org/jira/browse/FLINK-31956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jane Chan updated FLINK-31956:
------------------------------
    Description: 
At present, COMPILE/EXECUTE PLAN FOR '${plan.json}' only supports writing to/reading from a local file without the scheme. We propose to extend the support for Flink's FileSystem.
{code:sql}
-- before
COMPILE PLAN FOR '/tmp/foo/bar.json' <insert_query>
EXECUTE PLAN FOR '/tmp/foo/bar.json' 

-- after
COMPILE PLAN FOR 'file:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 'hdfs:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 's3:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 'oss:///tmp/foo/bar.json' <insert_query> 
EXECUTE PLAN FOR 'file:///tmp/foo/bar.json'
EXECUTE PLAN FOR 'hdfs:///tmp/foo/bar.json'
EXECUTE PLAN FOR 's3:///tmp/foo/bar.json'
EXECUTE PLAN FOR 'oss:///tmp/foo/bar.json' {code}

  was:
At present, COMPILE/EXECUTE PLAN FOR '${plan.json}' only supports writing to/reading from a local file without the scheme. We propose to extend the support for Flink's FileSystem.
{code:java}
-- before
COMPILE PLAN FOR '/tmp/foo/bar.json' <insert_query>
EXECUTE PLAN FOR '/tmp/foo/bar.json' 

-- after
COMPILE PLAN FOR 'file:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 'hdfs:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 's3:///tmp/foo/bar.json' <insert_query>
COMPILE PLAN FOR 'oss:///tmp/foo/bar.json' <insert_query> 
EXECUTE PLAN FOR 'file:///tmp/foo/bar.json'
EXECUTE PLAN FOR 'hdfs:///tmp/foo/bar.json'
EXECUTE PLAN FOR 's3:///tmp/foo/bar.json'
EXECUTE PLAN FOR 'oss:///tmp/foo/bar.json' {code}


> Extend the CompiledPlan to read from/write to Flink's FileSystem
> ----------------------------------------------------------------
>
>                 Key: FLINK-31956
>                 URL: https://issues.apache.org/jira/browse/FLINK-31956
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / Client, Table SQL / Planner
>    Affects Versions: 1.18.0
>            Reporter: Jane Chan
>            Priority: Major
>             Fix For: 1.18.0
>
>
> At present, COMPILE/EXECUTE PLAN FOR '${plan.json}' only supports writing to/reading from a local file without the scheme. We propose to extend the support for Flink's FileSystem.
> {code:sql}
> -- before
> COMPILE PLAN FOR '/tmp/foo/bar.json' <insert_query>
> EXECUTE PLAN FOR '/tmp/foo/bar.json' 
> -- after
> COMPILE PLAN FOR 'file:///tmp/foo/bar.json' <insert_query>
> COMPILE PLAN FOR 'hdfs:///tmp/foo/bar.json' <insert_query>
> COMPILE PLAN FOR 's3:///tmp/foo/bar.json' <insert_query>
> COMPILE PLAN FOR 'oss:///tmp/foo/bar.json' <insert_query> 
> EXECUTE PLAN FOR 'file:///tmp/foo/bar.json'
> EXECUTE PLAN FOR 'hdfs:///tmp/foo/bar.json'
> EXECUTE PLAN FOR 's3:///tmp/foo/bar.json'
> EXECUTE PLAN FOR 'oss:///tmp/foo/bar.json' {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)