[GitHub] [flink] XuPingyong opened a new pull request #8690: [FLINK-12801][table-planner-blink] set parallelism for batch SQL

[GitBox <gi...@apache.org>]

XuPingyong opened a new pull request #8690: [FLINK-12801][table-planner-blink] set parallelism for batch SQL
URL: https://github.com/apache/flink/pull/8690
 
 
   What is the purpose of the change
   
   DataStream user can set parallelism by SingleOutputStreamOperator#setParallelism and DataStreamSink#setParallelism. But SQL users cannot set parallelism  to operators while compiled jobGraphs from SQL are usally complex.
          Now we first set parallelism for batch SQL by config. We introduce two resourceSetting mode:
          InferMode.NONE:  User can set parallelism to source, sink and other nodes separately.
          InferMode.ONLY_SOURCE： Relative to  InferMode.NONE, source paralelism can be inferred by source row count.
           We also introduce ShuffleStage to make adjacent operatos parallelism same that there is no data shuffle between them.
   
   
   Brief change log
   
   Introduce parallelism setting for batch SQL
   
   Verifying this change
   
   ut
   
   Does this pull request potentially affect one of the following parts:
   
   Dependencies (does it add or upgrade a dependency): (no)
   The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
   The serializers: (no)
   The runtime per-record code paths (performance sensitive): (no)
   Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
   The S3 file system connector: (no)
   Documentation
   
   Does this pull request introduce a new feature? (yes)
   If yes, how is the feature documented? (JavaDocs)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services