[Proposal] Query mode in extraction of Streaming Engine

[ξ <13...@qq.com.INVALID>]

Hello everyone,
I'm Caideyipi, and is developing the Streaming Engine module. Here I wish to share a new feature I introduced to the "Extraction" phase of Streaming Engine.
Formerly, the Streaming Engine is long restricted to unbounded stream. For instance, if users want to extract the historical data only, they can only judge the end themselves by querying the receiver side, and drop the pipe manually. The inconvenience is apparent in not only the normal scenarios, but the data subscription and the tsFile export service, which may need a terminal event to halt the entire procedure.
Hence, a bound is introduced in the extractor phase. Now users can use "source.mode" in "with source" clause ("source" can be omitted) to decide whether to send the terminal event and drop the pipe automatically when the transmission is completed. When "source.mode" == "query" (default is "subscribe"), when the historical data is all processed/transmitted, a terminal event is sent to the subsequent steps. When all the pipe tasks has completed, the ConfigNode will collect the information in pipe heartbeats, and drop the pipe if all the pipeTasks have completed.
Fervently hope you like the feature, and glad to see any ideas or suggestions.
Thanks for your reading and support.

Sincerely,

Caideyipi