You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2023/08/18 01:23:05 UTC

[GitHub] [doris] yiguolei commented on a diff in pull request #23053: [feature](insert) Support wal for group commit insert

yiguolei commented on code in PR #23053:
URL: https://github.com/apache/doris/pull/23053#discussion_r1297893524


##########
be/src/runtime/stream_load/stream_load_executor.cpp:
##########
@@ -64,7 +64,8 @@ bvar::LatencyRecorder g_stream_load_begin_txn_latency("stream_load", "begin_txn"
 bvar::LatencyRecorder g_stream_load_precommit_txn_latency("stream_load", "precommit_txn");
 bvar::LatencyRecorder g_stream_load_commit_txn_latency("stream_load", "commit_txn");
 
-Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptr<StreamLoadContext> ctx) {
+Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptr<StreamLoadContext> ctx,

Review Comment:
   This method will be removed because we are using insert into select to replace stream load. 
   ### Use stream load with SQL
   
   You can add a `sql` parameter to the `Header` to replace the `column_separator`, `line_delimiter`, `where`, `columns` in the previous parameter, which is convenient to use.
   
   ```
   curl --location-trusted -u user:passwd 
   [-H "sql: ${load_sql}"...] 
   -T data.file 
   -XPUT http://fe_host:http_port/api/{db}/{table}/_stream_load_with_sql
   
   
   # -- load_sql
   # insert into db.table (col, ...) select stream_col, ... from stream("property1"="value1");
   
   # stream
   # (
   #     "column_separator" = ",",
   #     "format" = "CSV",
   #     ...
   # )
   ```
   
   Examples:
   
   ```
   curl  --location-trusted -u root: -T test.csv  -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from stream("format" = "CSV", "column_separator" = "," ) where age >= 30"  http://127.0.0.1:28030/api/demo/example_tbl_1/_stream_load_with_sql
   ```



##########
be/src/runtime/stream_load/stream_load_executor.cpp:
##########
@@ -64,7 +64,8 @@ bvar::LatencyRecorder g_stream_load_begin_txn_latency("stream_load", "begin_txn"
 bvar::LatencyRecorder g_stream_load_precommit_txn_latency("stream_load", "precommit_txn");
 bvar::LatencyRecorder g_stream_load_commit_txn_latency("stream_load", "commit_txn");
 
-Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptr<StreamLoadContext> ctx) {
+Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptr<StreamLoadContext> ctx,

Review Comment:
   This method will be removed because we are using insert into select to replace stream load. 
   ### Use stream load with SQL
   
   You can add a `sql` parameter to the `Header` to replace the `column_separator`, `line_delimiter`, `where`, `columns` in the previous parameter, which is convenient to use.
   
   ```
   curl --location-trusted -u user:passwd 
   [-H "sql: ${load_sql}"...] 
   -T data.file 
   -XPUT http://fe_host:http_port/api/{db}/{table}/_stream_load_with_sql
   
   
   # -- load_sql
   # insert into db.table (col, ...) select stream_col, ... from stream("property1"="value1");
   
   # stream
   # (
   #     "column_separator" = ",",
   #     "format" = "CSV",
   #     ...
   # )
   ```
   
   Examples:
   
   ```
   curl  --location-trusted -u root: -T test.csv  -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from stream("format" = "CSV", "column_separator" = "," ) where age >= 30"  http://127.0.0.1:28030/api/demo/example_tbl_1/_stream_load_with_sql
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org