[GitHub] [druid] FrankChen021 commented on issue #11929: Batch ingestion using SQL INSERT

[GitBox <gi...@apache.org>]

FrankChen021 commented on issue #11929:
URL: https://github.com/apache/druid/issues/11929#issuecomment-971324041


   @gianm This is a great improvement in Druid. I think the `EXTERN` is not intuitive for users. It still has the smell of old JSON supervisor/task spec. I think the way ClickHouse uses is more user friendly. It uses ENGINE or SETTINGS sub-clause to specify the parameters. Below are examples taken from ClickHouse user doc. I think this also answers @abhishekagarwal87 question.
   
   ```
   CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
   (
       name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1],
       name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2],
       ...
   ) ENGINE = Kafka()
   SETTINGS
       kafka_broker_list = 'host:port',
       kafka_topic_list = 'topic1,topic2,...',
       kafka_group_name = 'group_name',
       kafka_format = 'data_format'[,]
       [kafka_row_delimiter = 'delimiter_symbol',]
       [kafka_schema = '',]
       [kafka_num_consumers = N,]
       [kafka_max_block_size = 0,]
       [kafka_skip_broken_messages = N,]
       [kafka_commit_every_batch = 0,]
       [kafka_thread_per_consumer = 0]
   ```
   
   ```
   CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
   (
       name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1] [TTL expr1],
       name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2] [TTL expr2],
       ...
   ) ENGINE = MySQL('host:port', 'database', 'table', 'user', 'password'[, replace_query, 'on_duplicate_clause'])
   SETTINGS
       [connection_pool_size=16, ]
       [connection_max_tries=3, ]
       [connection_wait_timeout=5, ] /* 0 -- do not wait */
       [connection_auto_close=true ]
   ;
   ```
   
   ```
   CREATE TABLE s3_engine_table (name String, value UInt32)
       ENGINE = S3(path, [aws_access_key_id, aws_secret_access_key,] format, [compression])
       [SETTINGS ...]
   ```
   
   more info can be found here: https://clickhouse.com/docs/en/engines/table-engines/integrations/


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org