[jira] [Created] (FLINK-23719) Support switch WAL in Flink SQL DDL options

["Carl (Jira)" <ji...@apache.org>]

Carl created FLINK-23719:
----------------------------

             Summary: Support switch WAL in Flink SQL  DDL options
                 Key: FLINK-23719
                 URL: https://issues.apache.org/jira/browse/FLINK-23719
             Project: Flink
          Issue Type: New Feature
          Components: Connectors / HBase
    Affects Versions: 1.13.2
            Reporter: Carl


My application scenario is like this:

Use flick SQL to write the batch data of hive table to a new HBase table every day.

In this case, I think it is appropriate to disable WAL for the following reasons:

1. If an error occurs, i can re-run it. No high security is required.

2. The write speed will be greatly improved. No additional writing to the wal disk log

3. Save disk IO resources of the cluster

 

However, the HBase official website is described below,
h4. 70.6.10. Disabling the WAL
It is possible to disable the WAL, to improve performance in certain specific situations. However, disabling the WAL puts your data at risk. The only situation where this is recommended is during a bulk load. This is because, in the event of a problem, the bulk load can be re-run with no risk of data loss.
The WAL is disabled by calling the HBase client field {{Mutation.writeToWAL(false)}}. Use the {{Mutation.setDurability(Durability.SKIP_WAL)}} and Mutation.getDurability() methods to set and get the field’s value. There is no way to disable the WAL for only a specific table.

[http://hbase.apache.org/2.3/book.html#hbase_default_configurations]

 

So i think it needs API level support, not properties support.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)