You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jane Chan (Jira)" <ji...@apache.org> on 2022/05/20 02:35:00 UTC
[jira] [Created] (FLINK-27705) num-sorted-run.compaction-trigger should not interfere the num-levels
Jane Chan created FLINK-27705:
---------------------------------
Summary: num-sorted-run.compaction-trigger should not interfere the num-levels
Key: FLINK-27705
URL: https://issues.apache.org/jira/browse/FLINK-27705
Project: Flink
Issue Type: Improvement
Components: Table Store
Affects Versions: table-store-0.2.0
Reporter: Jane Chan
Fix For: table-store-0.2.0
h3. Issue Description
The default value for MergeTreeOptions.NUM_LEVELS is not a constant, and it varies with MergeTreeOptions.NUM_SORTED_RUNS_COMPACTION_TRIGGER. If users do not specify the MergeTreeOptions.NUM_LEVELS at the first, once MergeTreeOptions.NUM_SORTED_RUNS_COMPACTION_TRIGGER is changed, and the successive write task would get a chance to fail (to be specific, when the compaction trigger size shrinks and the previous writes had triggered compaction).
h3. How to Reproduce
add a test under ForceCompactionITCase
{code:java}
@Override
protected List<String> ddl() {
return Arrays.asList(
"CREATE TABLE IF NOT EXISTS T (\n"
+ " f0 INT\n, "
+ " f1 STRING\n, "
+ " f2 STRING\n"
+ ") PARTITIONED BY (f1)",
"CREATE TABLE IF NOT EXISTS T1 (\n"
+ " f0 INT\n, "
+ " f1 STRING\n, "
+ " f2 STRING\n"
+ ")");
}
@Test
public void test() throws Exception {
bEnv.executeSql("ALTER TABLE T1 SET ('commit.force-compact' = 'true')");
bEnv.executeSql(
"INSERT INTO T1 VALUES(1, 'Winter', 'Winter is Coming'),"
+ "(2, 'Winter', 'The First Snowflake'), "
+ "(2, 'Spring', 'The First Rose in Spring'), "
+ "(7, 'Summer', 'Summertime Sadness')")
.await();
bEnv.executeSql("INSERT INTO T1 VALUES(12, 'Winter', 'Last Christmas')").await();
bEnv.executeSql("INSERT INTO T1 VALUES(11, 'Winter', 'Winter is Coming')").await();
bEnv.executeSql("INSERT INTO T1 VALUES(10, 'Autumn', 'Refrain')").await();
bEnv.executeSql(
"INSERT INTO T1 VALUES(6, 'Summer', 'Watermelon Sugar'), "
+ "(4, 'Spring', 'Spring Water')")
.await();
bEnv.executeSql(
"INSERT INTO T1 VALUES(66, 'Summer', 'Summer Vibe'),"
+ " (9, 'Autumn', 'Wake Me Up When September Ends')")
.await();
bEnv.executeSql(
"INSERT INTO T1 VALUES(666, 'Summer', 'Summer Vibe'),"
+ " (9, 'Autumn', 'Wake Me Up When September Ends')")
.await();
bEnv.executeSql("ALTER TABLE T1 SET ('num-sorted-run.compaction-trigger' = '2')");
bEnv.executeSql(
"INSERT INTO T1 VALUES(666, 'Summer', 'Summer Vibe'),"
+ " (9, 'Autumn', 'Wake Me Up When September Ends')")
.await();
} {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)