You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2019/07/10 20:51:15 UTC
[GitHub] [flink] bowenli86 commented on a change in pull request #9067:
[FLINK-13069][hive] HiveTableSink should implement OverwritableTableSink
bowenli86 commented on a change in pull request #9067: [FLINK-13069][hive] HiveTableSink should implement OverwritableTableSink
URL: https://github.com/apache/flink/pull/9067#discussion_r302266277
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/batch/connectors/hive/HiveTableSinkTest.java
##########
@@ -235,10 +220,44 @@ public void testInsertIntoStaticPartition() throws Exception {
// make sure new partition is created
assertEquals(toWrite.size(), hiveCatalog.listPartitions(tablePath).size());
- CatalogPartition catalogPartition = hiveCatalog.getPartition(tablePath, new CatalogPartitionSpec(partSpec));
- String partitionLocation = catalogPartition.getProperties().get(HiveCatalogConfig.PARTITION_LOCATION);
- verifyWrittenData(new Path(partitionLocation, "0"), toWrite, 1);
+ verifyWrittenData(toWrite, hiveShell.executeQuery("select * from " + tblName));
+
+ hiveCatalog.dropTable(tablePath, false);
+ }
+
+ @Test
+ public void testInsertOverwrite() throws Exception {
+ String dbName = "default";
+ String tblName = "dest";
+ RowTypeInfo rowTypeInfo = createDestTable(dbName, tblName, 0);
+ ObjectPath tablePath = new ObjectPath(dbName, tblName);
+
+ ExecutionEnvironment execEnv = ExecutionEnvironment.createLocalEnvironment(1);
+ BatchTableEnvironment tableEnv = BatchTableEnvironment.create(execEnv);
+
+ // write some data and verify
+ List<Row> toWrite = generateRecords(5);
+ tableEnv.registerDataSet("src", execEnv.fromCollection(toWrite, rowTypeInfo));
+
+ CatalogTable table = (CatalogTable) hiveCatalog.getTable(tablePath);
+ tableEnv.registerTableSink("destSink", new HiveTableSink(new JobConf(hiveConf), tablePath, table));
+ tableEnv.sql("select * from src").insertInto("destSink");
+ execEnv.execute();
+
+ verifyWrittenData(toWrite, hiveShell.executeQuery("select * from " + tblName));
+
+ // write some data to overwrite existing data and verify
+ toWrite = generateRecords(3);
Review comment:
this test doesn't seem to serve its purpose. IIUIC, `generateRecords(3)` will generate the exact same three records as the first 3 rows from `generateRecords(5)` at line 240, and thus even if the `setOverwrite()` doesn't funciton, the table rows will remain the same, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services