[jira] [Commented] (FLINK-15003) Use embedded mode to improve hive test performance

["Rui Li (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986667#comment-16986667 ] 

Rui Li commented on FLINK-15003:
--------------------------------

I tried adding embedded mode to {{FlinkStandaloneHiveRunner}} but it's not very helpful with reducing test time. On my laptop, the time spent with a test case is 28s (standalone) vs 25s (embedded). So I think most of the time is spent setting up the metastore instance, no matter whether it's embedded or standalone.


 For tests that don't need the hive shell (to perform operations on hive side), you can simply call {{HiveTestUtils::createHiveCatalog}} to create a HiveCatalog backed by an embedded derby store.


 If hive shell is needed, you still need to use {{FlinkStandaloneHiveRunner}} at the moment. I'll investigate whether we can reduce the time for creating metastore instances.

> Use embedded mode to improve hive test performance
> --------------------------------------------------
>
>                 Key: FLINK-15003
>                 URL: https://issues.apache.org/jira/browse/FLINK-15003
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Hive
>            Reporter: Jingsong Lee
>            Priority: Major
>             Fix For: 1.11.0
>
>
> Now, our hive tests are too slow, for many tests, This leads us not to cover all formats well.
> We can just use embedded mode to improve test performance.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)