[jira] [Assigned] (SPARK-29911) Cache table may memory leak when session closed

["Sean R. Owen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-29911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean R. Owen reassigned SPARK-29911:
------------------------------------

    Assignee: Lantao Jin

> Cache table may memory leak when session closed
> -----------------------------------------------
>
>                 Key: SPARK-29911
>                 URL: https://issues.apache.org/jira/browse/SPARK-29911
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Lantao Jin
>            Assignee: Lantao Jin
>            Priority: Major
>         Attachments: Screen Shot 2019-11-15 at 2.03.49 PM.png
>
>
> How to reproduce:
> 1. create a local temporary view v1
> 2. cache it in memory
> 3. close session without drop v1.
> The application will hold the memory forever. In a long running thrift server scenario. It's worse.
> {code}
> 0: jdbc:hive2://localhost:10000> CACHE TABLE testCacheTable AS SELECT 1;
> CACHE TABLE testCacheTable AS SELECT 1;
> +---------+--+
> | Result  |
> +---------+--+
> +---------+--+
> No rows selected (1.498 seconds)
> 0: jdbc:hive2://localhost:10000> !close
> !close
> Closing: 0: jdbc:hive2://localhost:10000
> 0: jdbc:hive2://localhost:10000 (closed)> !connect 'jdbc:hive2://localhost:10000'
> !connect 'jdbc:hive2://localhost:10000'
> Connecting to jdbc:hive2://localhost:10000
> Enter username for jdbc:hive2://localhost:10000:
> lajin
> Enter password for jdbc:hive2://localhost:10000:
> ***
> Connected to: Spark SQL (version 3.0.0-SNAPSHOT)
> Driver: Hive JDBC (version 1.2.1.spark2)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> 1: jdbc:hive2://localhost:10000> select * from testCacheTable;
> select * from testCacheTable;
> Error: Error running query: org.apache.spark.sql.AnalysisException: Table or view not found: testCacheTable; line 1 pos 14;
> 'Project [*]
> +- 'UnresolvedRelation [testCacheTable] (state=,code=0)
> {code}
>  !Screen Shot 2019-11-15 at 2.03.49 PM.png! 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org