You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Michael Allman <mi...@videoamp.com> on 2015/03/19 18:51:14 UTC

Spark SQL ExternalSorter not stopped

I've examined the experimental support for ExternalSorter in Spark SQL, and it does not appear that the external sorted is ever stopped (ExternalSorter.stop). According to the API documentation, this suggests a resource leak. Before I file a bug report in Jira, can someone familiar with the codebase confirm this is indeed a bug?

Thanks,

Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org

Re: Spark SQL ExternalSorter not stopped

Posted by Yin Huai <yh...@databricks.com>.

Hi Michael,

Thanks for reporting it. Yes, it is a bug. I have created
https://issues.apache.org/jira/browse/SPARK-6437 to track it.

Thanks,

Yin

On Thu, Mar 19, 2015 at 10:51 AM, Michael Allman <mi...@videoamp.com>
wrote:

> I've examined the experimental support for ExternalSorter in Spark SQL,
> and it does not appear that the external sorted is ever stopped
> (ExternalSorter.stop). According to the API documentation, this suggests a
> resource leak. Before I file a bug report in Jira, can someone familiar
> with the codebase confirm this is indeed a bug?
>
> Thanks,
>
> Michael
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>