You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rajeshbalamohan <gi...@git.apache.org> on 2017/09/11 01:48:37 UTC
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
GitHub user rajeshbalamohan opened a pull request:
https://github.com/apache/spark/pull/19184
[SPARK-21971][CORE] Too many open files in Spark due to concurrent fi…
…les being opened
## What changes were proposed in this pull request?
In UnsafeExternalSorter::getIterator(), for every spillWriter a file is opened in UnsafeSorterSpillReader and these files get closed later point in time as a part of close() call.
However, when large number of spill files are present, number of files opened increases to a great extent and ends up throwing "Too many files" open exception.
This can easily be reproduced with TPC-DS Q67 at 1 TB scale in multi node cluster with multiple cores per executor.
There are ways to reduce the number of spill files that are generated in Q67. E.g, increase "spark.sql.windowExec.buffer.spill.threshold" where 4096 is the default. Another option is to increase ulimit to much higher values.
But those are workarounds.
This PR reduces the number of files that are kept open at in UnsafeSorterSpillReader.
## How was this patch tested?
Manual testing of Q67 in 1 TB and 10 TB scale on multi node cluster.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rajeshbalamohan/spark SPARK-21971
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19184.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19184
----
commit dcc2960d5f60add9bfd9446df59b0d0d06365947
Author: Rajesh Balamohan <rb...@apache.org>
Date: 2017-09-11T01:36:12Z
[SPARK-21971][CORE] Too many open files in Spark due to concurrent files being opened
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19184
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137988200
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
The valid fix should be to import a new config to control the concurrent number of opened spill files, it also means you should use some data structure to keep and track the request of open spill files.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19184
**[Test build #81628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81628/testReport)** for PR 19184 at commit [`ea5f9d9`](https://github.com/apache/spark/commit/ea5f9d9903185690670da6384dbd6d2b08b8177f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137981905
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
--- End diff --
This comment looks confusing. Maybe `It is the time to initialize and hold the input stream of the spill file for loading records. Keeps the input stream open too early will very possibly encounter too many file open issue.`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19184
After discussed with @mridulm offline. Though the patch here cannot address the issue of `getSortedIterator` - which uses a PriorityQueue, somehow it solves the problem of `getIterator(...)` which doesn't require merge. And in this specific case, it uses `getIterator` in `ExternalAppendOnlyUnsafeRowArray` and encounter too many file opened issue.
So this fix could solve the problem of `getIterator`, but no harm to `getSortedIterator`. Maybe we could accept it as a partial/point fix here.
What do you think @viirya @kiszk @maropu @jiangxb1987 ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by rajeshbalamohan <gi...@git.apache.org>.
Github user rajeshbalamohan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137973976
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
Good point. PR has been tried with queries involving window functions (e.g Q67) for which it worked fine.
During spill merges (esp getSortedIterator), it is possible to encounter too many open files issue.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137981394
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -72,12 +77,27 @@ public UnsafeSorterSpillReader(
bufferSizeBytes = DEFAULT_BUFFER_SIZE_BYTES;
}
+ try (InputStream bs = new NioBufferedFileInputStream(file, (int) bufferSizeBytes);
--- End diff --
Please add a comment here to say we don't need to hold the file open until we actually want to load the records.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19184
hmm, shouldn't we just change system config to increase the limit of open file?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19184
@viirya @jerryshao To take a step back here.
This specific issue is applicable to window operations and not to shuffle.
In shuffle, you a much larger volume of data written per file vs 4k records per file for window operation.
To get to 9k files with shuffle, you are typically processing a TB or more data per shuffle task (unless executor is strapped of memory and spilt large number of files).
On other hand, with 4k window size (the default in spark), getting to 9k files is possible within a single task.
From what I see, there is actually no functional/performance reason to keep all the files opened, unlike in shuffle.
Having said that, there is an additional cost we pay with this approach.
With N files, we incur an additional cost of `N * cost(open + read int + close)`
Practically though, the impact is much lower when compared to current code (since re-open + read will be disk read friendly).
While getting it fixed for all cases would be ideal, the solution for window operation does not transfer to shuffle (and vice versa) due to the difference in the nature of how files are used in both.
In case I missed something here, please let me know.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19184
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by rajeshbalamohan <gi...@git.apache.org>.
Github user rajeshbalamohan commented on the issue:
https://github.com/apache/spark/pull/19184
Thanks @mridulm , @jerryshao , @viirya . closing this PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19184
@rajeshbalamohan Thanks for updating. I think we need a complete fix as previous comments from the reviewers @jerryshao @kiszk @jiangxb1987 suggested. Can you try to fix this according to the comments? Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137971517
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
Can this solve the too many file open issue? When we do merging the readers, it is possibly that all the readers in priority queue still have records and are asked for records (so their files open). You still can encounter too many file open issue.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19184
@jerryshao Actually the second half of your comment is not valid in this case.
The PR is not targeting the merge sort in this case, but relevant when iterating over all tuples.
`UnsafeExternalSorter` has two methods to iterate over the tuples.
You are referring to `getSortedIterator` - which uses a PriorityQueue and requires all files to be opened at the same time (so that it can return a sorted iterator).
The primary usecase of this PR is for `getIterator` - where we are simply iterating over all tuples : and used in `ExternalAppendOnlyUnsafeRowArray` for example : there is no need to sort here.
This is used by various `WindowFunctionFrame` implementations for example.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19184
Thanks @jerryshao and @mridulm for investigating this further. It is very reasonable. I think we don't need this fix as the spill won't be too frequent in window operations now.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19184
**[Test build #81614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81614/testReport)** for PR 19184 at commit [`dcc2960`](https://github.com/apache/spark/commit/dcc2960d5f60add9bfd9446df59b0d0d06365947).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by rajeshbalamohan <gi...@git.apache.org>.
Github user rajeshbalamohan commented on the issue:
https://github.com/apache/spark/pull/19184
I got into this with the limit of 32K. "unlimited" is another option which can be a workaround for this. But that may not be a preferable option in production systems. For e.g, with Q67 I observed 9000+ spill files in the task. And with multiple tasks per executor, it ended up easily reaching the limits.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137980253
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
I think you need to first describe more about how to fix this issue in the description.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19184
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81614/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137981565
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -72,12 +77,27 @@ public UnsafeSorterSpillReader(
bufferSizeBytes = DEFAULT_BUFFER_SIZE_BYTES;
}
+ try (InputStream bs = new NioBufferedFileInputStream(file, (int) bufferSizeBytes);
+ DataInputStream dataIn = new DataInputStream(serializerManager.wrapStream(blockId, bs))) {
+ this.numRecords = dataIn.readInt();
+ this.numRecordsRemaining = numRecords;
+ }
+
+ this.buffSize = bufferSizeBytes;
+ this.file = file;
+ this.blockId = blockId;
+ this.serializerManager = serializerManager;
+
+ logger.debug("bufSize: {}, file: {}, records: {}", buffSize, file, this.numRecords);
--- End diff --
Is this log useful? If the number of spill readers is so many, I guess we don't want to see so many log info?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137981999
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
I agree with @viirya , we're using priority queue to do merge sort, this will turn out to be all the readers in the priority queue is opened, so still cannot solve this issue.
I think a valid fix is to control the number of concurrent merged files, like MR's `io.sort.factor`.
Also we still need to address similar issue in `ExternalSorter` and other places in Shuffle.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137982497
--- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many files exception)
+ initStreams();
+ }
--- End diff --
IIUC, this PR does not reduce the number of total open files. Since this PR tries to open files when they are required, this PR may reduce possibility of occurring an error of too may open files.
As @viirya pointed out, it is necessary to provide a feature to control the number of opening files at one point (e.g. priority queue).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by rajeshbalamohan <gi...@git.apache.org>.
Github user rajeshbalamohan commented on the issue:
https://github.com/apache/spark/pull/19184
Thanks @viirya . I have updated the patch to address your comments.
This fixes the "too many files open" issue for (e.g Q67, Q72, Q14 etc) which involves window functions; but for the merger the issue needs to be addressed still. Agreed that this would be partial patch.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19184
**[Test build #81628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81628/testReport)** for PR 19184 at commit [`ea5f9d9`](https://github.com/apache/spark/commit/ea5f9d9903185690670da6384dbd6d2b08b8177f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19184
**[Test build #81614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81614/testReport)** for PR 19184 at commit [`dcc2960`](https://github.com/apache/spark/commit/dcc2960d5f60add9bfd9446df59b0d0d06365947).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19184
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81628/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19184
Thanks to @jerryshao for pointing me to SPARK-21595.
The tests which @rajeshbalamohan did were with a version which did not include the changes in SPARK-21595; and unfortunately my local repo was not updated to reflect this too.
Given the changes in #18843 this PR is no longer relevant.
Can you close this @rajeshbalamohan ? Thanks.
In your tests, you can set the threshold to 512M - since that is the value going forward in spark 2.3
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19184
Hi @mridulm , sorry for late response. I agree with you that the scenario is different between here and shuffle, but the underlying structure and solutions to spill data is the same, so the problem is the same. While in the shuffle side, we could control the memory size to hold more data before spilling to avoid too many spills, but as you mentioned here we cannot do it.
Yes it is not necessary to open all the files beforehand. But since we're using priority queue to do merge sort, which will make all the file handler opened very likely. And this fix only reduces the chances to encounter too many files issue. Maybe we can call this fix as an intermittent fix, what do you think?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19184: [SPARK-21971][CORE] Too many open files in Spark ...
Posted by rajeshbalamohan <gi...@git.apache.org>.
Github user rajeshbalamohan closed the pull request at:
https://github.com/apache/spark/pull/19184
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19184: [SPARK-21971][CORE] Too many open files in Spark due to ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19184
cc @cloud-fan @jiangxb1987
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org