You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/08/08 12:39:26 UTC
[GitHub] [spark] yanxiaole opened a new pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
yanxiaole opened a new pull request #29392:
URL: https://github.com/apache/spark/pull/29392
### What changes were proposed in this pull request?
Synchronize listing iteration manually
### Why are the changes needed?
It seems to be a race condition in `FSHistoryProvider` that iterate the listing with high concurrency:
```java
val stale = listing.view(classOf[LogInfo])
.index("lastProcessed")
.last(newLastScanTime - 1)
.asScala
.toList
```
`toList` will iterate items in the listing, if one of listing entries was deleted by some clean thread, `checkForLogs` will throw out a `java.util.NoSuchElementException` and abort current execution.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
The race contention is hard to reproduce, but the current logic seems causing the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyejoe commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
zhouyejoe commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-672598759
Hi, @yanxiaole. I double checked the codes in checkForLogs(). I think SPARK-29043 does actually handles the race condition by filtering out the stale.filterNot(isProcessing), which makes sure that even though in mergeApplicationListing listing.delete may get triggered, there won't be race conditions. How you get your original stacktrace? Are you able to get the stacktrace which shows the race condition between checkForLogs() and replay task? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-672111685
@zhouyejoe You had looked at/fixed something similar IIRC - can you please take a look ? Thx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-671069843
I have pasted two stack traces below as they happened in two different functions. The line numbers are slightly different with the code in `master` because I'm using `branch-3.0` and have done some modifications.
```
20/08/06 21:24:25 ERROR Utils: Uncaught exception in thread spark-history-task-0
java.util.NoSuchElementException: 1^@__main__^@+hdfs://xx/logs/spark/application_1596120560040_553485.lz4.inprogress
at org.apache.spark.util.kvstore.LevelDB.get(LevelDB.java:120)
at org.apache.spark.util.kvstore.LevelDBIterator.next(LevelDBIterator.java:137)
at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at scala.collection.TraversableOnce.size(TraversableOnce.scala:110)
at scala.collection.TraversableOnce.size$(TraversableOnce.scala:108)
at scala.collection.AbstractTraversable.size(Traversable.scala:108)
at org.apache.spark.deploy.history.FsHistoryProvider.$anonfun$cleanLogs$1(FsHistoryProvider.scala:944)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.util.Utils$.tryLog(Utils.scala:1945)
at org.apache.spark.deploy.history.FsHistoryProvider.cleanLogs(FsHistoryProvider.scala:908)
at org.apache.spark.deploy.history.FsHistoryProvider.$anonfun$startPolling$4(FsHistoryProvider.scala:295)
at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1302)
at org.apache.spark.deploy.history.FsHistoryProvider.$anonfun$getRunner$1(FsHistoryProvider.scala:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
```
20/08/06 22:29:19 ERROR FsHistoryProvider: Exception in checking for event log updates
java.util.NoSuchElementException: 1^@__main__^@+hdfs://xx/logs/spark/application_1596120560040_583000.lz4.inprogress
at org.apache.spark.util.kvstore.LevelDB.get(LevelDB.java:120)
at org.apache.spark.util.kvstore.LevelDBIterator.next(LevelDBIterator.java:137)
at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:62)
at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:53)
at scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:184)
at scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:47)
at scala.collection.TraversableLike.to(TraversableLike.scala:678)
at scala.collection.TraversableLike.to$(TraversableLike.scala:675)
at scala.collection.AbstractTraversable.to(Traversable.scala:108)
at scala.collection.TraversableOnce.toList(TraversableOnce.scala:299)
at scala.collection.TraversableOnce.toList$(TraversableOnce.scala:299)
at scala.collection.AbstractTraversable.toList(Traversable.scala:108)
at org.apache.spark.deploy.history.FsHistoryProvider.checkForLogs(FsHistoryProvider.scala:564)
at org.apache.spark.deploy.history.FsHistoryProvider.$anonfun$startPolling$3(FsHistoryProvider.scala:290)
at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1302)
at org.apache.spark.deploy.history.FsHistoryProvider.$anonfun$getRunner$1(FsHistoryProvider.scala:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-672630663
the filter is called after the assignment of variable `stale`, but the race condition is happened in the assignment, the `asScala.toList`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670970887
I'm not sure, what is the problem here? is it this data structure getting modified or something else underneath it in the FS? if the latter, then this synchronization might not be a fix?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-672459512
Hi @zhouyejoe ,
Yes, you are right. I have checked the pool definition, and `checkForLogs()` and `cleanLogs()` won't run at same time, the only possible is the replay tasks. I think it's SPARK-29043 which removed the waiting to improve the performance.
Does that mean I only need to synchronize the delete in `mergeApplicationListing`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-671060582
Yeah, but I'm not seeing any of these operations that cause a delete. If they aren't somehow mutually excluded with a lock too, then this can't fix the problem, no?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #29392:
URL: https://github.com/apache/spark/pull/29392
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyejoe commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
zhouyejoe commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-673037833
@yanxiaole From the stacktrace, there is indeed race condition between the replayTask and checkForLogs()/cleanLogs(), where the latter two has codes which transfers the whole listing.ldb to Scala List, but the replayTask may delete entries from listing.ldb.
I do think we can refactor the PR https://github.com/apache/spark/pull/24966 which introduces the listing.delete in replay task.
Hi, @dongjoon-hyun. Since you were the original author for this PR, do you think there will be race condition? If there is, how should we resolve it? In relayTask, it can put this deletion into a ConcurrentLinkedQueue, and let the cleanLogs() to delete those entries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-671068777
How does this error arise BTW? when did you encounter it and what was the stack trace? I'm not so clear how this comes up in practice.
It sounds good to synchronize but this is serializing all reads now too, which has its own implications
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole edited a comment on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole edited a comment on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081
It happens when data structure getting modified, mainly `delete` causes the problem.
Take `LevelDB` implementation for example, in its iterator's `next` function it will first check `hasNext` then call `get`, if the entry was deleted between two calls then it will throw out a `NoSuchElementException` which will interrupt outside workflow. In our cluster, it either terminates the `cleanLogs` thread or interrupts `checkForLogs` which introduces inconsistency between log and app listing.
https://github.com/apache/spark/blob/2ec9b866285fc059cae6816033babca64b4da7ec/common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDBIterator.java#L125
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-770306238
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670924895
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-714388198
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyejoe commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
zhouyejoe commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-672428047
@yanxiaole I don't think there will be race condition between the checkForLogs() and cleanLogs() thread. These two threads are launched from the same pool, and the thread pool is defined as a single thread pool. So they won't be able to run simultaneously. But with the latest trunk, I do see there is race condition between the replay Task and checkForLogs(). In some former version of Spark, checkForLogs will wait for all the replay tasks to finish then it exists. But now, replay tasks can run simultaneously with checkForLogs() thread, where checkForLogs scanning stale entries will iterate the listing.ldb and replay tasks can trigger the listing.delete if the log file is out of maximum retention time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-671067561
Thanks @srowen , I just updated a new commit to lock the delete operation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670925021
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670961239
> What code that modifies this object is also synchronizing on this?
Sorry, I'll update them.
@srowen , do you think I should add synchronizing at every write and delete call in `FSHistoryProvider` or add synchronizing in the implementations, like `LevelDB` and `InMemoryStore`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670924895
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
yanxiaole commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081
It happened when data structure getting modified, mainly `delete` caused the problem.
Take `LevelDB` implementation for example, in its iterator's `next` function it will first check `hasNext` then call `get`, if the entry was deleted between two calls then there will be a `NoSuchElementException` which will interrupt outside workflow. In our cluster, it either terminate the `cleanLogs` thread or interrupt `checkForLogs` which introduces inconsistency between log and app listing.
https://github.com/apache/spark/blob/2ec9b866285fc059cae6816033babca64b4da7ec/common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDBIterator.java#L125
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29392:
URL: https://github.com/apache/spark/pull/29392#issuecomment-670925021
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org