You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2026/10/11 06:36:48 UTC
[PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
viirya opened a new pull request, #59360:
URL: https://github.com/apache/spark/pull/59360
### What changes were proposed in this pull request?
The bounded line reader used by `ReplayListenerBus.replay(InputStream, ...)` (added in SPARK-59407, byte accounting fixed in SPARK-59804) called `BufferedReader.read()` once per character. This PR makes it read decoded characters in bulk instead:
- `boundedLines` reads from the `InputStreamReader` into an 8192-char array and scans that array for `\n`, instead of wrapping the reader in a `BufferedReader` and calling `read()` per character. `BufferedReader.read()` takes a lock on every call.
- `BoundedLineBuffer` gets `append(chars, start, end)`, which has the same result as appending the characters one by one: it counts UTF-8 bytes per character, stops counting at the first character that exceeds the allowance, and appends the range in one call if it fits or releases the retained prefix otherwise. `append(c)` is kept and shares the width and release helpers.
The reader's semantics are unchanged: the UTF-8 content byte limit with the trailing-CR allowance, physical line indices including skipped lines, strict decoding, and releasing the buffer of an over-long line while it is drained. The `Iterator[String]` overload of `replay` is not affected.
### Why are the changes needed?
Since SPARK-59407, line splitting in this reader is several times slower than the previous `Source.getLines()`/`readLine()` (measured at about 14x in the review of #59074). Every replay through the `InputStream` overload pays this: loading application UIs in the History Server, event log compaction, and listing with fast in-progress parsing.
It also matters for #59327 (SPARK-60109), which routes the History Server end-event reparse through this reader. For compressed logs, that reparse reads most of the decompressed file, because the skip target is computed from the compressed length, so without this change the listing of completed applications would become several times slower.
A local measurement of a full replay through `replay(InputStream, ...)` with the History Server listing filter (210 MB of uncompressed event lines built from `spark-events/local-1642039451826` with randomized digits, median of 3 runs, JDK 21):
| codec | file size | decompression only | `Source.getLines()` + `replay(Iterator)` | bounded reader before | bounded reader after |
|---|---|---|---|---|---|
| lz4 | 56 MB | 0.08 s | 0.21 s | 1.13 s | 0.22 s |
| zstd | 16 MB | 0.06 s | 0.17-0.18 s | 1.10 s | 0.20 s |
### Does this PR introduce _any_ user-facing change?
No behavior change. Replaying event logs through the bounded reader (History Server UI loading, listing and event log compaction) is faster.
### How was this patch tested?
New tests in `ReplayListenerSuite`:
- `SPARK-60158: Line buffer range appends match single-character appends`: randomized characters (ASCII, CR, 2- and 3-byte characters, surrogate halves), limits and range splits; the result, the retained length and the release of the buffer match appending one character at a time.
- `SPARK-60158: Replay splits lines across read buffers like a reference splitter`: randomized inputs with lines up to 20000 characters, so that lines and multi-byte characters cross the 8192-char read buffer, compared with a reference that splits on `\n`, drops one trailing CR and keeps lines within the byte limit.
The existing SPARK-59804 tests (byte limit grid across the read buffer boundary, physical line numbers, skipped line diagnostics, malformed UTF-8 and stream IO errors) pass unchanged. `ReplayListenerSuite`, `FsHistoryProviderSuite` (RocksDB backend), `EventLogFileCompactorSuite`, `core/scalastyle` and `core/Test/scalastyle` pass.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #59360:
URL: https://github.com/apache/spark/pull/59360#discussion_r4247340459
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -275,19 +294,44 @@ private[spark] object ReplayListenerBus {
def append(c: Char): Unit = {
Review Comment:
**[2] Minor: `append(c: Char)` is now used only by tests.**
`fetchLine` only calls the range overload. That leaves two copies of the overflow and release logic, and only one of them runs in production. It also means the SPARK-59804 test `Replay line buffer stops retaining oversized content` now checks the path that replay no longer uses. Removing `append(c)` and moving that test to the range overload, or making `append(c)` delegate to `append(Array(c), 0, 1)`, would keep a single implementation. The equivalence test would then only be needed if both methods stay.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -116,15 +130,20 @@ private[spark] class ReplayListenerBus(
@tailrec private def fetchLine(): (String, Int) = {
val buffer = new BoundedLineBuffer(effectiveMaxLineLength)
- var c = reader.read()
- if (c == -1) {
+ if (!fill()) {
null
} else {
val index = lineIndex
lineIndex += 1
- while (c != -1 && c != '\n') {
- buffer.append(c.toChar)
- c = reader.read()
+ var endOfLine = false
+ while (!endOfLine && fill()) {
+ var i = pos
+ while (i < len && chars(i) != '\n') {
Review Comment:
**[3] Minor: each chunk is scanned twice.**
This loop looks for `\n`, and then `BoundedLineBuffer.append(chars, start, end)` walks the same range again to add up `utf8Width`. Counting the width in this loop (or passing it to the buffer) would make it a single pass. After the line goes over the limit, only the `\n` search is needed.
##########
core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala:
##########
@@ -207,6 +207,56 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
assert(diagnostic.length < 1200)
}
+ test("SPARK-60158: Line buffer range appends match single-character appends") {
+ val alphabet = Array('x', '\r', 0x00e9.toChar, 0x4e2d.toChar, 0xd83d.toChar, 0xde00.toChar)
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 2000) {
+ val chars = Array.fill(random.nextInt(64))(alphabet(random.nextInt(alphabet.length)))
+ val limit = random.nextInt(80)
+ val single = new ReplayListenerBus.BoundedLineBuffer(limit)
+ chars.foreach(single.append)
+ val ranged = new ReplayListenerBus.BoundedLineBuffer(limit)
+ var start = 0
+ while (start < chars.length) {
+ val end = math.min(chars.length, start + random.nextInt(8))
+ ranged.append(chars, start, end)
+ start = end
+ }
+ withClue(s"chars=${chars.map(_.toInt).mkString(",")} limit=$limit: ") {
+ assert(ranged.result() == single.result())
+ assert(ranged.length == single.length)
+ assert((ranged.capacity == 0) == (single.capacity == 0))
+ }
+ }
+ }
+
+ test("SPARK-60158: Replay splits lines across read buffers like a reference splitter") {
+ val alphabet = Seq("x", "\r", 0x00e9.toChar.toString, 0x4e2d.toChar.toString,
+ new String(Character.toChars(0x1f600)))
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 50) {
+ val lines = Seq.fill(random.nextInt(20)) {
+ val length = if (random.nextInt(4) == 0) random.nextInt(20000) else random.nextInt(100)
+ Seq.fill(length)(alphabet(random.nextInt(alphabet.length))).mkString
+ }
+ val data = lines.mkString("\n") + (if (random.nextBoolean()) "\n" else "")
+ val limit = 1 + random.nextInt(30000)
+ // Reference: split on '\n', drop one trailing CR, keep lines within the byte limit.
+ val expected = (if (data.isEmpty) Seq.empty else data.split("\n", -1).toSeq)
+ .dropRight(if (data.endsWith("\n")) 1 else 0)
+ .map(line => if (line.endsWith("\r")) line.dropRight(1) else line)
+ .filter(_.getBytes(StandardCharsets.UTF_8).length <= limit)
+ val seen = new ArrayBuffer[String]
+ assert(new ReplayListenerBus(limit).replay(
+ new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8)), "reference",
Review Comment:
**[1] Minor: the read pattern of codec streams is not exercised.**
`ByteArrayInputStream.available()` is always positive, so `StreamDecoder` keeps decoding until the 8192-char array is full, and every `reader.read(chars)` here returns a full chunk. With LZ4/ZStd streams, `available()` is often 0, so reads come back short and uneven. A chunk can then end right after a `\r`, right before a `\n`, or be a single `\n`. Wrapping the input in a stream that returns a random 1..N bytes per `read` and reports `available() == 0` would cover the `fill()`/`pos`/`len` handling under those reads.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -116,15 +130,20 @@ private[spark] class ReplayListenerBus(
@tailrec private def fetchLine(): (String, Int) = {
val buffer = new BoundedLineBuffer(effectiveMaxLineLength)
Review Comment:
**[4] Minor: optional fast path for lines that fit in one chunk.**
Each line still gets a new `StringBuilder` (initial capacity 16) that grows by doubling, about 10 copies for a 10 KB line, and is copied once more by `toString`/`substring`. Typical event lines fit in one 8192-char chunk, so when the `\n` is found in the same chunk and the line is within the limit, `new String(chars, pos, i - pos)` would skip the builder. This is optional, since the PR's numbers already match `Source.getLines()`.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -89,13 +89,27 @@ private[spark] class ReplayListenerBus(
val decoder = StandardCharsets.UTF_8.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
- val reader = new BufferedReader(new InputStreamReader(logData, decoder))
+ // Read decoded characters in bulk and scan them for '\n', instead of calling read() per
+ // character on a BufferedReader, which takes a lock on every call.
+ val reader = new InputStreamReader(logData, decoder)
Review Comment:
**[5] Design question (non-blocking): splitting on bytes.**
In UTF-8 the byte `0x0A` never appears inside a multi-byte sequence, so lines could be split and limited on raw bytes, and only the accepted lines decoded strictly. That gives the exact byte count without the per-char `utf8Width` rule, and an over-long line is skipped without being decoded. The trade-off is that malformed UTF-8 inside a skipped line would no longer fail the replay. I'm fine with the current approach. I'm only mentioning it in case the remaining overhead matters later.
##########
core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala:
##########
@@ -207,6 +207,56 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
assert(diagnostic.length < 1200)
}
+ test("SPARK-60158: Line buffer range appends match single-character appends") {
+ val alphabet = Array('x', '\r', 0x00e9.toChar, 0x4e2d.toChar, 0xd83d.toChar, 0xde00.toChar)
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 2000) {
+ val chars = Array.fill(random.nextInt(64))(alphabet(random.nextInt(alphabet.length)))
+ val limit = random.nextInt(80)
+ val single = new ReplayListenerBus.BoundedLineBuffer(limit)
+ chars.foreach(single.append)
+ val ranged = new ReplayListenerBus.BoundedLineBuffer(limit)
+ var start = 0
+ while (start < chars.length) {
+ val end = math.min(chars.length, start + random.nextInt(8))
+ ranged.append(chars, start, end)
+ start = end
+ }
+ withClue(s"chars=${chars.map(_.toInt).mkString(",")} limit=$limit: ") {
+ assert(ranged.result() == single.result())
+ assert(ranged.length == single.length)
+ assert((ranged.capacity == 0) == (single.capacity == 0))
+ }
+ }
+ }
+
+ test("SPARK-60158: Replay splits lines across read buffers like a reference splitter") {
Review Comment:
**[6] Minor: line indices across read buffers are not covered.**
This test sees only line contents through `eventsFilter`, not the physical index that `boundedLines` returns. The existing `Replay reports the physical location of skipped lines` input (about 4 KB) fits in one read buffer. So no test checks that `lineIndex` stays right when a skipped line spans several `fill()` calls. An example is a 20000-char skipped line followed by a malformed line, where the `Malformed line #N` and `first skipped line` numbers should still be the physical ones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #59360:
URL: https://github.com/apache/spark/pull/59360#discussion_r4250458249
##########
core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala:
##########
@@ -207,6 +207,56 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
assert(diagnostic.length < 1200)
}
+ test("SPARK-60158: Line buffer range appends match single-character appends") {
+ val alphabet = Array('x', '\r', 0x00e9.toChar, 0x4e2d.toChar, 0xd83d.toChar, 0xde00.toChar)
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 2000) {
+ val chars = Array.fill(random.nextInt(64))(alphabet(random.nextInt(alphabet.length)))
+ val limit = random.nextInt(80)
+ val single = new ReplayListenerBus.BoundedLineBuffer(limit)
+ chars.foreach(single.append)
+ val ranged = new ReplayListenerBus.BoundedLineBuffer(limit)
+ var start = 0
+ while (start < chars.length) {
+ val end = math.min(chars.length, start + random.nextInt(8))
+ ranged.append(chars, start, end)
+ start = end
+ }
+ withClue(s"chars=${chars.map(_.toInt).mkString(",")} limit=$limit: ") {
+ assert(ranged.result() == single.result())
+ assert(ranged.length == single.length)
+ assert((ranged.capacity == 0) == (single.capacity == 0))
+ }
+ }
+ }
+
+ test("SPARK-60158: Replay splits lines across read buffers like a reference splitter") {
+ val alphabet = Seq("x", "\r", 0x00e9.toChar.toString, 0x4e2d.toChar.toString,
+ new String(Character.toChars(0x1f600)))
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 50) {
+ val lines = Seq.fill(random.nextInt(20)) {
+ val length = if (random.nextInt(4) == 0) random.nextInt(20000) else random.nextInt(100)
+ Seq.fill(length)(alphabet(random.nextInt(alphabet.length))).mkString
+ }
+ val data = lines.mkString("\n") + (if (random.nextBoolean()) "\n" else "")
+ val limit = 1 + random.nextInt(30000)
+ // Reference: split on '\n', drop one trailing CR, keep lines within the byte limit.
+ val expected = (if (data.isEmpty) Seq.empty else data.split("\n", -1).toSeq)
+ .dropRight(if (data.endsWith("\n")) 1 else 0)
+ .map(line => if (line.endsWith("\r")) line.dropRight(1) else line)
+ .filter(_.getBytes(StandardCharsets.UTF_8).length <= limit)
+ val seen = new ArrayBuffer[String]
+ assert(new ReplayListenerBus(limit).replay(
+ new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8)), "reference",
Review Comment:
Added in 714c458a1f4. The reference splitter test now also runs each input through a stream that returns 1 to N bytes per `read` (N is 1, 3, 64 or 8192) and reports `available() == 0`, so chunks end at arbitrary points, including right after a `\r` and right before or at a `\n`. A mutation that does not treat a `\n` at the start of a new chunk as a line end passes all the existing tests but fails this one.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -275,19 +294,44 @@ private[spark] object ReplayListenerBus {
def append(c: Char): Unit = {
Review Comment:
Done in 714c458a1f4, together with [3]. `append(c)` and the range `append` are replaced by a single `appendUntilNewline(chars, start, end)`, which is what `fetchLine` calls, so there is one copy of the limit and release logic. The SPARK-59804 buffer test now uses it, and the equivalence test is replaced by one that splits random lines at random points and compares the result with a reference (drop one trailing CR, keep the line if its UTF-8 length fits), and checks that the buffer is released once the line exceeds the limit plus the CR allowance.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -116,15 +130,20 @@ private[spark] class ReplayListenerBus(
@tailrec private def fetchLine(): (String, Int) = {
val buffer = new BoundedLineBuffer(effectiveMaxLineLength)
- var c = reader.read()
- if (c == -1) {
+ if (!fill()) {
null
} else {
val index = lineIndex
lineIndex += 1
- while (c != -1 && c != '\n') {
- buffer.append(c.toChar)
- c = reader.read()
+ var endOfLine = false
+ while (!endOfLine && fill()) {
+ var i = pos
+ while (i < len && chars(i) != '\n') {
Review Comment:
Changed in 714c458a1f4: `appendUntilNewline` finds the `\n` and adds up the UTF-8 width in the same loop, and once the line goes over the limit it only searches for `\n`. The full-replay numbers stay the same (lz4 0.22 s, zstd 0.19 s, vs 0.20 s / 0.17 s for `Source.getLines()`).
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -116,15 +130,20 @@ private[spark] class ReplayListenerBus(
@tailrec private def fetchLine(): (String, Int) = {
val buffer = new BoundedLineBuffer(effectiveMaxLineLength)
Review Comment:
I'd prefer to leave this out. The numbers are already on par with `Source.getLines()`, and the fast path would need its own handling of the trailing CR and the limit, which would be a second place to keep in sync with `BoundedLineBuffer`.
##########
core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala:
##########
@@ -89,13 +89,27 @@ private[spark] class ReplayListenerBus(
val decoder = StandardCharsets.UTF_8.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
- val reader = new BufferedReader(new InputStreamReader(logData, decoder))
+ // Read decoded characters in bulk and scan them for '\n', instead of calling read() per
+ // character on a BufferedReader, which takes a lock on every call.
+ val reader = new InputStreamReader(logData, decoder)
Review Comment:
Thanks. I'd keep decoding every line for now: SPARK-59804 kept failing the replay on malformed UTF-8 inside a skipped line (`Replay still rejects malformed UTF-8 in skipped lines`), and splitting on raw bytes would drop that. Raw-byte splitting also comes up for an incomplete UTF-8 sequence at EOF (SPARK-60114), so I'll look at it there if the remaining overhead matters.
##########
core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala:
##########
@@ -207,6 +207,56 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
assert(diagnostic.length < 1200)
}
+ test("SPARK-60158: Line buffer range appends match single-character appends") {
+ val alphabet = Array('x', '\r', 0x00e9.toChar, 0x4e2d.toChar, 0xd83d.toChar, 0xde00.toChar)
+ val random = new scala.util.Random(60158)
+ for (_ <- 0 until 2000) {
+ val chars = Array.fill(random.nextInt(64))(alphabet(random.nextInt(alphabet.length)))
+ val limit = random.nextInt(80)
+ val single = new ReplayListenerBus.BoundedLineBuffer(limit)
+ chars.foreach(single.append)
+ val ranged = new ReplayListenerBus.BoundedLineBuffer(limit)
+ var start = 0
+ while (start < chars.length) {
+ val end = math.min(chars.length, start + random.nextInt(8))
+ ranged.append(chars, start, end)
+ start = end
+ }
+ withClue(s"chars=${chars.map(_.toInt).mkString(",")} limit=$limit: ") {
+ assert(ranged.result() == single.result())
+ assert(ranged.length == single.length)
+ assert((ranged.capacity == 0) == (single.capacity == 0))
+ }
+ }
+ }
+
+ test("SPARK-60158: Replay splits lines across read buffers like a reference splitter") {
Review Comment:
Added `SPARK-60158: Replay reports physical line numbers for lines spanning read buffers` in 714c458a1f4: a skipped 20000-char line followed by a malformed line, with full reads and with 1-byte and up-to-100-byte reads. It checks `first skipped line: 2` and `Malformed line #4`, and fails if the line index is advanced per chunk instead of per line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #59360:
URL: https://github.com/apache/spark/pull/59360#issuecomment-6133049484
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/a2659a761e96e13e5785f68b9639d6a4c4818434
- merged into branch-4.x https://github.com/apache/spark/commit/679157d660d7461c5a33dad509649b033ad1eb30
- merged into branch-4.3 https://github.com/apache/spark/commit/a4c63cb52722e15cb219606ec4f50c709bc03eac
- merged into branch-4.2 https://github.com/apache/spark/commit/5e0c84619d60d0e3dbc571c775a8dfbf7ad3983a
- merged into branch-4.1 https://github.com/apache/spark/commit/876d98f780ab2b36ab4663e9d5fde0bc4c212d7d
- merged into branch-4.0 https://github.com/apache/spark/commit/826617ea5c0bcfb488b7a8d98c5594d9dbcbef94
- merged into branch-3.5 https://github.com/apache/spark/commit/aca614f72153f89445f4a6e9ef87a89cac0b6ba2
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya closed pull request #59360: [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader
URL: https://github.com/apache/spark/pull/59360
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #59360:
URL: https://github.com/apache/spark/pull/59360#issuecomment-6133069997
Thanks @dongjoon-hyun !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60158][UI] Avoid per-character synchronized reads in the bounded event log line reader [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #59360:
URL: https://github.com/apache/spark/pull/59360#issuecomment-6149764770
> Hi folks @viirya and @dongjoon-hyun, please note that this has broken:
>
> * branch-4.0
>
> * branch-4.1
To clarify that, the broken branches (4.1/4.0/3.5) are not caused by this PR. It is caused by new PyArrow release. I fixed it in https://github.com/apache/spark/pull/59423.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org