You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/06/12 23:55:03 UTC
[PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
aglinxinyuan opened a new pull request, #5657:
URL: https://github.com/apache/texera/pull/5657
### What changes were proposed in this PR?
Pin behavior of two previously-uncovered helpers in `common/workflow-operator`. No production-code changes.
| Spec | Source class | Tests |
| --- | --- | --- |
| `AutoClosingIteratorSpec` | `AutoClosingIterator` | 10 |
| `UnionOpExecSpec` | `UnionOpExec` | 7 |
Both spec files follow the `<srcClassName>Spec.scala` one-to-one convention.
**Behavior pinned — `AutoClosingIterator`**
| Surface | Contract |
| --- | --- |
| `hasNext` (non-empty source) | `true`; `onClose` not invoked |
| `hasNext` (exhausted source) | `false`; `onClose` invoked exactly once |
| Repeated `hasNext` after exhaustion | does NOT re-fire `onClose` (`alreadyClosed` guard) |
| `next()` | delegates straight to the wrapped iterator (in order) |
| Full traversal via `toList` | yields every element; `onClose` fires once at the end |
| Already-empty source | first `hasNext` returns `false` and fires `onClose` |
| Mid-iteration | `onClose` stays un-fired between elements |
| `onClose` throws | exception propagates (no swallowing) |
| `onClose` throws + retry | current impl re-fires `onClose` (assignment to `alreadyClosed` runs AFTER `onClose()`); characterization pins this behavior |
**Behavior pinned — `UnionOpExec`**
| Surface | Contract |
| --- | --- |
| `processTuple(tuple, port = 0)` | yields a single-element iterator containing the tuple |
| `processTuple` (any port) | port-agnostic — same tuple passes through for ports 0, 1, 5, 99, MaxValue, -1 |
| Tuple identity | pass-through preserves the exact `Tuple` reference (no copy) |
| Successive calls | each returns an independent fresh iterator (no shared cursor) |
| Per-call iterator | yields exactly one element |
| `null` tuple | passes through as `null` (the impl does not null-check) |
| Type contract | `UnionOpExec` is an `OperatorExecutor` |
### Any related issues, documentation, discussions?
Closes #5653.
### How was this PR tested?
Pure unit-test additions; verified locally with:
- `sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.source.scan.AutoClosingIteratorSpec org.apache.texera.amber.operator.union.UnionOpExecSpec"` — 17 tests, all green
- `sbt scalafmtCheckAll` — clean
- CI to confirm
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.7 [1M context])
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan commented on code in PR #5657:
URL: https://github.com/apache/texera/pull/5657#discussion_r3406903150
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ // Step-by-step assertion: after each hasNext that returns TRUE,
+ // onClose MUST still be un-fired. Only the hasNext that returns
+ // false may flip `fired`. A bug that prematurely closed during a
+ // truthy hasNext would surface here, not just at the loop's exit.
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 1 is reachable")
+ assert(it.next() == 1)
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 2 is reachable")
+ assert(it.next() == 2)
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 3 is reachable")
+ assert(it.next() == 3)
+ // The final hasNext returns false — THIS is the call that fires onClose.
+ assert(!it.hasNext)
+ assert(fired, "after hasNext first returns false, onClose must have fired")
+ }
+
+ // ---------------------------------------------------------------------------
+ // onClose exception propagation
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator" should
+ "propagate exceptions thrown from onClose (no swallowing)" in {
+ val it = new AutoClosingIterator[Int](
+ Iterator.empty,
+ () => throw new IllegalStateException("close failed")
+ )
+ val ex = intercept[IllegalStateException] {
+ it.hasNext
+ }
+ assert(ex.getMessage == "close failed")
+ }
+
+ it should
+ "re-invoke onClose on a retry when the previous onClose threw (alreadyClosed is set AFTER onClose)" in {
+ // Reading the production code: `alreadyClosed = true` runs AFTER
+ // `onClose()`. So if onClose throws, alreadyClosed stays false and
+ // a second hasNext will re-invoke onClose. This is the OPPOSITE of
+ // an "alreadyClosed once close was attempted" contract — characterize
+ // the current (brittle) behavior so a refactor that swaps the order
+ // (running `alreadyClosed = true` BEFORE `onClose()`) surfaces here.
Review Comment:
Filed as a follow-up `starter-task` (Type=Bug): #5660. Once the production fix lands, the characterization test added in this PR will start failing and should be replaced with a positive assertion that a throwing `onClose` is not retried — full plan is in the issue body.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5657:
URL: https://github.com/apache/texera/pull/5657#issuecomment-4696636533
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/5657?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 52.96%. Comparing base ([`73c76f5`](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`18da18f`](https://app.codecov.io/gh/apache/texera/commit/18da18f27adc86cce6057f40a80af02ac5ef5be0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #5657 +/- ##
=========================================
Coverage 52.96% 52.96%
Complexity 2522 2522
=========================================
Files 1075 1075
Lines 41743 41743
Branches 4513 4513
=========================================
Hits 22109 22109
Misses 18329 18329
Partials 1305 1305
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.42% <ø> (ø)` | | |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.36% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.76% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `1.65% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.71% <ø> (ø)` | | |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.06% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.38% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.72% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [python](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.74% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/5657/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `58.69% <ø> (ø)` | | Carriedforward from [73c76f5](https://app.codecov.io/gh/apache/texera/commit/73c76f51920b0900de67bbc0baa1ee5be5b87bf0?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/5657?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on code in PR #5657:
URL: https://github.com/apache/texera/pull/5657#discussion_r3406887613
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ // Step-by-step assertion: after each hasNext that returns TRUE,
+ // onClose MUST still be un-fired. Only the hasNext that returns
+ // false may flip `fired`. A bug that prematurely closed during a
+ // truthy hasNext would surface here, not just at the loop's exit.
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 1 is reachable")
+ assert(it.next() == 1)
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 2 is reachable")
+ assert(it.next() == 2)
+ assert(it.hasNext); assert(!fired, "onClose must not fire while element 3 is reachable")
+ assert(it.next() == 3)
+ // The final hasNext returns false — THIS is the call that fires onClose.
+ assert(!it.hasNext)
+ assert(fired, "after hasNext first returns false, onClose must have fired")
+ }
+
+ // ---------------------------------------------------------------------------
+ // onClose exception propagation
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator" should
+ "propagate exceptions thrown from onClose (no swallowing)" in {
+ val it = new AutoClosingIterator[Int](
+ Iterator.empty,
+ () => throw new IllegalStateException("close failed")
+ )
+ val ex = intercept[IllegalStateException] {
+ it.hasNext
+ }
+ assert(ex.getMessage == "close failed")
+ }
+
+ it should
+ "re-invoke onClose on a retry when the previous onClose threw (alreadyClosed is set AFTER onClose)" in {
+ // Reading the production code: `alreadyClosed = true` runs AFTER
+ // `onClose()`. So if onClose throws, alreadyClosed stays false and
+ // a second hasNext will re-invoke onClose. This is the OPPOSITE of
+ // an "alreadyClosed once close was attempted" contract — characterize
+ // the current (brittle) behavior so a refactor that swaps the order
+ // (running `alreadyClosed = true` BEFORE `onClose()`) surfaces here.
Review Comment:
should fix it separately after this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #5657:
URL: https://github.com/apache/texera/pull/5657
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #5657:
URL: https://github.com/apache/texera/pull/5657#discussion_r3406831364
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ while (it.hasNext) {
+ it.next()
+ // After every next(), there is at least one more element OR the
+ // next hasNext call will fire onClose. Until that point, fired
+ // must remain false.
+ // (When the final next() consumes 3, fired is still false because
+ // the loop hasn't run hasNext again yet.)
+ }
+ assert(fired, "after the while loop's final hasNext, onClose must have fired")
+ }
+
+ // ---------------------------------------------------------------------------
+ // onClose exception propagation
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator" should
+ "propagate exceptions thrown from onClose (no swallowing)" in {
+ val it = new AutoClosingIterator[Int](
+ Iterator.empty,
+ () => throw new IllegalStateException("close failed")
+ )
+ val ex = intercept[IllegalStateException] {
+ it.hasNext
+ }
+ assert(ex.getMessage == "close failed")
+ }
+
+ it should
+ "still mark itself as closed even when onClose throws (close runs before alreadyClosed is set)" in {
Review Comment:
The test description says the iterator "still mark[s] itself as closed" when `onClose` throws, but the assertions (and production code order) actually pin the opposite: `alreadyClosed` is not set if `onClose()` throws, so a second `hasNext` re-invokes `onClose`. Renaming the test text will make the intent accurate and easier to maintain.
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ while (it.hasNext) {
+ it.next()
+ // After every next(), there is at least one more element OR the
+ // next hasNext call will fire onClose. Until that point, fired
+ // must remain false.
+ // (When the final next() consumes 3, fired is still false because
+ // the loop hasn't run hasNext again yet.)
+ }
+ assert(fired, "after the while loop's final hasNext, onClose must have fired")
+ }
Review Comment:
This test currently only asserts that `onClose` fired by the end of iteration, but it does not verify the stated contract that `onClose` stays un-fired *between* elements. As written, it would still pass if `onClose` were incorrectly invoked early (e.g., during a `hasNext` that returns true). Consider rewriting to assert `fired == false` after each `hasNext` that returns true, and only becomes true when `hasNext` first returns false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan commented on code in PR #5657:
URL: https://github.com/apache/texera/pull/5657#discussion_r3406858234
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ while (it.hasNext) {
+ it.next()
+ // After every next(), there is at least one more element OR the
+ // next hasNext call will fire onClose. Until that point, fired
+ // must remain false.
+ // (When the final next() consumes 3, fired is still false because
+ // the loop hasn't run hasNext again yet.)
+ }
+ assert(fired, "after the while loop's final hasNext, onClose must have fired")
+ }
Review Comment:
Fixed in 8c283bfef7 — rewrote the case to step through `hasNext`/`next` explicitly. After each truthy `hasNext`, the test now asserts `fired == false`; only the final `hasNext` (which returns false) may flip it. A bug that prematurely closed during a truthy `hasNext` would now surface here, not just at the loop exit.
##########
common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/AutoClosingIteratorSpec.scala:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.source.scan
+
+import org.scalatest.flatspec.AnyFlatSpec
+
+class AutoClosingIteratorSpec extends AnyFlatSpec {
+
+ // ---------------------------------------------------------------------------
+ // hasNext + onClose firing semantics
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.hasNext (non-empty underlying)" should
+ "return true and NOT invoke onClose" in {
+ var closed = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closed = true)
+ assert(it.hasNext)
+ assert(!closed, "onClose must not fire while elements remain")
+ }
+
+ "AutoClosingIterator.hasNext (exhausted underlying)" should
+ "return false and invoke onClose exactly once" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(closeCount == 1)
+ }
+
+ it should "NOT invoke onClose again on a second hasNext after exhaustion" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => closeCount += 1)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(!it.hasNext)
+ assert(closeCount == 1, s"onClose must fire exactly once, got $closeCount calls")
+ }
+
+ // ---------------------------------------------------------------------------
+ // next() — delegates straight through
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator.next" should "delegate to the wrapped iterator (in order)" in {
+ val it = new AutoClosingIterator[Int](Iterator(10, 20, 30), () => ())
+ assert(it.next() == 10)
+ assert(it.next() == 20)
+ assert(it.next() == 30)
+ }
+
+ // ---------------------------------------------------------------------------
+ // Full traversal — onClose fires exactly once at the end
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator full traversal" should
+ "yield every element of the wrapped iterator in order" in {
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3, 4, 5), () => ())
+ assert(it.toList == List(1, 2, 3, 4, 5))
+ }
+
+ it should "fire onClose exactly once when toList finishes consuming" in {
+ var closeCount = 0
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => closeCount += 1)
+ val _ = it.toList
+ assert(closeCount == 1, s"expected single onClose firing, got $closeCount")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Already-empty source — close fires on the very first hasNext call
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator over an already-empty source" should
+ "fire onClose on the very first hasNext call" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator.empty, () => fired = true)
+ val _ = it.hasNext
+ assert(fired, "onClose must fire when the source is already empty")
+ }
+
+ // ---------------------------------------------------------------------------
+ // Mid-iteration close behavior — onClose does NOT fire before exhaustion
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator (mid-iteration)" should
+ "leave onClose un-fired between elements (only fires after hasNext returns false)" in {
+ var fired = false
+ val it = new AutoClosingIterator[Int](Iterator(1, 2, 3), () => fired = true)
+ while (it.hasNext) {
+ it.next()
+ // After every next(), there is at least one more element OR the
+ // next hasNext call will fire onClose. Until that point, fired
+ // must remain false.
+ // (When the final next() consumes 3, fired is still false because
+ // the loop hasn't run hasNext again yet.)
+ }
+ assert(fired, "after the while loop's final hasNext, onClose must have fired")
+ }
+
+ // ---------------------------------------------------------------------------
+ // onClose exception propagation
+ // ---------------------------------------------------------------------------
+
+ "AutoClosingIterator" should
+ "propagate exceptions thrown from onClose (no swallowing)" in {
+ val it = new AutoClosingIterator[Int](
+ Iterator.empty,
+ () => throw new IllegalStateException("close failed")
+ )
+ val ex = intercept[IllegalStateException] {
+ it.hasNext
+ }
+ assert(ex.getMessage == "close failed")
+ }
+
+ it should
+ "still mark itself as closed even when onClose throws (close runs before alreadyClosed is set)" in {
Review Comment:
Fixed in 8c283bfef7 — renamed the case to "re-invoke onClose on a retry when the previous onClose threw (alreadyClosed is set AFTER onClose)". The new wording matches what the assertions actually pin (closeCount == 2 on retry).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for AutoClosingIterator and UnionOpExec [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5657:
URL: https://github.com/apache/texera/pull/5657#issuecomment-4696640430
<!-- texera-benchmarks-comment -->
## ✅ No material benchmark regressions detected
🟢 13 better · 🔴 0 worse · ⚪ 2 noise (<±5%) · 0 without baseline
> CI benchmark results are noisy; treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/27449855931)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🟢 | bs=10 sw=10 sl=64 | 531 | 0.324 | 18,120/30,650/30,650 us | 🟢 +26.2% / 🟢 +33.4% |
| 🟢 | bs=100 sw=10 sl=64 | 1,155 | 0.705 | 83,130/113,026/113,026 us | 🟢 +37.5% / 🟢 +32.6% |
| 🟢 | bs=1000 sw=10 sl=64 | 1,434 | 0.875 | 696,402/740,892/740,892 us | 🟢 +51.3% / 🟢 +41.9% |
<details><summary>Baseline details</summary>
Latest main `73c76f5` from 2026-06-12T22:35:57.354Z
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 531 tuples/sec | 420.77 tuples/sec | 398.12 tuples/sec | +26.2% | +33.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.324 MB/s | 0.257 MB/s | 0.243 MB/s | +26.2% | +33.3% |
| bs=10 sw=10 sl=64 | p50 | 18,120 us | 23,004 us | 24,372 us | -21.2% | -25.7% |
| bs=10 sw=10 sl=64 | p95 | 30,650 us | 31,220 us | 36,082 us | -1.8% | -15.1% |
| bs=10 sw=10 sl=64 | p99 | 30,650 us | 31,220 us | 36,082 us | -1.8% | -15.1% |
| bs=100 sw=10 sl=64 | throughput | 1,155 tuples/sec | 839.89 tuples/sec | 870.8 tuples/sec | +37.5% | +32.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.705 MB/s | 0.513 MB/s | 0.531 MB/s | +37.5% | +32.6% |
| bs=100 sw=10 sl=64 | p50 | 83,130 us | 119,184 us | 114,334 us | -30.3% | -27.3% |
| bs=100 sw=10 sl=64 | p95 | 113,026 us | 160,656 us | 140,980 us | -29.6% | -19.8% |
| bs=100 sw=10 sl=64 | p99 | 113,026 us | 160,656 us | 140,980 us | -29.6% | -19.8% |
| bs=1000 sw=10 sl=64 | throughput | 1,434 tuples/sec | 948.1 tuples/sec | 1,010 tuples/sec | +51.3% | +41.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.875 MB/s | 0.579 MB/s | 0.617 MB/s | +51.2% | +41.9% |
| bs=1000 sw=10 sl=64 | p50 | 696,402 us | 1,048,503 us | 994,062 us | -33.6% | -29.9% |
| bs=1000 sw=10 sl=64 | p95 | 740,892 us | 1,107,534 us | 1,046,212 us | -33.1% | -29.2% |
| bs=1000 sw=10 sl=64 | p99 | 740,892 us | 1,107,534 us | 1,046,212 us | -33.1% | -29.2% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,376.69,200,128000,531,0.324,18119.54,30650.42,30650.42
1,100,10,64,20,1731.98,2000,1280000,1155,0.705,83129.94,113025.98,113025.98
2,1000,10,64,20,13946.57,20000,12800000,1434,0.875,696401.73,740891.83,740891.83
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org