You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2026/05/02 09:23:24 UTC

[PR] chore(test): instrument DataProcessingSpec with per-test timing log [texera]

Yicong-Huang opened a new pull request, #4677:
URL: https://github.com/apache/texera/pull/4677

   ### What changes were proposed in this PR?
   
   **Draft / diagnostic** — wrap \`DataProcessingSpec.withFixture\` with a nano-timer and dump a sorted per-test elapsed-time table in \`afterAll\`.
   
   \`DataProcessingSpec\` is the dominant contributor to scala test wall time on a typical main-push run (its 16 e2e workflow tests run for the full ~6 minute test phase, with each test bounded by a 1-minute \`Await\`). Without per-test numbers there is no way to decide which workflows to consolidate or delete.
   
   This change is intended to land long enough to gather one CI run's worth of timings; once the slow tests are identified (#4670 follow-up), this instrumentation is expected to be removed or rewritten as a stricter timeout per test.
   
   \`\`\`
   [info] DataProcessingSpec per-test timings (sorted desc, total NNNNms across 16 tests):
   [info]    XXXXXms  Engine should execute csv->keyword->averageAndGroupBy workflow normally
   [info]    ...
   \`\`\`
   
   ### Any related issues, documentation, discussions?
   
   Diagnostic for the scala build wall-time discussion. Findings will feed a follow-up that either consolidates redundant e2e workflows or fixes the underlying retry flake the suite documents.
   
   ### How was this PR tested?
   
   No new tests. The instrumentation runs as part of the existing \`DataProcessingSpec\` execution under the python/scala/agent-service \`Build\` matrix. After this PR's CI run, the scala job log under \`build / scala (ubuntu-22.04, 11)\` will contain the sorted timing table.
   
   ### Was this PR authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Opus 4.7, 1M context)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore(test): instrument DataProcessingSpec with per-test timing log [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang closed pull request #4677: chore(test): instrument DataProcessingSpec with per-test timing log
URL: https://github.com/apache/texera/pull/4677


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore(test): instrument DataProcessingSpec with per-test timing log [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on PR #4677:
URL: https://github.com/apache/texera/pull/4677#issuecomment-4363493634

   Closing — will gather timings locally instead of through CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore(test): instrument DataProcessingSpec with per-test timing log [texera]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #4677:
URL: https://github.com/apache/texera/pull/4677#issuecomment-4363498080

   ## [Codecov](https://app.codecov.io/gh/apache/texera/pull/4677?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 85.05%. Comparing base ([`8841990`](https://app.codecov.io/gh/apache/texera/commit/8841990f1d74dc5195d6e760352a3886a7d3a1f2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6ec0a63`](https://app.codecov.io/gh/apache/texera/commit/6ec0a6379441c929ea3458e3e403c21f287b908e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff             @@
   ##             main    #4677       +/-   ##
   ===========================================
   + Coverage   52.46%   85.05%   +32.59%     
   ===========================================
     Files         353      157      -196     
     Lines       21338     7803    -13535     
     Branches     1791        0     -1791     
   ===========================================
   - Hits        11195     6637     -4558     
   + Misses      10026     1166     -8860     
   + Partials      117        0      -117     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/texera/pull/4677/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [agent-service](https://app.codecov.io/gh/apache/texera/pull/4677/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [frontend](https://app.codecov.io/gh/apache/texera/pull/4677/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [python](https://app.codecov.io/gh/apache/texera/pull/4677/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.05% <ø> (ø)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/texera/pull/4677?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org