You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/06/18 22:55:09 UTC

[I] add unit test coverage for TextGenCodegen [texera]

aglinxinyuan opened a new issue, #5778:
URL: https://github.com/apache/texera/issues/5778

   ### Task Summary
   
   Add a `TextGenCodegenSpec.scala` that pins the Python snippets generated for the Hugging Face `text-generation` task — the OpenAI chat-completions payload shape and the response-parsing path — plus the safety property that no user-provided strings are inlined.
   
   ## Background
   
   `TextGenCodegen` (`operator/huggingFace/codegen/TextGenCodegen.scala`) is a `TaskCodegen` `object` that emits two Python snippets spliced into the shared Hugging Face inference template: `payloadPython(ctx)` (builds the request `payload`) and `parsePython(ctx)` (extracts the model output). It is the default/fallback codegen in `HuggingFaceInferenceOpDesc.registeredCodegens`, and has no dedicated unit-spec.
   
   ## Behavior to pin
   
   | Surface | Contract |
   | --- | --- |
   | `TextGenCodegen.task` | equals `"text-generation"` |
   | `payloadPython(ctx)` | opens with `if task == "text-generation":`; builds a `payload` dict with a `messages` list (a `system` + a `user` message), `max_tokens`, and `temperature`; includes the `else: payload = {"inputs": prompt_value}` fallback |
   | `parsePython(ctx)` | returns the branch reading `body["choices"][0]["message"]["content"]` |
   | no raw-string leakage | both snippets reference the per-instance attributes (`self.MODEL_ID`, `self.SYSTEM_PROMPT`, `self.MAX_NEW_TOKENS`, `self.TEMPERATURE`) and loop locals (`prompt_value`, `task`) — they must NOT contain the literal values from the passed `CodegenContext` |
   | context-independent | the text-generation snippets ignore `ctx`: two different `CodegenContext`s produce identical output |
   
   `CodegenContext`'s string fields are `EncodableString` (a `String` type-alias), so plain string literals work in a test. See `HuggingFaceInferenceOpDesc` (~line 164) for a `CodegenContext(...)` construction example.
   
   ## Scope
   
   - New spec: `TextGenCodegenSpec.scala` under `common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/`.
   - Assert on snippet content with substring/contains checks (the snippets are deterministic Python text); no Python execution or engine needed.
   - No production-code changes.
   
   ### Task Type
   - [ ] Refactor / Cleanup
   - [ ] DevOps / Deployment / CI
   - [x] Testing / QA
   - [ ] Documentation
   - [ ] Performance
   - [ ] Other
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] add unit test coverage for TextGenCodegen [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan closed issue #5778: add unit test coverage for TextGenCodegen
URL: https://github.com/apache/texera/issues/5778


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] add unit test coverage for TextGenCodegen [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5778:
URL: https://github.com/apache/texera/issues/5778#issuecomment-4746808193

   <!-- texera:template-compliance -->
   👋 Thanks for opening this issue, @aglinxinyuan!
   
   It looks like the issue description doesn't quite follow our template yet:
   
   - This issue doesn't appear to use one of our templates (Bug, Feature, or Task). Please open it using a template so the required details are captured.
   
   Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.
   
   You can find the template prompts by editing the description, or see [CONTRIBUTING.md](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) for the full contribution flow.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] add unit test coverage for TextGenCodegen [texera]

Posted by "EmilySun621 (via GitHub)" <gi...@apache.org>.
EmilySun621 commented on issue #5778:
URL: https://github.com/apache/texera/issues/5778#issuecomment-4748349711

   /take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org