You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by "guan404ming (via GitHub)" <gi...@apache.org> on 2026/01/21 16:17:28 UTC

[PR] [Relax][NN] Add batch_flatten operator [tvm]

guan404ming opened a new pull request, #18677:
URL: https://github.com/apache/tvm/pull/18677

   ## Why                                                                                                                   
                                                                                                                              
   The `batch_flatten` operator was needed for CoreML backend support but was not implemented in Relax, causing the CoreML  
   pattern and converter to be commented out.                                                                               
                                                                                                                            
   ## How                                                                                                                   
                                                                                                                            
   - Add C++ operator `relax.nn.batch_flatten` with struct info inference                                                   
   - Add Python wrapper in `relax.op.nn`                                                                                    
   - Add legalization via `topi.reshape`                                                                                    
   - Enable CoreML pattern and converter for `nn.batch_flatten`                                                             
   - Add unit tests for operator, struct info inference, and legalization


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Relax][NN] Add batch_flatten operator [tvm]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #18677:
URL: https://github.com/apache/tvm/pull/18677#issuecomment-3782627669

   cc @tlopex @mshr-h 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Relax][NN] Add batch_flatten operator [tvm]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #18677:
URL: https://github.com/apache/tvm/pull/18677#discussion_r2713356700


##########
python/tvm/relax/transform/legalize_ops/nn.py:
##########
@@ -775,3 +775,8 @@ def nll_loss_without_weight(predictions, targets, reduction, ignore_index):
         reduction=call.attrs.reduction,
         ignore_index=call.attrs.ignore_index,
     )
+
+
+@register_legalize("relax.nn.batch_flatten")
+def _nn_batch_flatten(bb: BlockBuilder, call: Call) -> Expr:
+    return bb.call_te(topi.reshape, call.args[0], call.struct_info.shape.values)

Review Comment:
   ![high](https://www.gstatic.com/codereviewagent/high-priority.svg)
   
   The legalization for `relax.nn.batch_flatten` assumes that `call.struct_info.shape` is always available. However, if the input tensor to `batch_flatten` has an unknown number of dimensions or unknown shape values, the struct info inference for `batch_flatten` will correctly produce a `TensorStructInfo` without a concrete shape. In this scenario, `call.struct_info.shape` will be `None`, and attempting to access `call.struct_info.shape.values` will result in an `AttributeError`, crashing the compiler.
   
   To make the legalization more robust, you should add a check to ensure `call.struct_info.shape` is defined before attempting to use it. If it's not defined, the operator should not be legalized, and the original call should be returned. This is a common pattern in this file for other operators.
   
   It would also be great to add a new test case in `tests/python/relax/test_transform_legalize_ops_nn.py` to cover this scenario, for example with an input tensor of `R.Tensor(ndim=4, dtype="float32")`, and assert that `batch_flatten` is not legalized in this case.
   
   ```suggestion
   def _nn_batch_flatten(bb: BlockBuilder, call: Call) -> Expr:
       if call.struct_info.shape is None:
           return call
       return bb.call_te(topi.reshape, call.args[0], call.struct_info.shape.values)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Relax][NN] Add batch_flatten operator [tvm]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #18677:
URL: https://github.com/apache/tvm/pull/18677#issuecomment-3815514786

   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Relax][NN] Add batch_flatten operator [tvm]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #18677:
URL: https://github.com/apache/tvm/pull/18677#issuecomment-3779311681

   ## Summary of Changes
   
   Hello @guan404ming, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
   
   This pull request implements the `batch_flatten` operator within the Relax framework, addressing a critical dependency for the CoreML backend. The new operator flattens all dimensions of a tensor except the batch dimension, which was previously a missing feature preventing full CoreML integration. The changes encompass the entire lifecycle of the operator, from its low-level C++ definition and Python bindings to its shape inference, legalization, and thorough unit testing, ensuring its correct and robust operation within the system.
   
   ### Highlights
   
   * **New `batch_flatten` Operator**: Introduced `relax.nn.batch_flatten` to flatten all dimensions except the batch dimension, crucial for neural network architectures.
   * **CoreML Backend Support Enabled**: The absence of `batch_flatten` previously caused CoreML patterns and converters to be commented out; this PR enables them.
   * **Comprehensive Implementation**: The operator includes a C++ definition, Python wrapper, struct info inference, and legalization via `topi.reshape`.
   * **Extensive Unit Testing**: New unit tests cover operator correctness, struct info inference (including symbolic shapes), and legalization, ensuring robust functionality.
   
   🧠 **New Feature in Public Preview:** You can now enable **Memory** to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.**
   
   
   
   
   
   
   
   <details>
   <summary><b>Using Gemini Code Assist</b></summary>
   <br>
   
   The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.
   
   <b>Invoking Gemini</b>
   
   You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page.
   
   Feature | Command | Description
   --- | --- | ---
   Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
   Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
   Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments.
   Help | `/gemini help` | Displays a list of available commands.
   
   <b>Customization</b>
   
   To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github).
   
   <b>Limitations & Feedback</b>
   
   Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG).
   
   <b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b>
   </details>
   
   
   
   
   [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Relax][NN] Add batch_flatten operator [tvm]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming merged PR #18677:
URL: https://github.com/apache/tvm/pull/18677


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org