You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "jeyhunkarimov (via GitHub)" <gi...@apache.org> on 2024/01/05 16:47:06 UTC

[PR] [FLINK-33996][table-planner]: Avoid merging projects if leads to redundant computation [flink]

jeyhunkarimov opened a new pull request, #24033:
URL: https://github.com/apache/flink/pull/24033

   
   ## What is the purpose of the change
   
   The current optimizer/planner merges two Projection/Calc nodes if they are deterministic. 
   In some scenarios, this check is not enough. Especially, when the top Projection/Calc 
   node uses the already computed value from the bottom Project/Calc node, merging
   the two of them might lead to redundant computation (recomputing the same 
   expressions several times). In fact, our tests contained many tests that 
   computed the same expression several times.
   
   
   ## Brief change log
   
     - Extend the `mergeable` method and add an additional check
     - Add tests
   
   
   ## Verifying this change
   
   This change added tests to `CalcMergeTestBase`
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): ( no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (no)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-33996][table-planner]: Avoid merging projects if leads to redundant computation [flink]

Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24033:
URL: https://github.com/apache/flink/pull/24033#issuecomment-1879308733

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-33996][table-planner]: Avoid merging projects if leads to redundant computation [flink]

Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24033:
URL: https://github.com/apache/flink/pull/24033#issuecomment-1879615483

   HI @LadyForest could you please review this PR in your available time? Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-33996][table-planner]: Avoid merging projects if leads to redundant computation [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #24033:
URL: https://github.com/apache/flink/pull/24033#issuecomment-1878973201

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d687103dabf6ac2a0af763e8a3b845ef9629c7ba",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d687103dabf6ac2a0af763e8a3b845ef9629c7ba",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d687103dabf6ac2a0af763e8a3b845ef9629c7ba UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-33996][table-planner]: Avoid merging projects if leads to redundant computation [flink]

Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24033:
URL: https://github.com/apache/flink/pull/24033#issuecomment-1879333784

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org