[GitHub] eric-haibin-lin commented on a change in pull request #9055: Re-enable all op segments when in batch mode

[GitBox <gi...@apache.org>]

eric-haibin-lin commented on a change in pull request #9055: Re-enable all op segments when in batch mode
URL: https://github.com/apache/incubator-mxnet/pull/9055#discussion_r156762280
 
 

 ##########
 File path: src/executor/graph_executor.cc
 ##########
 @@ -1356,8 +1356,9 @@ void GraphExecutor::InitOpSegs() {
   // The maximum number of node in a segment executed in bulk
   size_t num_nodes_threshold = dmlc::GetEnv("MXNET_EXEC_BULK_EXEC_MAX_NODE_TRAIN", 15);
   if (prefer_bulk_exec_inference && num_forward_nodes_ == total_num_nodes) {
-    // bulk the whole graph for inference
-    num_nodes_threshold = std::numeric_limits<size_t>::max();
+    // Bulk the whole graph for inference
+    cached_seg_opr_[0] = this->CreateCachedSegOpr(0, num_forward_nodes_);
+    return;
   }
 
   if (prefer_bulk_exec) {
 
 Review comment:
   We create just one segment because `kLocal` and `kCrossDeviceCopy` ops should not be included in bulk for inference. We still need to visit all nodes in the graph, but for inference we don't need to create a new segment if `node->is_variable()`.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services