You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "LeshengJin (via GitHub)" <gi...@apache.org> on 2023/11/20 21:41:48 UTC

[PR] [WIP][SLIM] Disco Integration [tvm]

LeshengJin opened a new pull request, #16153:
URL: https://github.com/apache/tvm/pull/16153

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1414307231


##########
src/runtime/relax_vm/ndarray_cache_support.cc:
##########
@@ -268,6 +267,63 @@ class NDArrayCache {
     }
   }
 
+  /*!
+   * \brief Load parameters from i-th file record and append them.
+   *
+   * \param metadata_json_str The metadata json string.
+   * \param cache_path The cache to path.
+   * \param i The index of file record.
+   * \param device_type The type of device to be loaded.
+   * \param device_id The device id.
+   */
+  static Array<String> LoadFromRecord(const std::string& metadata_json_str,

Review Comment:
   extract into helper method



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1416116820


##########
include/tvm/runtime/disco/utils.h:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef TVM_RUNTIME_DISCO_UTILS_H_
+#define TVM_RUNTIME_DISCO_UTILS_H_
+
+namespace tvm {
+namespace runtime {
+
+/*!

Review Comment:
   move this to builtin.h



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1413212090


##########
src/runtime/disco/utils.cc:
##########
@@ -16,11 +16,9 @@
  * specific language governing permissions and limitations
  * under the License.
  */
-#ifndef TVM_RUNTIME_DISCO_UTILS_H_
-#define TVM_RUNTIME_DISCO_UTILS_H_
-

Review Comment:
   why change .h to .cc?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [WIP][SLIM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #16153:
URL: https://github.com/apache/tvm/pull/16153#issuecomment-1819873036

   Instead of moving `worker.h` to `include/tvm/runtime/disco/worker.h`, let's only expose `DiscoWorker` and continue keep `WorkerZeroData` and `DiscoWorkerThread` internal


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "Hzfengsy (via GitHub)" <gi...@apache.org>.
Hzfengsy commented on PR #16153:
URL: https://github.com/apache/tvm/pull/16153#issuecomment-1848865089

   A minor comment: it would be great to use `NNModule` or `Frontend` as the tag in the title. `SLM` is somehow a name used in MLC-LLM, rather than TVM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [WIP][SLIM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1399798309


##########
python/tvm/runtime/disco/process_pool.py:
##########
@@ -47,9 +47,10 @@ class DiscoPopenWorker:
         The standard error streams handler specified for the popen process.
     """
 
-    def __init__(self, worker_id: int, num_workers: int, stdout=None, stderr=None):
+    def __init__(self, worker_id: int, num_workers: int, entry_point="tvm.exec.disco_worker", stdout=None, stderr=None):

Review Comment:
   ```suggestion
       def __init__(self, worker_id: int, num_workers: int, entrypoint="tvm.exec.disco_worker", stdout=None, stderr=None):
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [WIP][SLIM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1399796523


##########
include/tvm/runtime/disco/builtin.h:
##########
@@ -51,7 +49,7 @@ NDArray DiscoEmptyNDArray(ShapeTuple shape, DataType dtype, Device device);
  * \param reduce_kind The kind of reduction operation (e.g. sum, avg, min, max)
  * \return The outcome of allreduce
  */
-void AllReduce(NDArray send, ReduceKind reduce_kind, NDArray recv);

Review Comment:
   Move the definition of `ReduceKind` to this file



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Disco] Expose `DiscoWorker` and `ndarray_cache_support` in header [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao merged PR #16153:
URL: https://github.com/apache/tvm/pull/16153


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #16153:
URL: https://github.com/apache/tvm/pull/16153#issuecomment-1841867267

   I just pushed a few refactoring 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [WIP][SLIM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1399805951


##########
src/runtime/disco/loader.cc:
##########
@@ -1,380 +1,380 @@
-/*

Review Comment:
   let's keep it as it is



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1413212090


##########
src/runtime/disco/utils.cc:
##########
@@ -16,11 +16,9 @@
  * specific language governing permissions and limitations
  * under the License.
  */
-#ifndef TVM_RUNTIME_DISCO_UTILS_H_
-#define TVM_RUNTIME_DISCO_UTILS_H_
-

Review Comment:
   why change h to cc?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SLM] Disco Integration [tvm]

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #16153:
URL: https://github.com/apache/tvm/pull/16153#discussion_r1416121509


##########
src/runtime/disco/nccl/nccl.cc:
##########
@@ -28,7 +28,7 @@
 #include <vector>
 
 #include "../../../support/process_id.h"
-#include "../utils.h"
+#include "../utils.cc"

Review Comment:
   do not include a cc file



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org