You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "jinhongyii (via GitHub)" <gi...@apache.org> on 2023/09/05 21:19:15 UTC
[GitHub] [tvm] jinhongyii opened a new pull request, #15673: [Unity][Disco] Refactor shard loader
jinhongyii opened a new pull request, #15673:
URL: https://github.com/apache/tvm/pull/15673
1. support default f_shard
2. add API ShardLoaderLoadAll
3. broadcast when no sharding dim is specified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on a diff in pull request #15673: [Unity][Disco] Refactor shard loader
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #15673:
URL: https://github.com/apache/tvm/pull/15673#discussion_r1316423495
##########
src/runtime/disco/loader.cc:
##########
@@ -43,7 +43,7 @@ class ShardLoaderObj : public Object {
/*! \brief Create a shard loader. */
static ObjectRef Create(const std::string& path_to_metadata, const std::string& metadata,
const std::string& shard_info,
- TypedPackedFunc<void(DLTensor*, int, DLTensor*)> f_shard);
+ PackedFunc f_shard);
Review Comment:
Why this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on a diff in pull request #15673: [Unity][Disco] Refactor shard loader
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #15673:
URL: https://github.com/apache/tvm/pull/15673#discussion_r1316421990
##########
src/runtime/disco/loader.cc:
##########
@@ -93,9 +93,18 @@ inline std::vector<ShapeTuple::index_type> ShardShape(const ShapeTuple& shape, i
return result;
}
+class ShardInfoCompare {
+ public:
+ bool operator()(const ShardLoaderObj::ShardInfo& a, const ShardLoaderObj::ShardInfo& b) const {
+ int a_id = std::stoi(a.param->name.substr(6));
+ int b_id = std::stoi(b.param->name.substr(6));
+ return a_id < b_id;
Review Comment:
you don't need an actual "sort", instead you may retrieve the parameter names in ndarray-cache-metadata and calculate their names correspondingly by "params_0", "params_1", etc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jinhongyii commented on a diff in pull request #15673: [Unity][Disco] Refactor shard loader
Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii commented on code in PR #15673:
URL: https://github.com/apache/tvm/pull/15673#discussion_r1316425796
##########
src/runtime/disco/loader.cc:
##########
@@ -43,7 +43,7 @@ class ShardLoaderObj : public Object {
/*! \brief Create a shard loader. */
static ObjectRef Create(const std::string& path_to_metadata, const std::string& metadata,
const std::string& shard_info,
- TypedPackedFunc<void(DLTensor*, int, DLTensor*)> f_shard);
+ PackedFunc f_shard);
Review Comment:
TypedPackedFunc cannot be null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on a diff in pull request #15673: [Unity][Disco] Refactor shard loader
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #15673:
URL: https://github.com/apache/tvm/pull/15673#discussion_r1317585783
##########
src/runtime/disco/loader.cc:
##########
@@ -187,5 +231,16 @@ TVM_REGISTER_GLOBAL("runtime.disco.ShardLoaderLoad")
return loader->Load(IntegerFromShapeTuple(weight_index));
});
+TVM_REGISTER_GLOBAL("runtime.disco.ShardLoaderLoadAll").set_body_typed([](ObjectRef loader_obj) {
+ const auto* loader = loader_obj.as<ShardLoaderObj>();
+ CHECK(loader != nullptr) << "TypeError: Expected ShardLoaderObj, but gets: "
+ << loader_obj->GetTypeKey();
+ Array<NDArray> shards;
+ for (int i = 0; i < static_cast<int>(loader->shard_info_.size()); i++) {
Review Comment:
does `shard_info_.size()` imply all parameters?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao merged pull request #15673: [Disco] Add LoadAll method to Disco Shard Loader
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao merged PR #15673:
URL: https://github.com/apache/tvm/pull/15673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on pull request #15673: [Disco] Add LoadAll method to Disco Shard Loader
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #15673:
URL: https://github.com/apache/tvm/pull/15673#issuecomment-1711142582
To facilitate PR merge, I made the related changes accordingly which will save @jinhongyii's time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jinhongyii commented on pull request #15673: [Unity][Disco] Refactor shard loader
Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii commented on PR #15673:
URL: https://github.com/apache/tvm/pull/15673#issuecomment-1707323862
cc: @junrushao @tqchen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org