You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2022/05/31 07:17:38 UTC
[GitHub] [incubator-mxnet] bgawrych opened a new pull request, #21047: oneDNN FullyConnected weight caching & refactor
bgawrych opened a new pull request, #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047
## Description ##
Caching mechanism for FC
BRGEMM always enabled for int8
Refactor Forward function using smaller functions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1173452655
@mxnet-bot run ci [windows-gpu, unix-gpu, centos-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych merged pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych merged PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] DominikaJedynak commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
DominikaJedynak commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1168981053
> @DominikaJedynak please check if I merged correctly your type fix
Yes, it seems to be ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1170861273
@mxnet-bot run ci [sanity]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bartekkuncer commented on a diff in pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bartekkuncer commented on code in PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#discussion_r905908528
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -63,11 +65,26 @@ class SgDNNLFCOp {
}
private:
+ enum { kDataMin = 0, kDataMax, kWeightMin, kWeightMax, kBiasMin, kBiasMax, kSumMin, kSumMax };
+
+ NDArray PrepareOutputWithSum(const NDArray& sum_input, const NDArray& output);
+ bool CheckInitializationConditions(const std::vector<NDArray>& inputs,
+ const std::vector<float>& minmaxvec,
Review Comment:
Maybe min_max_vec to make it more readable?
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -112,104 +128,54 @@ void SgDNNLFCOp::Forward(const OpContext& ctx,
const int out_max_index = out_quantized ? index++ : 0;
CHECK_EQ(out_data.size(), index); // index is equal to total number of outputs
- float data_min = 0.0f;
- float data_max = 0.0f;
- float weight_min = 0.0f;
- float weight_max = 0.0f;
- float bias_min = 0.0f;
- float bias_max = 0.0f;
+ std::vector<float> minmaxvec(8);
Review Comment:
Magic number.
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -112,104 +128,54 @@ void SgDNNLFCOp::Forward(const OpContext& ctx,
const int out_max_index = out_quantized ? index++ : 0;
CHECK_EQ(out_data.size(), index); // index is equal to total number of outputs
- float data_min = 0.0f;
- float data_max = 0.0f;
- float weight_min = 0.0f;
- float weight_max = 0.0f;
- float bias_min = 0.0f;
- float bias_max = 0.0f;
+ std::vector<float> minmaxvec(8);
+ minmaxvec[kDataMin] = 0.0f;
+ minmaxvec[kDataMax] = 0.0f;
+ minmaxvec[kWeightMin] = 0.0f;
+ minmaxvec[kWeightMax] = 0.0f;
+ minmaxvec[kBiasMin] = 0.0f;
+ minmaxvec[kBiasMax] = 0.0f;
- const float sum_min = idx.sum_min ? in_data[idx.sum_min].data().dptr<float>()[0] : 0.0;
- const float sum_max = idx.sum_max ? in_data[idx.sum_max].data().dptr<float>()[0] : 0.0;
+ minmaxvec[kSumMin] = idx.sum_min ? in_data[idx.sum_min].data().dptr<float>()[0] : 0.0;
Review Comment:
Maybe change to 0.0f to make it consistent?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1167271901
@mxnet-bot run ci [unix-cpu, clang]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on a diff in pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on code in PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#discussion_r906056223
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -112,104 +128,54 @@ void SgDNNLFCOp::Forward(const OpContext& ctx,
const int out_max_index = out_quantized ? index++ : 0;
CHECK_EQ(out_data.size(), index); // index is equal to total number of outputs
- float data_min = 0.0f;
- float data_max = 0.0f;
- float weight_min = 0.0f;
- float weight_max = 0.0f;
- float bias_min = 0.0f;
- float bias_max = 0.0f;
+ std::vector<float> minmaxvec(8);
Review Comment:
done
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -112,104 +128,54 @@ void SgDNNLFCOp::Forward(const OpContext& ctx,
const int out_max_index = out_quantized ? index++ : 0;
CHECK_EQ(out_data.size(), index); // index is equal to total number of outputs
- float data_min = 0.0f;
- float data_max = 0.0f;
- float weight_min = 0.0f;
- float weight_max = 0.0f;
- float bias_min = 0.0f;
- float bias_max = 0.0f;
+ std::vector<float> minmaxvec(8);
+ minmaxvec[kDataMin] = 0.0f;
+ minmaxvec[kDataMax] = 0.0f;
+ minmaxvec[kWeightMin] = 0.0f;
+ minmaxvec[kWeightMax] = 0.0f;
+ minmaxvec[kBiasMin] = 0.0f;
+ minmaxvec[kBiasMax] = 0.0f;
- const float sum_min = idx.sum_min ? in_data[idx.sum_min].data().dptr<float>()[0] : 0.0;
- const float sum_max = idx.sum_max ? in_data[idx.sum_max].data().dptr<float>()[0] : 0.0;
+ minmaxvec[kSumMin] = idx.sum_min ? in_data[idx.sum_min].data().dptr<float>()[0] : 0.0;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161401819
Jenkins CI successfully triggered : [centos-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1170861371
Jenkins CI successfully triggered : [sanity]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1160493503
Jenkins CI successfully triggered : [unix-cpu, centos-cpu, clang]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] DominikaJedynak commented on a diff in pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
DominikaJedynak commented on code in PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#discussion_r904906959
##########
src/operator/subgraph/dnnl/dnnl_fc_property.h:
##########
@@ -30,6 +30,7 @@
#include <string>
#include <vector>
+#include <random>
Review Comment:
Do we need to include random?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161400582
Jenkins CI successfully triggered : [unix-cpu, clang, centos-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1141755473
Hey @bgawrych , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [windows-gpu, edge, centos-cpu, website, unix-gpu, windows-cpu, centos-gpu, unix-cpu, sanity, clang, miscellaneous]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161401719
@mxnet-bot run ci [centos-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1160493373
@mxnet-bot run ci [centos-cpu, clang, unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1173452794
Jenkins CI successfully triggered : [unix-gpu, windows-gpu, centos-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1174969033
Jenkins CI successfully triggered : [windows-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1174968983
@mxnet-bot run ci [windows-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on a diff in pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on code in PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#discussion_r906056073
##########
src/operator/subgraph/dnnl/dnnl_fc_property.h:
##########
@@ -30,6 +30,7 @@
#include <string>
#include <vector>
+#include <random>
Review Comment:
now not, done
##########
src/operator/subgraph/dnnl/dnnl_fc.cc:
##########
@@ -63,11 +65,26 @@ class SgDNNLFCOp {
}
private:
+ enum { kDataMin = 0, kDataMax, kWeightMin, kWeightMax, kBiasMin, kBiasMax, kSumMin, kSumMax };
+
+ NDArray PrepareOutputWithSum(const NDArray& sum_input, const NDArray& output);
+ bool CheckInitializationConditions(const std::vector<NDArray>& inputs,
+ const std::vector<float>& minmaxvec,
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161467920
Jenkins CI successfully triggered : [unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161400448
@mxnet-bot run ci [centos-cpu, clang, unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1161467808
@mxnet-bot run ci [unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1165569077
@DominikaJedynak please check if I merged correctly your type fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1167272001
Jenkins CI successfully triggered : [clang, unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1170935486
Jenkins CI successfully triggered : [website, windows-cpu, centos-cpu, miscellaneous, clang, unix-gpu, centos-gpu, edge, windows-gpu, sanity, unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #21047: oneDNN FullyConnected weight caching & refactor
Posted by GitBox <gi...@apache.org>.
bgawrych commented on PR #21047:
URL: https://github.com/apache/incubator-mxnet/pull/21047#issuecomment-1170935280
@mxnet-bot run ci [all]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org