You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/01/18 16:47:21 UTC
[GitHub] [incubator-mxnet] ArmageddonKnight opened a new pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
ArmageddonKnight opened a new pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376
## Description ##
implement storage tagging, the first half of the memory profiler
## Checklist ##
### Essentials ###
Please feel free to remove inapplicable items for your PR.
- [x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes)
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] Code is well-documented:
- For user-facing API changes, API doc string has been updated.
- For new C++ functions in header files, their functionalities and arguments are documented.
- For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
- Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
- [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
### Changes ###
- [x] This PR is the first half of the GPU memory profiler. It implements storage tagging which adds profiler scope, name, and data structure information to each allocated storage handle.
FYI, @szha @eric-haibin-lin
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r371554452
##########
File path: include/mxnet/resource.h
##########
@@ -148,33 +153,38 @@ struct Resource {
* \brief Get space requested as mshadow Tensor in specified type.
* The caller can request arbitrary size.
*
- * \param shape the Shape of returning tensor.
- * \param stream the stream of retruning tensor.
+ * \param shape the Shape of returning tensor.
+ * \param stream the Stream of returning tensor.
+ * \parma name the Name of the operator requesting the resource.
* \return the mshadow tensor requested.
* \tparam xpu the device type of random number generator.
* \tparam ndim the number of dimension of the tensor requested.
*/
template<typename xpu, int ndim, typename DType>
inline mshadow::Tensor<xpu, ndim, DType> get_space_typed(
- mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream) const {
+ mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream,
+ const std::string &name =
+ MXNET_STORAGE_DEFAULT_NAME_FARG("temp_space")) const {
Review comment:
should the name include where the allocation happened (e.g. GPU/stream)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r372047388
##########
File path: include/mxnet/storage.h
##########
@@ -26,15 +26,52 @@
#define MXNET_STORAGE_H_
#include <memory>
+#include <string>
#include "./base.h"
namespace mxnet {
+namespace {
+/// \brief Given a path, extract the filename.
+inline std::string __extract_fname(const std::string& path) {
+ std::size_t last_dir_pos = path.find_last_of("/\\");
+ if (last_dir_pos == std::string::npos) {
+ return path;
+ }
+ return path.substr(last_dir_pos + 1);
+}
+} // namespace anonymous
+
+#if __GNUG__ // if compiled with GCC
+#define MXNET_STORAGE_DEFAULT_NAME_FARG(tag) \
+ std::string(tag) \
+ + "_" + __extract_fname(__FILE__) \
+ + "+" + std::to_string(__LINE__) \
+ + "_" + __extract_fname(__builtin_FILE()) \
+ + "+" + std::to_string(__builtin_LINE())
+#else // !__GNUG__
+#define MXNET_STORAGE_DEFAULT_NAME_FARG(tag) \
+ std::string(tag) \
+ + "_" + __extract_fname(__FILE__) \
+ + "+" + std::to_string(__LINE__)
+#endif // __GNUG__
+#define MXNET_STORAGE_DEFAULT_PROFILER_SCOPE_CSTR "<unk>:"
+
/*!
* \brief Storage manager across multiple devices.
*/
class Storage {
public:
+ enum class DataStruct {
Review comment:
Let me just remove this. Since `profiler_scope` contains information regarding the purpose of allocation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368258935
##########
File path: src/resource.cc
##########
@@ -420,27 +422,29 @@ void* Resource::get_host_space_internal(size_t size) const {
#if MXNET_USE_CUDNN == 1
void Resource::get_cudnn_dropout_desc(
- cudnnDropoutDescriptor_t* dropout_desc,
+ cudnnDropoutDescriptor_t *dropout_desc,
mshadow::Stream<gpu> *stream,
- const float dropout,
- uint64_t seed) const {
+ const float dropout, uint64_t seed,
+ const std::string &name) const {
CHECK_EQ(req.type, ResourceRequest::kCuDNNDropoutDesc);
auto state_space = static_cast<resource::SpaceAllocator*>(ptr_);
CHECK_EQ(state_space->ctx.dev_id, stream->dev_id)
- << "The device id of cudnn dropout state space doesn't match that from stream.";
+ << "The device id of cuDNN dropout state space doesn't match that from stream.";
if (!state_space->handle.size) {
// not initialized yet.
size_t dropout_state_size;
CUDNN_CALL(cudnnDropoutGetStatesSize(stream->dnn_handle_, &dropout_state_size));
// reserve GPU space
Storage::Get()->DirectFree(
- Storage::Get()->Alloc(dropout_state_size, state_space->ctx));
+ Storage::Get()->Alloc(dropout_state_size,
+ state_space->ctx,
+ "ephemeral:", "dropout_reserve",
Review comment:
Storage allocations that belong to the `ephemeral:` profiler scope will be ignored and not recorded in the memory profiler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight closed pull request #17376:
[MXNET-1404] Implement storage tagging, the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight closed pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368257799
##########
File path: src/resource.cc
##########
@@ -420,27 +422,29 @@ void* Resource::get_host_space_internal(size_t size) const {
#if MXNET_USE_CUDNN == 1
void Resource::get_cudnn_dropout_desc(
- cudnnDropoutDescriptor_t* dropout_desc,
+ cudnnDropoutDescriptor_t *dropout_desc,
mshadow::Stream<gpu> *stream,
- const float dropout,
- uint64_t seed) const {
+ const float dropout, uint64_t seed,
+ const std::string &name) const {
CHECK_EQ(req.type, ResourceRequest::kCuDNNDropoutDesc);
auto state_space = static_cast<resource::SpaceAllocator*>(ptr_);
CHECK_EQ(state_space->ctx.dev_id, stream->dev_id)
- << "The device id of cudnn dropout state space doesn't match that from stream.";
+ << "The device id of cuDNN dropout state space doesn't match that from stream.";
if (!state_space->handle.size) {
// not initialized yet.
size_t dropout_state_size;
CUDNN_CALL(cudnnDropoutGetStatesSize(stream->dnn_handle_, &dropout_state_size));
// reserve GPU space
Storage::Get()->DirectFree(
- Storage::Get()->Alloc(dropout_state_size, state_space->ctx));
+ Storage::Get()->Alloc(dropout_state_size,
+ state_space->ctx,
+ "ephemeral:", "dropout_reserve",
Review comment:
Why ephemeral and dropout_reserve?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r373242645
##########
File path: src/profiler/gpu_memory_profiler.h
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef MXNET_PROFILER_GPU_MEMORY_PROFILER_H_
+#define MXNET_PROFILER_GPU_MEMORY_PROFILER_H_
+
+#include <mxnet/storage.h>
+#include <fstream>
+#include <string>
+
+namespace mxnet {
+namespace profiler {
+
+class GpuMemoryProfiler {
+ public:
+ /// \brief Get the global instance of the memory profiler
+ /// to record an allocation entry.
+ static GpuMemoryProfiler* Get();
+ GpuMemoryProfiler();
+ /// \brief Set/Get the profiler scope.
+ static void SetCurrentScope(const std::string& scope);
+ static std::string GetCurrentScope();
+ /// \brief Record an allocation entry in the profiler.
+ void addEntry(const Storage::Handle& handle,
Review comment:
could you use CamelCase: AddEntry
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r372126439
##########
File path: src/ndarray/ndarray.cc
##########
@@ -193,7 +203,9 @@ NDArray::NDArray(const std::shared_ptr<mkldnn::memory> &mkldnn_mem)
auto mem_desc = mkldnn_mem->get_desc();
shape_ = mxnet::TShape(mem_desc.data.dims, mem_desc.data.dims + mem_desc.data.ndims);
dtype_ = get_mxnet_type(mem_desc.data.data_type);
- ptr_ = std::make_shared<Chunk>(shape_, Context::CPU(), true, dtype_);
+ ptr_ = std::make_shared<Chunk>(shape_, Context::CPU(), true, dtype_,
+ "X", "X",
+ Storage::DataStruct::kUnknown);
Review comment:
same here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r371607120
##########
File path: python/mxnet/gluon/block.py
##########
@@ -53,18 +54,24 @@ def __init__(self, block):
@staticmethod
def create(prefix, params, hint):
- """Creates prefix and params for new `Block`."""
+ """
+ Creates prefix, params, and profiler scope name for new `Block`.
+ The profiler scope is to support the GPU memory profiler.
+ """
current = getattr(_BlockScope._current, "value", None)
if current is None:
if prefix is None:
if not hasattr(_name.NameManager._current, "value"):
_name.NameManager._current.value = _name.NameManager()
prefix = _name.NameManager._current.value.get(None, hint) + '_'
+ # replace the trailing underscore with colon
+ profiler_scope_name = (prefix[:-1] if prefix.endswith('_') \
Review comment:
from the logic below, it looks like prefix can still be None, which doesn't have `.endswith`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r372102800
##########
File path: src/ndarray/ndarray.cc
##########
@@ -183,7 +191,9 @@ NDArray::NDArray(const mkldnn::memory::desc &md)
: storage_type_(kDefaultStorage), entry_(nullptr) {
shape_ = mxnet::TShape(md.data.dims, md.data.dims + md.data.ndims);
dtype_ = get_mxnet_type(md.data.data_type);
- ptr_ = std::make_shared<Chunk>(shape_, Context::CPU(), true, dtype_);
+ ptr_ = std::make_shared<Chunk>(shape_, Context::CPU(), true, dtype_,
+ "X", "X",
+ Storage::DataStruct::kUnknown);
Review comment:
This seems to be additional memory used by MKLDNN. Let's capture it accordingly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r373313529
##########
File path: include/mxnet/ndarray.h
##########
@@ -1057,6 +1119,41 @@ class NDArray {
bool IsDefault() const;
#endif
+ /// \brief Cast storage type and auxiliary index to string,
+ /// used for recording storage allocations.
+ inline std::string STypeToString() const {
+ if (storage_type == kUndefinedStorage) {
+ return "undef";
+ } else if (storage_type == kRowSparseStorage) {
+ return "row_sparse";
+ } else if (storage_type == kCSRStorage) {
+ return "csr";
+ } else {
+ return "unknown";
+ }
+ }
Review comment:
@eric-haibin-lin Sorry I did not notice that. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r371570985
##########
File path: include/mxnet/resource.h
##########
@@ -148,33 +153,38 @@ struct Resource {
* \brief Get space requested as mshadow Tensor in specified type.
* The caller can request arbitrary size.
*
- * \param shape the Shape of returning tensor.
- * \param stream the stream of retruning tensor.
+ * \param shape the Shape of returning tensor.
+ * \param stream the Stream of returning tensor.
+ * \parma name the Name of the operator requesting the resource.
* \return the mshadow tensor requested.
* \tparam xpu the device type of random number generator.
* \tparam ndim the number of dimension of the tensor requested.
*/
template<typename xpu, int ndim, typename DType>
inline mshadow::Tensor<xpu, ndim, DType> get_space_typed(
- mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream) const {
+ mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream,
+ const std::string &name =
+ MXNET_STORAGE_DEFAULT_NAME_FARG("temp_space")) const {
Review comment:
I agree that we should include the GPU, but for stream I think we need to discuss.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r371608742
##########
File path: python/mxnet/operator.py
##########
@@ -734,7 +734,13 @@ def do_register(prop_cls):
def creator(op_type, argc, keys, vals, ret):
"""internal function"""
assert py_str(op_type) == reg_name
- kwargs = dict([(py_str(keys[i]), py_str(vals[i])) for i in range(argc)])
+ kwargs = dict()
Review comment:
initialize dict with `kwargs = {}`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r373243452
##########
File path: include/mxnet/ndarray.h
##########
@@ -1057,6 +1119,41 @@ class NDArray {
bool IsDefault() const;
#endif
+ /// \brief Cast storage type and auxiliary index to string,
+ /// used for recording storage allocations.
+ inline std::string STypeToString() const {
+ if (storage_type == kUndefinedStorage) {
+ return "undef";
+ } else if (storage_type == kRowSparseStorage) {
+ return "row_sparse";
+ } else if (storage_type == kCSRStorage) {
+ return "csr";
+ } else {
+ return "unknown";
+ }
+ }
Review comment:
it exists in https://github.com/apache/incubator-mxnet/blob/master/src/common/utils.h#L386-L397 . can we reuse it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r373242986
##########
File path: include/mxnet/ndarray.h
##########
@@ -998,7 +1054,10 @@ class NDArray {
<< "CheckAndAlloc(dbytes) is only intended for kDefaultStorage";
dbytes = std::max(dbytes, static_cast<uint64_t>(shandle.size));
if (delay_alloc) {
- shandle = Storage::Get()->Alloc(dbytes, shandle.ctx);
+ shandle = Storage::Get()->Alloc(dbytes, shandle.ctx,
+ shandle.profiler_scope,
+ shandle.name + "_realloc",
Review comment:
Why append `_realloc`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368549404
##########
File path: include/mxnet/storage.h
##########
@@ -26,15 +26,52 @@
#define MXNET_STORAGE_H_
#include <memory>
+#include <string>
#include "./base.h"
namespace mxnet {
+namespace {
+/// \brief Given a path, extract the filename.
+inline std::string __extract_fname(const std::string& path) {
+ std::size_t last_dir_pos = path.find_last_of("/\\");
+ if (last_dir_pos == std::string::npos) {
+ return path;
+ }
+ return path.substr(last_dir_pos + 1);
+}
+} // namespace anonymous
+
+#if __GNUG__ // if compiled with GCC
+#define MXNET_STORAGE_DEFAULT_NAME_FARG(tag) \
+ std::string(tag) \
+ + "_" + __extract_fname(__FILE__) \
+ + "+" + std::to_string(__LINE__) \
+ + "_" + __extract_fname(__builtin_FILE()) \
+ + "+" + std::to_string(__builtin_LINE())
+#else // !__GNUG__
+#define MXNET_STORAGE_DEFAULT_NAME_FARG(tag) \
+ std::string(tag) \
+ + "_" + __extract_fname(__FILE__) \
+ + "+" + std::to_string(__LINE__)
+#endif // __GNUG__
+#define MXNET_STORAGE_DEFAULT_PROFILER_SCOPE_CSTR "<unk>:"
+
/*!
* \brief Storage manager across multiple devices.
*/
class Storage {
public:
+ enum class DataStruct {
Review comment:
this name seems too generic. It looks like this enum differentiates the type of data allocation. Shall we think of a more informative name in the line of that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on issue #17376:
[MXNET-1404] Implement storage tagging, the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on issue #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#issuecomment-589991949
Thanks to all for the valuable feedbacks. I opened [**another PR**](https://github.com/apache/incubator-mxnet/pull/17656) to address them. Closing this PR now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r372047804
##########
File path: python/mxnet/gluon/block.py
##########
@@ -53,18 +54,24 @@ def __init__(self, block):
@staticmethod
def create(prefix, params, hint):
- """Creates prefix and params for new `Block`."""
+ """
+ Creates prefix, params, and profiler scope name for new `Block`.
+ The profiler scope is to support the GPU memory profiler.
+ """
current = getattr(_BlockScope._current, "value", None)
if current is None:
if prefix is None:
if not hasattr(_name.NameManager._current, "value"):
_name.NameManager._current.value = _name.NameManager()
prefix = _name.NameManager._current.value.get(None, hint) + '_'
+ # replace the trailing underscore with colon
+ profiler_scope_name = (prefix[:-1] if prefix.endswith('_') \
Review comment:
May I ask why? Since `prefix` is assigned a value `_name.NameManager._current.value.get(None, hint) + '_'` when it is `None`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368256642
##########
File path: include/mxnet/c_api.h
##########
@@ -313,6 +313,13 @@ MXNET_DLL int MXSetProcessProfilerState(int state, int profile_process,
*/
MXNET_DLL int MXSetProfilerState(int state);
+/*!
+ * \brief Set the scope of profiler for current process
Review comment:
What possible values can go in the scope?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368256685
##########
File path: include/mxnet/c_api.h
##########
@@ -313,6 +313,13 @@ MXNET_DLL int MXSetProcessProfilerState(int state, int profile_process,
*/
MXNET_DLL int MXSetProfilerState(int state);
+/*!
+ * \brief Set the scope of profiler for current process
Review comment:
@ChaiBapchya As an example, for Gluon applications, the profiler scope is set to be the Gluon block name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r371092755
##########
File path: include/mxnet/ndarray.h
##########
@@ -1057,6 +1119,41 @@ class NDArray {
bool IsDefault() const;
#endif
+ /// \brief Cast storage type and auxiliary index to string,
+ /// used for recording storage allocations.
+ inline std::string STypeToString() const {
+ if (storage_type == kUndefinedStorage) {
+ return "undef";
+ } else if (storage_type == kRowSparseStorage) {
+ return "row_sparse";
+ } else if (storage_type == kCSRStorage) {
+ return "csr";
+ } else {
+ return "unknown";
+ }
+ }
Review comment:
@eric-haibin-lin do you have suggestions on where to put this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] szha commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r382702032
##########
File path: include/mxnet/resource.h
##########
@@ -148,33 +153,38 @@ struct Resource {
* \brief Get space requested as mshadow Tensor in specified type.
* The caller can request arbitrary size.
*
- * \param shape the Shape of returning tensor.
- * \param stream the stream of retruning tensor.
+ * \param shape the Shape of returning tensor.
+ * \param stream the Stream of returning tensor.
+ * \parma name the Name of the operator requesting the resource.
* \return the mshadow tensor requested.
* \tparam xpu the device type of random number generator.
* \tparam ndim the number of dimension of the tensor requested.
*/
template<typename xpu, int ndim, typename DType>
inline mshadow::Tensor<xpu, ndim, DType> get_space_typed(
- mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream) const {
+ mshadow::Shape<ndim> shape, mshadow::Stream<xpu> *stream,
+ const std::string &name =
+ MXNET_STORAGE_DEFAULT_NAME_FARG("temp_space")) const {
Review comment:
Let's just add GPU and leave out the stream for now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on a change in pull request
#17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368257888
##########
File path: src/profiler/gpu_memory_profiler.cc
##########
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <dmlc/parameter.h>
+#include "./gpu_memory_profiler.h"
+
+#ifdef _WIN32
+#include <windows.h>
+#include <tchar.h>
+#endif
+#include <ctime>
+#include <string>
+
+namespace mxnet {
+namespace profiler {
+
+/// \brief Return the path to the temporary directory.
+/// The implementation is taken from `boost::temp_directory_path`
+static std::string temp_dir_path() {
+#ifdef _WIN32
+#define TEMP_DIR_MAX_LEN 1024
+ TCHAR temp_dir_c_str[TEMP_DIR_MAX_LEN];
+ GetTempPath(TEMP_DIR_MAX_LEN,
+ temp_dir_c_str);
+ return std::string(temp_dir_c_str);
+#undef MAX_PATH
+#else // !_WIN32
+ const char* temp_dir_c_str = NULL;
+
+ (temp_dir_c_str = std::getenv("TMPDIR")) ||
+ (temp_dir_c_str = std::getenv("TMP")) ||
+ (temp_dir_c_str = std::getenv("TEMP")) ||
+ (temp_dir_c_str = std::getenv("TEMPDIR"));
+#ifdef __ANDROID__
+ const char* default_temp_dir_c_str = "/data/local/tmp";
+#else
+ const char* default_temp_dir_c_str = "/tmp";
+#endif
+ std::string temp_dir = std::string(temp_dir_c_str == NULL ?
+ default_temp_dir_c_str : temp_dir_c_str);
+ return temp_dir;
+#endif // _WIN32
+}
+
+/// \brief Convert a `Storage::DataStruct` to string.
+static std::string data_struct_to_str(
+ const Storage::DataStruct data_struct) {
+ switch (data_struct) {
+ case Storage::DataStruct::kDataEntry: return "data entry";
+ case Storage::DataStruct::kTempSpace: return "temp space";
+ case Storage::DataStruct::kParameter: return "param";
+ case Storage::DataStruct::kParameterGrad: return "param grad";
+ case Storage::DataStruct::kOptimizerState: return "optimizer state";
+ case Storage::DataStruct::kAuxState: return "aux state";
+ case Storage::DataStruct::kEphemeral: return "ephemeral";
+ default: return "unknown";
+ }
+}
+
+GpuMemoryProfiler*
Review comment:
why on a separate line?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ArmageddonKnight commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
ArmageddonKnight commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r368373989
##########
File path: src/profiler/gpu_memory_profiler.cc
##########
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <dmlc/parameter.h>
+#include "./gpu_memory_profiler.h"
+
+#ifdef _WIN32
+#include <windows.h>
+#include <tchar.h>
+#endif
+#include <ctime>
+#include <string>
+
+namespace mxnet {
+namespace profiler {
+
+/// \brief Return the path to the temporary directory.
+/// The implementation is taken from `boost::temp_directory_path`
+static std::string temp_dir_path() {
+#ifdef _WIN32
+#define TEMP_DIR_MAX_LEN 1024
+ TCHAR temp_dir_c_str[TEMP_DIR_MAX_LEN];
+ GetTempPath(TEMP_DIR_MAX_LEN,
+ temp_dir_c_str);
+ return std::string(temp_dir_c_str);
+#undef MAX_PATH
+#else // !_WIN32
+ const char* temp_dir_c_str = NULL;
+
+ (temp_dir_c_str = std::getenv("TMPDIR")) ||
+ (temp_dir_c_str = std::getenv("TMP")) ||
+ (temp_dir_c_str = std::getenv("TEMP")) ||
+ (temp_dir_c_str = std::getenv("TEMPDIR"));
+#ifdef __ANDROID__
+ const char* default_temp_dir_c_str = "/data/local/tmp";
+#else
+ const char* default_temp_dir_c_str = "/tmp";
+#endif
+ std::string temp_dir = std::string(temp_dir_c_str == NULL ?
+ default_temp_dir_c_str : temp_dir_c_str);
+ return temp_dir;
+#endif // _WIN32
+}
+
+/// \brief Convert a `Storage::DataStruct` to string.
+static std::string data_struct_to_str(
+ const Storage::DataStruct data_struct) {
+ switch (data_struct) {
+ case Storage::DataStruct::kDataEntry: return "data entry";
+ case Storage::DataStruct::kTempSpace: return "temp space";
+ case Storage::DataStruct::kParameter: return "param";
+ case Storage::DataStruct::kParameterGrad: return "param grad";
+ case Storage::DataStruct::kOptimizerState: return "optimizer state";
+ case Storage::DataStruct::kAuxState: return "aux state";
+ case Storage::DataStruct::kEphemeral: return "ephemeral";
+ default: return "unknown";
+ }
+}
+
+GpuMemoryProfiler*
Review comment:
Fixed. Thanks for the advice.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull
request #17376: [MXNET-1404] Implement storage tagging,
the first half of the memory profiler
Posted by GitBox <gi...@apache.org>.
eric-haibin-lin commented on a change in pull request #17376: [MXNET-1404] Implement storage tagging, the first half of the memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/17376#discussion_r373277999
##########
File path: src/profiler/gpu_memory_profiler.h
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef MXNET_PROFILER_GPU_MEMORY_PROFILER_H_
+#define MXNET_PROFILER_GPU_MEMORY_PROFILER_H_
+
+#include <mxnet/storage.h>
+#include <fstream>
+#include <string>
+
+namespace mxnet {
+namespace profiler {
+
+class GpuMemoryProfiler {
+ public:
+ /// \brief Get the global instance of the memory profiler
+ /// to record an allocation entry.
+ static GpuMemoryProfiler* Get();
+ GpuMemoryProfiler();
+ /// \brief Set/Get the profiler scope.
+ static void SetCurrentScope(const std::string& scope);
+ static std::string GetCurrentScope();
+ /// \brief Record an allocation entry in the profiler.
+ void addEntry(const Storage::Handle& handle,
+ const size_t actual_alloc_size, const bool reuse);
+ private:
+ bool enabled_;
+ std::ofstream alloc_csv_fout_;
Review comment:
could you add a mutex to make this class thread safe?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services