You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/04/06 20:40:50 UTC
[GitHub] [incubator-mxnet] barry-jin opened a new pull request #20131: [WIP][2.0] Add cpp-package
barry-jin opened a new pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131
## Description ##
Migrate cpp-package to MXNet2.0. Still work in progress.
## Checklist ##
### Essentials ###
- [x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] All changes have test coverage
- [x] Code is well-documented
### Changes ###
- [ ] Feature1, tests, (and when applicable, API doc)
- [ ] Feature2, tests, (and when applicable, API doc)
## Comments ##
- If this change is a backward incompatible change, why must this change be made.
- Interesting edge cases to note here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r628983692
##########
File path: cpp-package/README.md
##########
@@ -0,0 +1,62 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+# MXNet C++ Package
+
+The MXNet C++ Package provides C++ API bindings to the users of MXNet. Currently, these bindings are not available as standalone package.
+The users of these bindings are required to build this package as mentioned below.
+
+## Building C++ Package
+
+The cpp-package directory contains the implementation of C++ API. As mentioned above, users are required to build this directory or package before using it.
+**The cpp-package is built while building the MXNet shared library, *libmxnet.so*.**
+
+### Steps to build the C++ package:
+1. Building the MXNet C++ package requires building MXNet from source.
+2. Clone the MXNet GitHub repository **recursively** to ensure the code in submodules is available for building MXNet.
+ ```
+ git clone --recursive https://github.com/apache/incubator-mxnet mxnet
+ ```
+
+3. Install the [prerequisites](<https://mxnet.apache.org/install/build_from_source#prerequisites>), desired [BLAS libraries](<https://mxnet.apache.org/install/build_from_source#blas-library>) and optional [OpenCV, CUDA, and cuDNN](<https://mxnet.apache.org/install/build_from_source#optional>) for building MXNet from source.
+4. There is a configuration file for make, [make/config.mk](<https://github.com/apache/incubator-mxnet/blob/master/make/config.mk>) that contains all the compilation options. You can edit this file and set the appropriate options prior to running the **make** command.
Review comment:
the build instructions are no longer accurate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816996435
Jenkins CI successfully triggered : [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816196140
@mxnet-bot run ci [sanity, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-835376740
Jenkins CI successfully triggered : [centos-gpu, windows-cpu, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816807178
Jenkins CI successfully triggered : [sanity, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-835376688
@mxnet-bot run ci [centos-gpu, windows-cpu, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin closed pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin closed pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816807126
@mxnet-bot run ci [sanity, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-839130885
Jenkins CI successfully triggered : [centos-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-839130790
@mxnet-bot run ci [centos-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin closed pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin closed pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-830478237
@mxnet-bot run ci [centos-gpu, miscellaneous, unix-cpu, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on a change in pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r612016609
##########
File path: cpp-package/scripts/OpWrapperGenerator.py
##########
@@ -0,0 +1,499 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# -*- coding: utf-8 -*-
+# This is a python script that generates operator wrappers such as FullyConnected,
+# based on current libmxnet.dll. This script is written so that we don't need to
+# write new operator wrappers when new ones are added to the library.
+
+from ctypes import *
+from ctypes.util import find_library
+import os
+import logging
+import platform
+import re
+import sys
+import tempfile
+import filecmp
+import shutil
+import codecs
+
+def gen_enum_value(value):
+ return 'k' + value[0].upper() + value[1:]
+
+class EnumType:
+ name = ''
+ enumValues = []
+ def __init__(self, typeName = 'ElementWiseOpType', \
+ typeString = "{'avg', 'max', 'sum'}"):
+ self.name = typeName
+ if (typeString[0] == '{'): # is a enum type
+ isEnum = True
+ # parse enum
+ self.enumValues = typeString[typeString.find('{') + 1:typeString.find('}')].split(',')
+ for i in range(0, len(self.enumValues)):
+ self.enumValues[i] = self.enumValues[i].strip().strip("'")
+ else:
+ logging.warn("trying to parse none-enum type as enum: %s" % typeString)
+ def GetDefinitionString(self, indent = 0):
+ indentStr = ' ' * indent
+ ret = indentStr + 'enum class %s {\n' % self.name
+ for i in range(0, len(self.enumValues)):
+ ret = ret + indentStr + ' %s = %d' % (gen_enum_value(self.enumValues[i]), i)
+ if (i != len(self.enumValues) -1):
+ ret = ret + ","
+ ret = ret + "\n"
+ ret = ret + "};\n"
+ return ret
+ def GetDefaultValueString(self, value = ''):
+ return self.name + "::" + gen_enum_value(value)
+ def GetEnumStringArray(self, indent = 0):
+ indentStr = ' ' * indent
+ ret = indentStr + 'static const char *%sValues[] = {\n' % self.name
+ for i in range(0, len(self.enumValues)):
+ ret = ret + indentStr + ' "%s"' % self.enumValues[i]
+ if (i != len(self.enumValues) -1):
+ ret = ret + ","
+ ret = ret + "\n"
+ ret = ret + indentStr + "};\n"
+ return ret
+ def GetConvertEnumVariableToString(self, variable=''):
+ return "%sValues[int(%s)]" % (self.name, variable)
+
+
+class Arg:
+ typeDict = {'boolean':'bool',\
+ 'boolean or None':'dmlc::optional<bool>',\
+ 'Shape(tuple)':'Shape',\
+ 'Symbol':'Symbol',\
+ 'NDArray':'Symbol',\
+ 'NDArray-or-Symbol':'Symbol',\
+ 'Symbol[]':'const std::vector<Symbol>&',\
+ 'Symbol or Symbol[]':'const std::vector<Symbol>&',\
+ 'NDArray[]':'const std::vector<Symbol>&',\
+ 'caffe-layer-parameter':'::caffe::LayerParameter',\
+ 'NDArray-or-Symbol[]':'const std::vector<Symbol>&',\
+ 'float':'mx_float',\
+ 'real_t':'mx_float',\
+ 'int':'int',\
+ 'int (non-negative)': 'uint32_t',\
+ 'long (non-negative)': 'uint64_t',\
+ 'int or None':'dmlc::optional<int>',\
+ 'float or None':'dmlc::optional<float>',\
+ 'long':'int64_t',\
+ 'double':'double',\
+ 'double or None':'dmlc::optional<double>',\
+ 'Shape or None':'dmlc::optional<Shape>',\
+ 'string':'const std::string&',\
+ 'tuple of <float>':'nnvm::Tuple<mx_float>',\
+ 'tuple of <>':'mxnet::cpp::Shape',\
+ '':'index_t'}
+ name = ''
+ type = ''
+ description = ''
+ isEnum = False
+ enum = None
+ hasDefault = False
+ defaultString = ''
+ def __init__(self, opName = '', argName = '', typeString = '', descString = ''):
+ self.name = argName
+ self.description = descString
+ if (typeString[0] == '{'): # is enum type
+ self.isEnum = True
+ self.enum = EnumType(self.ConstructEnumTypeName(opName, argName), typeString)
+ self.type = self.enum.name
+ else:
+ try:
+ self.type = self.typeDict[typeString.split(',')[0]]
+ except:
+ print('argument "%s" of operator "%s" has unknown type "%s"' % (argName, opName, typeString))
+ pass
+ if typeString.find('default=') != -1:
+ self.hasDefault = True
+ self.defaultString = typeString.split('default=')[1].strip().strip("'")
+ if typeString.startswith('string'):
+ self.defaultString = self.MakeCString(self.defaultString)
+ elif self.isEnum:
+ self.defaultString = self.enum.GetDefaultValueString(self.defaultString)
+ elif self.defaultString == 'None':
+ self.defaultString = self.type + '()'
+ elif self.type == "bool":
+ if self.defaultString == "1" or self.defaultString == "True":
+ self.defaultString = "true"
+ else:
+ self.defaultString = "false"
+ elif self.defaultString[0] == '(':
+ self.defaultString = 'Shape' + self.defaultString
+ elif self.defaultString[0] == '[':
+ self.defaultString = 'Shape(' + self.defaultString[1:-1] + ")"
+ elif self.type == 'dmlc::optional<int>':
+ self.defaultString = self.type + '(' + self.defaultString + ')'
+ elif self.type == 'dmlc::optional<bool>':
+ self.defaultString = self.type + '(' + self.defaultString + ')'
+ elif typeString.startswith('caffe-layer-parameter'):
+ self.defaultString = 'textToCaffeLayerParameter(' + self.MakeCString(self.defaultString) + ')'
+ hasCaffe = True
+
+ def MakeCString(self, str):
+ str = str.replace('\n', "\\n")
+ str = str.replace('\t', "\\t")
+ return '\"' + str + '\"'
+
+ def ConstructEnumTypeName(self, opName = '', argName = ''):
+ a = opName[0].upper()
+ # format ArgName so instead of act_type it returns ActType
+ argNameWords = argName.split('_')
+ argName = ''
+ for an in argNameWords:
+ argName = argName + an[0].upper() + an[1:]
+ typeName = a + opName[1:] + argName
+ return typeName
+
+class Op:
+ name = ''
+ description = ''
+ args = []
+
+ def __init__(self, name = '', description = '', args = []):
+ self.name = name
+ self.description = description
+ # add a 'name' argument
+ nameArg = Arg(self.name, \
+ 'symbol_name', \
+ 'string', \
+ 'name of the resulting symbol')
+ args.insert(0, nameArg)
+ # reorder arguments, put those with default value to the end
+ orderedArgs = []
+ for arg in args:
+ if not arg.hasDefault:
+ orderedArgs.append(arg)
+ for arg in args:
+ if arg.hasDefault:
+ orderedArgs.append(arg)
+ self.args = orderedArgs
+
+ def WrapDescription(self, desc = ''):
+ ret = []
+ sentences = desc.split('.')
+ lines = desc.split('\n')
+ for line in lines:
+ line = line.strip()
+ if len(line) <= 80:
+ ret.append(line.strip())
+ else:
+ while len(line) > 80:
+ pos = line.rfind(' ', 0, 80)+1
+ if pos <= 0:
+ pos = line.find(' ')
+ if pos < 0:
+ pos = len(line)
+ ret.append(line[:pos].strip())
+ line = line[pos:]
+ return ret
+
+ def GenDescription(self, desc = '', \
+ firstLineHead = ' * \\brief ', \
+ otherLineHead = ' * '):
+ ret = ''
+ descs = self.WrapDescription(desc)
+ ret = ret + firstLineHead
+ if len(descs) == 0:
+ return ret.rstrip()
+ ret = (ret + descs[0]).rstrip() + '\n'
+ for i in range(1, len(descs)):
+ ret = ret + (otherLineHead + descs[i]).rstrip() + '\n'
+ return ret
+
+ def GetOpDefinitionString(self, use_name, indent=0):
+ ret = ''
+ indentStr = ' ' * indent
+ # define enums if any
+ for arg in self.args:
+ if arg.isEnum and use_name:
+ # comments
+ ret = ret + self.GenDescription(arg.description, \
+ '/*! \\brief ', \
+ ' * ')
+ ret = ret + " */\n"
+ # definition
+ ret = ret + arg.enum.GetDefinitionString(indent) + '\n'
+ # create function comments
+ ret = ret + self.GenDescription(self.description, \
+ '/*!\n * \\brief ', \
+ ' * ')
+ for arg in self.args:
+ if arg.name != 'symbol_name' or use_name:
+ ret = ret + self.GenDescription(arg.name + ' ' + arg.description, \
+ ' * \\param ', \
+ ' * ')
+ ret = ret + " * \\return new symbol\n"
+ ret = ret + " */\n"
+ # create function header
+ declFirstLine = indentStr + 'inline Symbol %s(' % self.name
+ ret = ret + declFirstLine
+ argIndentStr = ' ' * len(declFirstLine)
+ arg_start = 0 if use_name else 1
+ if len(self.args) > arg_start:
+ ret = ret + self.GetArgString(self.args[arg_start])
+ for i in range(arg_start+1, len(self.args)):
+ ret = ret + ',\n'
+ ret = ret + argIndentStr + self.GetArgString(self.args[i])
+ ret = ret + ') {\n'
+ # create function body
+ # if there is enum, generate static enum<->string mapping
+ for arg in self.args:
+ if arg.isEnum:
+ ret = ret + arg.enum.GetEnumStringArray(indent + 2)
+ # now generate code
+ ret = ret + indentStr + ' return Operator(\"%s\")\n' % self.name
+ for arg in self.args: # set params
+ if arg.type == 'Symbol' or \
+ arg.type == 'const std::string&' or \
+ arg.type == 'const std::vector<Symbol>&':
+ continue
+ v = arg.name
+ if arg.isEnum:
+ v = arg.enum.GetConvertEnumVariableToString(v)
+ ret = ret + indentStr + ' ' * 11 + \
+ '.SetParam(\"%s\", %s)\n' % (arg.name, v)
+ #ret = ret[:-1] # get rid of the last \n
+ symbols = ''
+ inputAlreadySet = False
+ for arg in self.args: # set inputs
+ if arg.type != 'Symbol':
+ continue
+ inputAlreadySet = True
+ #if symbols != '':
+ # symbols = symbols + ', '
+ #symbols = symbols + arg.name
+ ret = ret + indentStr + ' ' * 11 + \
+ '.SetInput(\"%s\", %s)\n' % (arg.name, arg.name)
+ for arg in self.args: # set input arrays vector<Symbol>
+ if arg.type != 'const std::vector<Symbol>&':
+ continue
+ if (inputAlreadySet):
+ logging.error("op %s has both Symbol[] and Symbol inputs!" % self.name)
+ inputAlreadySet = True
+ symbols = arg.name
+ ret = ret + '(%s)\n' % symbols
+ ret = ret + indentStr + ' ' * 11
+ if use_name:
+ ret = ret + '.CreateSymbol(symbol_name);\n'
+ else:
+ ret = ret + '.CreateSymbol();\n'
+ ret = ret + indentStr + '}\n'
+ return ret
+
+ def GetArgString(self, arg):
+ ret = '%s %s' % (arg.type, arg.name)
+ if arg.hasDefault:
+ ret = ret + ' = ' + arg.defaultString
+ return ret
+
+
+def ParseAllOps():
+ """
+ MXNET_DLL int MXSymbolListAtomicSymbolCreators(mx_uint *out_size,
+ AtomicSymbolCreator **out_array);
+
+ MXNET_DLL int MXSymbolGetAtomicSymbolInfo(AtomicSymbolCreator creator,
+ const char **name,
+ const char **description,
+ mx_uint *num_args,
+ const char ***arg_names,
+ const char ***arg_type_infos,
+ const char ***arg_descriptions,
+ const char **key_var_num_args);
+ """
+ cdll.libmxnet = cdll.LoadLibrary(find_lib_path()[0])
Review comment:
Basing the cpp-package on the ability to `dlopen` libmxnet at compile-time makes cross-compiling impossible unless the OS is setup for emulating foreign architecture code (ie. transparently running aarch64 on x86). That's not very common. Can we design the cpp package so that it integrates well with the normal compilation workflows?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r624851546
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -320,6 +320,8 @@ build_ubuntu_cpu_openblas() {
-DUSE_BLAS=Open \
-DUSE_ONEDNN=OFF \
-DUSE_CUDA=OFF \
+ -DUSE_CPP_PACKAGE=ON \
+ -DMXNET_USE_CPU=ON \
Review comment:
I'm not sure whether this build option makes much sense as mxnet always has cpu mode enabled. For the lenet example, it would be great to have the code detect the GPU count instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r631514587
##########
File path: cpp-package/README.md
##########
@@ -0,0 +1,67 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+# MXNet - C++ API
+
+The MXNet C++ Package provides C++ API bindings to the users of MXNet. Currently, these bindings are not available as standalone package.
+The users of these bindings are required to build this package as mentioned below.
+
+## Building C++ Package
+
+The cpp-package directory contains the implementation of C++ API. As mentioned above, users are required to build this directory or package before using it.
+**The cpp-package is built while building the MXNet shared library, *libmxnet.so*.**
Review comment:
Thanks for pointing it out. Let me update the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r630578595
##########
File path: cpp-package/include/mxnet-cpp/.gitignore
##########
@@ -0,0 +1,2 @@
+# Rebuildable file(s)
+op.h
Review comment:
Why is op.h placed inside the build dir? Usually we only support out of source build in cmake. This .gitignore may no longer be needed as Makefile in-source build is no longer supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-842529870
@mxnet-bot run ci [centos-gpu, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-830405865
Jenkins CI successfully triggered : [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r631800668
##########
File path: cpp-package/include/mxnet-cpp/.gitignore
##########
@@ -0,0 +1,2 @@
+# Rebuildable file(s)
+op.h
Review comment:
> op.h is generated by OpWrapperGenerator.py from user side
OpWrapperGenerator.py is invoked by cmake inside the `build` folder ("out of source build"). Usually out-of-source build should not modify the code / files outside of the `build` folder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin edited a comment on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin edited a comment on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-833075945
C++ package for master branch is now ready for review.
Changes are listed as follows:
1. Adopt `MXCreateCachedOp` in Executor constructor in `cpp-package/include/mxnet-cpp/executor.hpp`
2. Adopt `MXInvokeCachedOp`, `MXAutogradSetIsTraining` and `MXAutogradSetIsRecording` APIs on executor forward pass to execute graph.
3. Adopt `MXAutogradBackwardEx` API on executor backward pass.
4. Add tests back to CI(unix-gpu) to run some examples in `cpp-package/tests/ci_test.sh`
5. Add multi_threaded_inference test.
6. Add documentation back and add instructions for cross-compilation for cpp-package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816196233
Jenkins CI successfully triggered : [sanity, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu merged pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu merged pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-816996396
@mxnet-bot run ci [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r630574196
##########
File path: cpp-package/tests/travis/run_test.sh
##########
@@ -0,0 +1,42 @@
+#!/bin/bash
Review comment:
Please don't add unused files
##########
File path: cpp-package/tests/ci_test.sh
##########
@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e # exit on the first error
+cd $(dirname $(readlink -f $0))/../example
+echo $PWD
+export LD_LIBRARY_PATH=$(readlink -f /work/build):$LD_LIBRARY_PATH
+echo $LD_LIBRARY_PATH
+ls -l /work/build/
+
+./get_data.sh
+
+cp /work/build/cpp-package/example/lenet .
+./lenet 1
+
+cp /work/build/cpp-package/example/alexnet .
+./alexnet 1
+
+cp /work/build/cpp-package/example/lenet_with_mxdataiter .
+./lenet_with_mxdataiter 1
+
+cp /work/build/cpp-package/example/resnet .
+./resnet 1
+
+cp /work/build/cpp-package/example/inception_bn .
+./inception_bn 1
+
+cp /work/build/cpp-package/example/mlp .
+./mlp 150
+
+cp /work/build/cpp-package/example/mlp_cpu .
+./mlp_cpu
+
+cp /work/build/cpp-package/example/mlp_gpu .
+./mlp_gpu
+
+cp /work/build/cpp-package/example/test_optimizer .
+./test_optimizer
+
+cp /work/build/cpp-package/example/test_kvstore .
+./test_kvstore
+
+cp /work/build/cpp-package/example/test_score .
+./test_score 0.93
+
+cp /work/build/cpp-package/example/test_ndarray_copy .
+./test_ndarray_copy
+
+# skippping temporarily, tracked by https://github.com/apache/incubator-mxnet/issues/20011
+#cp ../../build/cpp-package/example/test_regress_label .
+#./test_regress_label
+
+# sh unittests/unit_test_mlp_csv.sh
Review comment:
Have you tried running?
##########
File path: cpp-package/README.md
##########
@@ -0,0 +1,67 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+# MXNet - C++ API
+
+The MXNet C++ Package provides C++ API bindings to the users of MXNet. Currently, these bindings are not available as standalone package.
+The users of these bindings are required to build this package as mentioned below.
+
+## Building C++ Package
+
+The cpp-package directory contains the implementation of C++ API. As mentioned above, users are required to build this directory or package before using it.
+**The cpp-package is built while building the MXNet shared library, *libmxnet.so*.**
Review comment:
There is no reason for that. cpp-package depends only on libmxnet.so C APIs. It would be better to keep a separate CMakeLists.txt for the cpp-package with the only requirement to find libmxnet.so. This implies removing the `USE_CPP_PACKAGE` in the main CMakeLists.txt
##########
File path: cpp-package/include/mxnet-cpp/.gitignore
##########
@@ -0,0 +1,2 @@
+# Rebuildable file(s)
+op.h
Review comment:
Why is op.h placed inside the build dir? Usually we only support out of source build in cmake
##########
File path: src/operator/softmax_output-inl.h
##########
@@ -0,0 +1,476 @@
+/*
Review comment:
These files were removed in https://github.com/apache/incubator-mxnet/pull/18531
What's the intention of adding them back? If you like to reconsider the plan for supporting the Module API in MXNet 2, why just add back SoftmaxOutput and not the other loss function? Should this be discussed in the https://github.com/apache/incubator-mxnet/issues/16167 RFC (ie. providing an update that you want to support the Module API?)
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -625,12 +625,16 @@ build_ubuntu_gpu_onednn_nocudnn() {
build_ubuntu_gpu() {
set -ex
cd /work/build
+ ln -s -f /usr/local/cuda/targets/x86_64-linux/lib/stubs/libcuda.so libcuda.so.1
+ export LIBRARY_PATH=${LIBRARY_PATH}:/work/build
+ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/work/build
Review comment:
This should be removed after https://github.com/apache/incubator-mxnet/issues/17858
Would you add a comment pointing out the temporary nature of this?
##########
File path: cpp-package/example/inference/unit_test_imagenet_inference.sh
##########
@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -ex
+# create ./model directory if not existed
+if [ ! -d model ]; then
+ mkdir -p model
+fi
+# create ./data directory if not existed
+if [ ! -d data ]; then
+ mkdir -p data
+fi
+# Downloading the data and model if not existed
+model_file=./model/Inception-BN-symbol.json
+params_file=./model/Inception-BN-0126.params
+if [ ! -f ${model_file} ] || [ ! -f ${params_file} ]; then
+ wget -nc http://data.mxnet.io/models/imagenet/inception-bn.tar.gz
+ tar -xvzf inception-bn.tar.gz -C model
+fi
+cd model
+wget -nc https://raw.githubusercontent.com/dmlc/gluon-cv/master/gluoncv/model_zoo/quantized/resnet50_v1_int8-symbol.json
+cd ../data
+wget -nc http://data.mxnet.io/data/val_256_q90.rec
+cd ..
+
+# Running inference on imagenet.
+if [ "$(uname)" == "Darwin" ]; then
+ echo ">>> INFO: FP32 real data"
+ DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --params_file "./model/Inception-BN-0126.params" --dataset "./data/val_256_q90.rec" --rgb_mean "123.68 116.779 103.939" --batch_size 1 --num_skipped_batches 50 --num_inference_batches 500
+
+ echo ">>> INFO: FP32 dummy data"
+ DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --batch_size 1 --num_inference_batches 500 --benchmark
+else
+ echo ">>> INFO: FP32 real data"
+ LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --params_file "./model/Inception-BN-0126.params" --dataset "./data/val_256_q90.rec" --rgb_mean "123.68 116.779 103.939" --batch_size 1 --num_skipped_batches 50 --num_inference_batches 500
+
+ echo ">>> INFO: FP32 dummy data"
+ LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --batch_size 1 --num_inference_batches 500 --benchmark
+
+ lib_name=$(ls -a ../../../lib | grep -oE 'mkldnn' | tail -1)
Review comment:
You need to check this. Likely should be onednn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-814425300
Hey @barry-jin , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [unix-gpu, edge, miscellaneous, centos-cpu, clang, website, sanity, windows-gpu, windows-cpu, centos-gpu, unix-cpu]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
leezu commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r630575819
##########
File path: src/operator/softmax_output-inl.h
##########
@@ -0,0 +1,476 @@
+/*
Review comment:
These files were removed in https://github.com/apache/incubator-mxnet/pull/18531 If you like to reconsider the plan for supporting the Module-like API in MXNet 2, why just add back SoftmaxOutput and not the other loss function? Should this be discussed in the https://github.com/apache/incubator-mxnet/issues/16167 RFC (ie. providing an update that you want to support the Module API?)
##########
File path: cpp-package/example/inference/unit_test_imagenet_inference.sh
##########
@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -ex
+# create ./model directory if not existed
+if [ ! -d model ]; then
+ mkdir -p model
+fi
+# create ./data directory if not existed
+if [ ! -d data ]; then
+ mkdir -p data
+fi
+# Downloading the data and model if not existed
+model_file=./model/Inception-BN-symbol.json
+params_file=./model/Inception-BN-0126.params
+if [ ! -f ${model_file} ] || [ ! -f ${params_file} ]; then
+ wget -nc http://data.mxnet.io/models/imagenet/inception-bn.tar.gz
+ tar -xvzf inception-bn.tar.gz -C model
+fi
+cd model
+wget -nc https://raw.githubusercontent.com/dmlc/gluon-cv/master/gluoncv/model_zoo/quantized/resnet50_v1_int8-symbol.json
+cd ../data
+wget -nc http://data.mxnet.io/data/val_256_q90.rec
+cd ..
+
+# Running inference on imagenet.
+if [ "$(uname)" == "Darwin" ]; then
+ echo ">>> INFO: FP32 real data"
+ DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --params_file "./model/Inception-BN-0126.params" --dataset "./data/val_256_q90.rec" --rgb_mean "123.68 116.779 103.939" --batch_size 1 --num_skipped_batches 50 --num_inference_batches 500
+
+ echo ">>> INFO: FP32 dummy data"
+ DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --batch_size 1 --num_inference_batches 500 --benchmark
+else
+ echo ">>> INFO: FP32 real data"
+ LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --params_file "./model/Inception-BN-0126.params" --dataset "./data/val_256_q90.rec" --rgb_mean "123.68 116.779 103.939" --batch_size 1 --num_skipped_batches 50 --num_inference_batches 500
+
+ echo ">>> INFO: FP32 dummy data"
+ LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../../../lib ./imagenet_inference --symbol_file "./model/Inception-BN-symbol.json" --batch_size 1 --num_inference_batches 500 --benchmark
+
+ lib_name=$(ls -a ../../../lib | grep -oE 'mkldnn' | tail -1)
Review comment:
Please double check this. Likely should be onednn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-838824435
Jenkins CI successfully triggered : [centos-gpu, centos-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-829783440
@mxnet-bot run ci [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-830478257
Jenkins CI successfully triggered : [centos-gpu, miscellaneous, unix-gpu, unix-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin edited a comment on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin edited a comment on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-833075945
C++ package for master branch is now ready for review.
Changes are listed as follows:
1. Adopt `MXCreateCachedOp` in Executor constructor in `cpp-package/include/mxnet-cpp/executor.hpp`
2. Adopt `MXInvokeCachedOp`, `MXAutogradSetIsTraining` and `MXAutogradSetIsRecording` APIs on executor forward pass to execute graph.
3. Adopt `MXAutogradBackwardEx` API on executor backward pass.
4. Add tests back to CI(unix-gpu) to run some examples in `cpp-package/tests/ci_test.sh`
5. Add multi_threaded_inference in `cpp-package/example/inference/multi_threaded_inference`
6. Add documentation back and add instructions for cross-compilation for cpp-package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r631516061
##########
File path: cpp-package/include/mxnet-cpp/.gitignore
##########
@@ -0,0 +1,2 @@
+# Rebuildable file(s)
+op.h
Review comment:
op.h is generated by OpWrapperGenerator.py from user side, which should not be tracked by git. Should I just put this .gitignore in cpp-package directory or merged with incubator-mxnet/.gitignore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#discussion_r628986387
##########
File path: cpp-package/example/Makefile
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
since we switched to cmake for the overall build system, it's better not to add make support back.
##########
File path: cpp-package/include/mxnet-cpp/CPPLINT.cfg
##########
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
we had cpplint in sanity check before so it's probably a good idea to bring it back.
##########
File path: cpp-package/example/README.md
##########
@@ -0,0 +1,131 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+# MXNet C++ Package Examples
+
+## Building C++ examples
+
+The examples in this folder demonstrate the **training** workflow. The **inference workflow** related examples can be found in [inference](<https://github.com/apache/incubator-mxnet/blob/master/cpp-package/example/inference>) folder.
+Please build the MXNet C++ Package as explained in the [README](<https://github.com/apache/incubator-mxnet/tree/master/cpp-package#building-c-package>) File before building these examples manually.
+The examples in this folder are built while building the MXNet library and cpp-package from source. However, they can be built manually as follows
+
+From cpp-package/examples directory
+
+- Build all examples in release mode: **make all**
+- Build all examples in debug mode: **make debug**
Review comment:
switch to cmake
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-838824281
@mxnet-bot run ci [centos-gpu, centos-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-842529914
Jenkins CI successfully triggered : [centos-gpu, unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-829783467
Jenkins CI successfully triggered : [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-833075945
C++ package for master branch is now ready for review.
Changes are listed as follows:
1. Adopt `MXCreateCachedOp` in Executor constructor in `cpp-package/include/mxnet-cpp/executor.hpp`
2. Adopt `MXInvokeCachedOp`, `MXAutogradSetIsTraining` and `MXAutogradSetIsRecording` APIs on executor forward pass to execute graph.
3. Adopt `MXAutogradBackwardEx` API on executor backward pass.
4. Add tests back to CI(unix-gpu) to run some examples in `cpp-package/tests/ci_test.sh`
5. Add documentation back and add instructions for cross-compilation for cpp-package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on pull request #20131: [WIP][2.0] Add cpp-package
Posted by GitBox <gi...@apache.org>.
barry-jin commented on pull request #20131:
URL: https://github.com/apache/incubator-mxnet/pull/20131#issuecomment-830405814
@mxnet-bot run ci [unix-gpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org