You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Anton Chernov <me...@gmail.com> on 2018/12/05 21:06:55 UTC
Re: using conan to manage Apache Incubator MXNet project dependencies
What I rather meant is a 'sandboxed build', for example without internet
connection. Would this be possible?
Or is it possible to define and maintain own conan package registry?
ср, 28 нояб. 2018 г. в 16:33, Konstantin Ivlev <to...@gmail.com>:
> > Is it possible to have an offline build somehow? For example, if all
> > dependencies would be stored locally. Probably, that would require some
> > modifications to the conan file, right?
> as soon as all dependencies are stored locally,
> conan will use binaries from the local cache.
> no modifications are required to conan file or workflow in this case.
>
> ср, 28 нояб. 2018 г. в 22:24, Anton Chernov <me...@gmail.com>:
>
> > Great, thank you for your answers.
> >
> > Is it possible to have an offline build somehow? For example, if all
> > dependencies would be stored locally. Probably, that would require some
> > modifications to the conan file, right?
> >
> >
> > вт, 27 нояб. 2018 г. в 16:59, Konstantin Ivlev <to...@gmail.com>:
> >
> > > > Would it be possible to define cmake itself as a conan dependency
> for
> > > MXNet?
> > > yes, it is definitely possible
> > > for instance, to declare CMake 3.13.0 as a dependency, the following
> line
> > > has to be added to the conanfile.py:
> > >
> > > build_requires = "cmake_installer/3.13.0@conan/stable"
> > >
> > > other build tools might be added in a similar manner.
> > >
> > > вт, 27 нояб. 2018 г. в 22:54, Anton Chernov <me...@gmail.com>:
> > >
> > > > I think I asked this already, but want to confirm in regards to the
> > > > following discussion:
> > > >
> > > > MXNet CMake build - raise minimal required version
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/09772f9447ebff72cdfea5512ae70e295db8a21ba0ddb39359cd0a77@%3Cdev.mxnet.apache.org%3E
> > > >
> > > > Would it be possible to define cmake itself as a conan dependency for
> > > > MXNet?
> > > >
> > > > Best
> > > > Anton
> > > >
> > > > вт, 27 нояб. 2018 г. в 15:44, Konstantin Ivlev <tomskside@gmail.com
> >:
> > > >
> > > > > both questions are totally valid.
> > > > > > Is it easy to create a build which will build dependencies from
> > > source?
> > > > > 1. yep, it's very easy, just add `--build` argument to the `conan
> > > > install`
> > > > > command line
> > > > > > What guarantees you get with conan with regards to ABI / C++
> > stdlib
> > > > > binary compatibility of the pulled dependencies?
> > > > > 2. the default binary compatibility model may be found in
> > settings.yml
> > > (
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/conan-io/conan/blob/develop/conans/client/conf/__init__.py#L14
> > > > > )
> > > > > it includes the operation system, architecture, compiler, its
> version
> > > and
> > > > > C++ stdlib.
> > > > > while it's true that it won't cover 100% of use-cases (e.g. for
> older
> > > > > distros or older glibc), it's very good default for most users.
> > > > > you may always define custom binary compatibility attributes in
> your
> > > > > settings.yml (e.g. distro, microarchitecture, sanitizers, etc.).
> > > > >
> > > > >
> > > > > вт, 27 нояб. 2018 г. в 21:17, Pedro Larroy <
> > > pedro.larroy.lists@gmail.com
> > > > >:
> > > > >
> > > > > > Thanks both for the detailed explanations. Couple of more
> > questions:
> > > > > >
> > > > > > Is it easy to create a build which will build dependencies from
> > > source?
> > > > > > What guarantees you get with conan with regards to ABI / C++
> stdlib
> > > > > > binary compatibility of the pulled dependencies?
> > > > > >
> > > > > > Just to clarify: My concerns are in terms of reproducible builds
> /
> > > > > > source only distribution and undefined behaviour due to different
> > > > > > compiler / stdlib versions. Are these valid or is it oudated
> > > > > > knowledge?
> > > > > >
> > > > > > Pedro.
> > > > > > On Tue, Nov 27, 2018 at 2:34 PM Diego Rodriguez-Losada
> > > > > > <di...@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi Pedro,
> > > > > > >
> > > > > > > Conan is distributed. So besides building from sources the
> > > > > dependencies,
> > > > > > it
> > > > > > > is also possible to create binaries yourself for those
> > dependencies
> > > > > (with
> > > > > > > the existing recipes, or your own recipes), and host them in
> your
> > > own
> > > > > > repo
> > > > > > > (Bintray OSS repo, or Artifactory).
> > > > > > >
> > > > > > > This will provide both the security that you own the
> dependencies
> > > > > > binaries
> > > > > > > and the convenience and speed of not having to build from
> > sources.
> > > > Even
> > > > > > if
> > > > > > > you provide the binaries, consumers can always fallback to
> build
> > > from
> > > > > > > sources too.
> > > > > > >
> > > > > > > Kind regards,
> > > > > > > Diego
> > > > > > >
> > > > > > > El mar., 27 nov. 2018 a las 13:34, Konstantin Ivlev (<
> > > > > > tomskside@gmail.com>)
> > > > > > > escribió:
> > > > > > >
> > > > > > > > Hi Pedro,
> > > > > > > >
> > > > > > > > yes, you're absolutely right, by default, conan will be
> pulling
> > > > > > prebuilt
> > > > > > > > binaries for the libraries from the bintray.
> > > > > > > > however, if prebuilt binaries are not available (e.g. because
> > you
> > > > use
> > > > > > some
> > > > > > > > different compiler for which we don't have prebuilt
> binaries),
> > > > > > > > or if you want to build binaries yourself for some another
> > > reason,
> > > > > > > > then libraries always might be built from source (by passing
> > e.g.
> > > > > > "--build
> > > > > > > > always", "--build missing" or "--build <library>" to the
> conan
> > > > > install
> > > > > > > > command line).
> > > > > > > >
> > > > > > > > yours sincerely, Konstantin
> > > > > > > >
> > > > > > > > вт, 27 нояб. 2018 г. в 19:27, Pedro Larroy <
> > > > > > pedro.larroy.lists@gmail.com>:
> > > > > > > >
> > > > > > > > > Hi Konstantin
> > > > > > > > >
> > > > > > > > > Thanks for this contribution. With your proposed changes,
> > when
> > > > > > > > > building MXNet we will be pulling binaries for the
> libraries
> > > > > managed
> > > > > > > > > by conan?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Pedro.
> > > > > > > > > On Mon, Nov 26, 2018 at 11:43 AM Konstantin Ivlev <
> > > > > > tomskside@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hello, Ivan,
> > > > > > > > > >
> > > > > > > > > > could you possibly clarify your question (may be
> explaining
> > > the
> > > > > > > > use-case
> > > > > > > > > > behind it)?
> > > > > > > > > > Gradle appears to be build system, AFAIK more popular in
> > Java
> > > > > > world.
> > > > > > > > > > meanwhile, Apache Incubator MXNet project uses CMake as
> its
> > > > build
> > > > > > > > system.
> > > > > > > > > > please correct me, I am wrong.
> > > > > > > > > > in general, conan, as a package manager, is pretty
> > > > > > > > build-system-agnostic,
> > > > > > > > > > and it may work with arbitrary build systems (to count a
> > few,
> > > > > > CMake,
> > > > > > > > > > premake, qmake, full list:
> > > > > > > > > >
> https://docs.conan.io/en/latest/reference/generators.html
> > ).
> > > I
> > > > > > don't
> > > > > > > > > think
> > > > > > > > > > Gradle is exception here.
> > > > > > > > > > also, for instance, Android Studio also uses Gradle for
> > > Android
> > > > > C++
> > > > > > > > > > projects, and conan works flawlessly in this particular
> > case
> > > (
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://blog.conan.io/2018/02/13/Android-Studio-project-Conan-Boost.html
> > > > > > > > > ).
> > > > > > > > > >
> > > > > > > > > > yours sincerely, Konstantin
> > > > > > > > > >
> > > > > > > > > > пн, 26 нояб. 2018 г. в 16:43, Ivan Serdyuk <
> > > > > > > > local.tourist.kiev@gmail.com
> > > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Kostantin, and what (overall) option with using Gradle?
> > > Does
> > > > > this
> > > > > > > > your
> > > > > > > > > > > suggested package manager has been supported by Gradle?
> > > > > > > > > > >
> > > > > > > > > > > Ivan
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 26, 2018 at 9:43 AM Konstantin Ivlev <
> > > > > > > > tomskside@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > hello,
> > > > > > > > > > > >
> > > > > > > > > > > > this email is related to the following PR and JIRA
> > > ticket:
> > > > > > > > > > > > - [MXNET-1229] use OpenBLAS, lapack & OpenCV from
> conan
> > > > > > > > > > > > <
> https://github.com/apache/incubator-mxnet/pull/13400>
> > > > > > > > > > > > - use conan to manage project dependencies
> > > > > > > > > > > > <https://issues.apache.org/jira/browse/MXNET-1229>
> > > > > > > > > > > >
> > > > > > > > > > > > conan <https://conan.io> is an open-source package
> > > manager
> > > > > > for C++
> > > > > > > > > > > > projects. it allows to manage project dependencies in
> > > > > > transparent
> > > > > > > > and
> > > > > > > > > > > > declarative manner.
> > > > > > > > > > > >
> > > > > > > > > > > > currently, apache incubator-mxnet project uses the
> > > > following
> > > > > > > > > different
> > > > > > > > > > > > ways to manage its dependencies:
> > > > > > > > > > > >
> > > > > > > > > > > > - download GitHub archives during the build
> > > > > > > > > > > > - OpenBLAS <https://github.com/xianyi/OpenBLAS>
> > > > > > > > > > > > - OpenCV <https://github.com/opencv/opencv>
> > > > > > > > > > > > - conda <https://conda.io/> (alternative way to
> GitHub
> > > > > > archives)
> > > > > > > > > > > > - download from CMake
> > > > > > > > > > > > - Intel Math Kernel Library <
> > > > > > > > > https://software.intel.com/en-us/mkl>
> > > > > > > > > > > > (MKL)
> > > > > > > > > > > > - Git submodules
> > > > > > > > > > > > - cub <https://github.com/dmlc/cub>
> > > > > > > > > > > > - dlpack <https://github.com/dmlc/dlpack>
> > > > > > > > > > > > - dmlc-core <https://github.com/dmlc/dmlc-core>
> > > > > > > > > > > > - googletest <
> https://github.com/abseil/googletest
> > >
> > > > > > > > > > > > - mkldnn <https://github.com/intel/mkl-dnn>
> > > > > > > > > > > > - mshadow <https://github.com/dmlc/mshadow>
> > > > > > > > > > > > - onnx-tensorrt <
> > > https://github.com/onnx/onnx-tensorrt
> > > > >
> > > > > > > > > > > > - openmp <https://github.com/llvm-mirror/openmp>
> > > > > > > > > > > > - ps-lite <https://github.com/dmlc/ps-lite>
> > > > > > > > > > > > - tvm <https://github.com/dmlc/tvm>
> > > > > > > > > > > >
> > > > > > > > > > > > this appears to be very heterogeneous and hard to
> > > > > > manage/maintain,
> > > > > > > > as
> > > > > > > > > > > > multiple various commands are in use to achieve
> > > > dependencies
> > > > > > > > > > > installation,
> > > > > > > > > > > > as well as multiple places are to look for dependency
> > > > > versions
> > > > > > and
> > > > > > > > > their
> > > > > > > > > > > > updates.
> > > > > > > > > > > >
> > > > > > > > > > > > with conan, it may became much more straightforward,
> as
> > > > > > > > dependencies
> > > > > > > > > will
> > > > > > > > > > > > be declared in single place (conanfile) and installed
> > via
> > > > > > single
> > > > > > > > > command
> > > > > > > > > > > > (conan install).
> > > > > > > > > > > >
> > > > > > > > > > > > as project is very complex, and has lots of
> > dependencies,
> > > > for
> > > > > > the
> > > > > > > > > first
> > > > > > > > > > > > prototype I've used only very few of dependencies
> from
> > > > conan:
> > > > > > > > OpenCV
> > > > > > > > > > > > <
> > > https://bintray.com/conan-community/conan/opencv%3Aconan
> > > > >,
> > > > > > > > OpenBLAS
> > > > > > > > > > > > <
> > > > https://bintray.com/conan-community/conan/openblas%3Aconan>
> > > > > > and
> > > > > > > > > lapack
> > > > > > > > > > > > <
> > > https://bintray.com/conan-community/conan/lapack%3Aconan
> > > > >.
> > > > > > > > > > > > others may be easily added then one by one, but they
> > > first
> > > > > has
> > > > > > to
> > > > > > > > be
> > > > > > > > > > > > packaged (not all of them are packaged yet, e.g.
> > > GoogleTest
> > > > > > > > > > > > <
> > > > > > https://bintray.com/bincrafters/public-conan/gtest%3Abincrafters
> >
> > > > > > > > > is
> > > > > > > > > > > > available, while MKL is not).
> > > > > > > > > > > >
> > > > > > > > > > > > I attach patch which adds an initial conan support as
> > > proof
> > > > > of
> > > > > > > > > concept.
> > > > > > > > > > > > also, I attach two simple build scripts, which I've
> > used
> > > to
> > > > > > test
> > > > > > > > (for
> > > > > > > > > > > > Windows and Linux / Mac OS X). Google Mail blocks .sh
> > and
> > > > > .cmd
> > > > > > > > > > > extensions,
> > > > > > > > > > > > so you'll need to rename files.
> > > > > > > > > > > > lemme know if you have any further questions.
> > > > > > > > > > > >
> > > > > > > > > > > > yours sincerely, Konstantin
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: using conan to manage Apache Incubator MXNet project dependencies
Posted by Diego Rodriguez-Losada <di...@gmail.com>.
Yes, it is totally possible to define and maintain your own conan package
registry with your own packages and binaries, for your dependencies and
your own packages too:
- You can use a free Bintray OSS account, create your own repository
(registry) there.
- You can use an Artifactory, there are several flavors, you could
self-host it (Artifactory CE for C and C++ Conan is free), use an online
cloud one (paid monthly) one, try to apply for an OSS program (that will
get you a free Artifactory online, if approved)
A sandboxed build would be a totally different approach, and barely
maintainable and usable, in my opinion. You'd need to bundle there all the
dependencies for all the possible binary configurations (humongous image),
or maintain a lot of different sandboxes (images). And how do you retrieve
it in the first place if you have no internet connection? Conan follows the
same paradigm as other language and system package managers, it is a
decentralized client-server approach, in which the client is able to
install packages from the remote registries (repositorie/remotes) you want.
El mié., 5 dic. 2018 a las 22:07, Anton Chernov (<me...@gmail.com>)
escribió:
> What I rather meant is a 'sandboxed build', for example without internet
> connection. Would this be possible?
>
> Or is it possible to define and maintain own conan package registry?
>
> ср, 28 нояб. 2018 г. в 16:33, Konstantin Ivlev <to...@gmail.com>:
>
> > > Is it possible to have an offline build somehow? For example, if all
> > > dependencies would be stored locally. Probably, that would require some
> > > modifications to the conan file, right?
> > as soon as all dependencies are stored locally,
> > conan will use binaries from the local cache.
> > no modifications are required to conan file or workflow in this case.
> >
> > ср, 28 нояб. 2018 г. в 22:24, Anton Chernov <me...@gmail.com>:
> >
> > > Great, thank you for your answers.
> > >
> > > Is it possible to have an offline build somehow? For example, if all
> > > dependencies would be stored locally. Probably, that would require some
> > > modifications to the conan file, right?
> > >
> > >
> > > вт, 27 нояб. 2018 г. в 16:59, Konstantin Ivlev <to...@gmail.com>:
> > >
> > > > > Would it be possible to define cmake itself as a conan dependency
> > for
> > > > MXNet?
> > > > yes, it is definitely possible
> > > > for instance, to declare CMake 3.13.0 as a dependency, the following
> > line
> > > > has to be added to the conanfile.py:
> > > >
> > > > build_requires = "cmake_installer/3.13.0@conan/stable"
> > > >
> > > > other build tools might be added in a similar manner.
> > > >
> > > > вт, 27 нояб. 2018 г. в 22:54, Anton Chernov <me...@gmail.com>:
> > > >
> > > > > I think I asked this already, but want to confirm in regards to the
> > > > > following discussion:
> > > > >
> > > > > MXNet CMake build - raise minimal required version
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/09772f9447ebff72cdfea5512ae70e295db8a21ba0ddb39359cd0a77@%3Cdev.mxnet.apache.org%3E
> > > > >
> > > > > Would it be possible to define cmake itself as a conan dependency
> for
> > > > > MXNet?
> > > > >
> > > > > Best
> > > > > Anton
> > > > >
> > > > > вт, 27 нояб. 2018 г. в 15:44, Konstantin Ivlev <
> tomskside@gmail.com
> > >:
> > > > >
> > > > > > both questions are totally valid.
> > > > > > > Is it easy to create a build which will build dependencies from
> > > > source?
> > > > > > 1. yep, it's very easy, just add `--build` argument to the `conan
> > > > > install`
> > > > > > command line
> > > > > > > What guarantees you get with conan with regards to ABI / C++
> > > stdlib
> > > > > > binary compatibility of the pulled dependencies?
> > > > > > 2. the default binary compatibility model may be found in
> > > settings.yml
> > > > (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/conan-io/conan/blob/develop/conans/client/conf/__init__.py#L14
> > > > > > )
> > > > > > it includes the operation system, architecture, compiler, its
> > version
> > > > and
> > > > > > C++ stdlib.
> > > > > > while it's true that it won't cover 100% of use-cases (e.g. for
> > older
> > > > > > distros or older glibc), it's very good default for most users.
> > > > > > you may always define custom binary compatibility attributes in
> > your
> > > > > > settings.yml (e.g. distro, microarchitecture, sanitizers, etc.).
> > > > > >
> > > > > >
> > > > > > вт, 27 нояб. 2018 г. в 21:17, Pedro Larroy <
> > > > pedro.larroy.lists@gmail.com
> > > > > >:
> > > > > >
> > > > > > > Thanks both for the detailed explanations. Couple of more
> > > questions:
> > > > > > >
> > > > > > > Is it easy to create a build which will build dependencies from
> > > > source?
> > > > > > > What guarantees you get with conan with regards to ABI / C++
> > stdlib
> > > > > > > binary compatibility of the pulled dependencies?
> > > > > > >
> > > > > > > Just to clarify: My concerns are in terms of reproducible
> builds
> > /
> > > > > > > source only distribution and undefined behaviour due to
> different
> > > > > > > compiler / stdlib versions. Are these valid or is it oudated
> > > > > > > knowledge?
> > > > > > >
> > > > > > > Pedro.
> > > > > > > On Tue, Nov 27, 2018 at 2:34 PM Diego Rodriguez-Losada
> > > > > > > <di...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi Pedro,
> > > > > > > >
> > > > > > > > Conan is distributed. So besides building from sources the
> > > > > > dependencies,
> > > > > > > it
> > > > > > > > is also possible to create binaries yourself for those
> > > dependencies
> > > > > > (with
> > > > > > > > the existing recipes, or your own recipes), and host them in
> > your
> > > > own
> > > > > > > repo
> > > > > > > > (Bintray OSS repo, or Artifactory).
> > > > > > > >
> > > > > > > > This will provide both the security that you own the
> > dependencies
> > > > > > > binaries
> > > > > > > > and the convenience and speed of not having to build from
> > > sources.
> > > > > Even
> > > > > > > if
> > > > > > > > you provide the binaries, consumers can always fallback to
> > build
> > > > from
> > > > > > > > sources too.
> > > > > > > >
> > > > > > > > Kind regards,
> > > > > > > > Diego
> > > > > > > >
> > > > > > > > El mar., 27 nov. 2018 a las 13:34, Konstantin Ivlev (<
> > > > > > > tomskside@gmail.com>)
> > > > > > > > escribió:
> > > > > > > >
> > > > > > > > > Hi Pedro,
> > > > > > > > >
> > > > > > > > > yes, you're absolutely right, by default, conan will be
> > pulling
> > > > > > > prebuilt
> > > > > > > > > binaries for the libraries from the bintray.
> > > > > > > > > however, if prebuilt binaries are not available (e.g.
> because
> > > you
> > > > > use
> > > > > > > some
> > > > > > > > > different compiler for which we don't have prebuilt
> > binaries),
> > > > > > > > > or if you want to build binaries yourself for some another
> > > > reason,
> > > > > > > > > then libraries always might be built from source (by
> passing
> > > e.g.
> > > > > > > "--build
> > > > > > > > > always", "--build missing" or "--build <library>" to the
> > conan
> > > > > > install
> > > > > > > > > command line).
> > > > > > > > >
> > > > > > > > > yours sincerely, Konstantin
> > > > > > > > >
> > > > > > > > > вт, 27 нояб. 2018 г. в 19:27, Pedro Larroy <
> > > > > > > pedro.larroy.lists@gmail.com>:
> > > > > > > > >
> > > > > > > > > > Hi Konstantin
> > > > > > > > > >
> > > > > > > > > > Thanks for this contribution. With your proposed changes,
> > > when
> > > > > > > > > > building MXNet we will be pulling binaries for the
> > libraries
> > > > > > managed
> > > > > > > > > > by conan?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Pedro.
> > > > > > > > > > On Mon, Nov 26, 2018 at 11:43 AM Konstantin Ivlev <
> > > > > > > tomskside@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hello, Ivan,
> > > > > > > > > > >
> > > > > > > > > > > could you possibly clarify your question (may be
> > explaining
> > > > the
> > > > > > > > > use-case
> > > > > > > > > > > behind it)?
> > > > > > > > > > > Gradle appears to be build system, AFAIK more popular
> in
> > > Java
> > > > > > > world.
> > > > > > > > > > > meanwhile, Apache Incubator MXNet project uses CMake as
> > its
> > > > > build
> > > > > > > > > system.
> > > > > > > > > > > please correct me, I am wrong.
> > > > > > > > > > > in general, conan, as a package manager, is pretty
> > > > > > > > > build-system-agnostic,
> > > > > > > > > > > and it may work with arbitrary build systems (to count
> a
> > > few,
> > > > > > > CMake,
> > > > > > > > > > > premake, qmake, full list:
> > > > > > > > > > >
> > https://docs.conan.io/en/latest/reference/generators.html
> > > ).
> > > > I
> > > > > > > don't
> > > > > > > > > > think
> > > > > > > > > > > Gradle is exception here.
> > > > > > > > > > > also, for instance, Android Studio also uses Gradle for
> > > > Android
> > > > > > C++
> > > > > > > > > > > projects, and conan works flawlessly in this particular
> > > case
> > > > (
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://blog.conan.io/2018/02/13/Android-Studio-project-Conan-Boost.html
> > > > > > > > > > ).
> > > > > > > > > > >
> > > > > > > > > > > yours sincerely, Konstantin
> > > > > > > > > > >
> > > > > > > > > > > пн, 26 нояб. 2018 г. в 16:43, Ivan Serdyuk <
> > > > > > > > > local.tourist.kiev@gmail.com
> > > > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Kostantin, and what (overall) option with using
> Gradle?
> > > > Does
> > > > > > this
> > > > > > > > > your
> > > > > > > > > > > > suggested package manager has been supported by
> Gradle?
> > > > > > > > > > > >
> > > > > > > > > > > > Ivan
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 26, 2018 at 9:43 AM Konstantin Ivlev <
> > > > > > > > > tomskside@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > hello,
> > > > > > > > > > > > >
> > > > > > > > > > > > > this email is related to the following PR and JIRA
> > > > ticket:
> > > > > > > > > > > > > - [MXNET-1229] use OpenBLAS, lapack & OpenCV from
> > conan
> > > > > > > > > > > > > <
> > https://github.com/apache/incubator-mxnet/pull/13400>
> > > > > > > > > > > > > - use conan to manage project dependencies
> > > > > > > > > > > > > <https://issues.apache.org/jira/browse/MXNET-1229>
> > > > > > > > > > > > >
> > > > > > > > > > > > > conan <https://conan.io> is an open-source package
> > > > manager
> > > > > > > for C++
> > > > > > > > > > > > > projects. it allows to manage project dependencies
> in
> > > > > > > transparent
> > > > > > > > > and
> > > > > > > > > > > > > declarative manner.
> > > > > > > > > > > > >
> > > > > > > > > > > > > currently, apache incubator-mxnet project uses the
> > > > > following
> > > > > > > > > > different
> > > > > > > > > > > > > ways to manage its dependencies:
> > > > > > > > > > > > >
> > > > > > > > > > > > > - download GitHub archives during the build
> > > > > > > > > > > > > - OpenBLAS <https://github.com/xianyi/OpenBLAS
> >
> > > > > > > > > > > > > - OpenCV <https://github.com/opencv/opencv>
> > > > > > > > > > > > > - conda <https://conda.io/> (alternative way to
> > GitHub
> > > > > > > archives)
> > > > > > > > > > > > > - download from CMake
> > > > > > > > > > > > > - Intel Math Kernel Library <
> > > > > > > > > > https://software.intel.com/en-us/mkl>
> > > > > > > > > > > > > (MKL)
> > > > > > > > > > > > > - Git submodules
> > > > > > > > > > > > > - cub <https://github.com/dmlc/cub>
> > > > > > > > > > > > > - dlpack <https://github.com/dmlc/dlpack>
> > > > > > > > > > > > > - dmlc-core <https://github.com/dmlc/dmlc-core
> >
> > > > > > > > > > > > > - googletest <
> > https://github.com/abseil/googletest
> > > >
> > > > > > > > > > > > > - mkldnn <https://github.com/intel/mkl-dnn>
> > > > > > > > > > > > > - mshadow <https://github.com/dmlc/mshadow>
> > > > > > > > > > > > > - onnx-tensorrt <
> > > > https://github.com/onnx/onnx-tensorrt
> > > > > >
> > > > > > > > > > > > > - openmp <
> https://github.com/llvm-mirror/openmp>
> > > > > > > > > > > > > - ps-lite <https://github.com/dmlc/ps-lite>
> > > > > > > > > > > > > - tvm <https://github.com/dmlc/tvm>
> > > > > > > > > > > > >
> > > > > > > > > > > > > this appears to be very heterogeneous and hard to
> > > > > > > manage/maintain,
> > > > > > > > > as
> > > > > > > > > > > > > multiple various commands are in use to achieve
> > > > > dependencies
> > > > > > > > > > > > installation,
> > > > > > > > > > > > > as well as multiple places are to look for
> dependency
> > > > > > versions
> > > > > > > and
> > > > > > > > > > their
> > > > > > > > > > > > > updates.
> > > > > > > > > > > > >
> > > > > > > > > > > > > with conan, it may became much more
> straightforward,
> > as
> > > > > > > > > dependencies
> > > > > > > > > > will
> > > > > > > > > > > > > be declared in single place (conanfile) and
> installed
> > > via
> > > > > > > single
> > > > > > > > > > command
> > > > > > > > > > > > > (conan install).
> > > > > > > > > > > > >
> > > > > > > > > > > > > as project is very complex, and has lots of
> > > dependencies,
> > > > > for
> > > > > > > the
> > > > > > > > > > first
> > > > > > > > > > > > > prototype I've used only very few of dependencies
> > from
> > > > > conan:
> > > > > > > > > OpenCV
> > > > > > > > > > > > > <
> > > > https://bintray.com/conan-community/conan/opencv%3Aconan
> > > > > >,
> > > > > > > > > OpenBLAS
> > > > > > > > > > > > > <
> > > > > https://bintray.com/conan-community/conan/openblas%3Aconan>
> > > > > > > and
> > > > > > > > > > lapack
> > > > > > > > > > > > > <
> > > > https://bintray.com/conan-community/conan/lapack%3Aconan
> > > > > >.
> > > > > > > > > > > > > others may be easily added then one by one, but
> they
> > > > first
> > > > > > has
> > > > > > > to
> > > > > > > > > be
> > > > > > > > > > > > > packaged (not all of them are packaged yet, e.g.
> > > > GoogleTest
> > > > > > > > > > > > > <
> > > > > > >
> https://bintray.com/bincrafters/public-conan/gtest%3Abincrafters
> > >
> > > > > > > > > > is
> > > > > > > > > > > > > available, while MKL is not).
> > > > > > > > > > > > >
> > > > > > > > > > > > > I attach patch which adds an initial conan support
> as
> > > > proof
> > > > > > of
> > > > > > > > > > concept.
> > > > > > > > > > > > > also, I attach two simple build scripts, which I've
> > > used
> > > > to
> > > > > > > test
> > > > > > > > > (for
> > > > > > > > > > > > > Windows and Linux / Mac OS X). Google Mail blocks
> .sh
> > > and
> > > > > > .cmd
> > > > > > > > > > > > extensions,
> > > > > > > > > > > > > so you'll need to rename files.
> > > > > > > > > > > > > lemme know if you have any further questions.
> > > > > > > > > > > > >
> > > > > > > > > > > > > yours sincerely, Konstantin
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>