Re: Apache MXNet Scala nightly-build now available on Apache Nexus

[YiZhi Liu <li...@apache.org>]

Thanks for attention. Here's an example demonstrates how to configure
pom.xml to use the nightly build in your own project:

https://github.com/yzhliu/mxnet-scala-example/blob/master/pom.xml#L152-L163
On Mon, Oct 29, 2018 at 2:53 PM Carin Meier <ca...@gmail.com> wrote:
>
> Fantastic! This is going to be great for the Clojure MXNet users as well.
>
> On Mon, Oct 29, 2018 at 1:39 PM YiZhi Liu <li...@apache.org> wrote:
>
> > Hi,
> >
> >
> >
> > I am happy to announce that the availability of the experimental
> > nightly-build Scala package on Maven in Nexus. The nightly-builds,
> > currently published as 1.3.1-SNAPSHOT version, include the latest changes
> > from the master branch from apache/incubator-mxnet GitHub repo and refresh
> > every day. Furthermore, we completely overhauled the MXNet library shipped
> > inside and adopted the binary build logic used for Python pip, so now Scala
> > users can also enjoy the portable MXNet with fully loaded features.
> > Currently, the supported artifacts are:
> >
> > - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU)
> >
> > - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0)
> >
> > - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU)
> >
> >
> >
> > FAQ:
> >
> > - What's new in the nightly Scala packages compared to the previous Scala
> > Maven artifacts?
> >
> > Most notably, better usability, more supported features, and better
> > platform compatibility. The new packages now support Ubuntu 14.04 and
> > Amazon Linux and support the same broad set of features as the existing pip
> > packages. Besides, users are no longer required to install all the
> > dependencies to use the nightly Scala packages as the included MXNet
> > libraries are fully loaded. For the list of supported platforms and
> > features, see [1].
> >
> >
> >
> > - What's different in the build approach compared to what's in the previous
> > Scala Maven packages?
> >
> > The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the
> > same build commands that regular users would use when building from source
> > [2]. This naive approach limits the compatibility to only similar
> > environments as where the packages were built (which was Ubuntu 16.04 for
> > Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more
> > CentOS or Amazon Linux and may suffer the problem of mismatching of
> > dependency versions due to the difference in their environment.
> >
> > On the other hand, Python users have long been enjoying portable and fully
> > loaded MXNet packages from pip. The MXNet libraries in pip packages are
> > built in more controlled environments with compatibility in mind.
> > Dependencies for extended features are statically linked and masked so that
> > no additional steps are needed.
> >
> > Now, in the new Scala nightly packages, the same approach as pip is used
> > for building the MXNet library so that the Scala users can benefit from the
> > same great features.
> >
> >
> >
> > - How do I get the new nightly packages?
> >
> > You can find the packages on Apache (Snapshots) repository [3] including
> > pom files and jars, which you can download and add to your projects.
> > Alternatively, you can configure your pom.xml to use the Maven repository.
> > Remember to add repository config:
> >
> > <repositories>
> >
> >   <repository>
> >
> >     <id>Apache Snapshot</id>
> >
> >     <url>https://repository.apache.org/content/groups/snapshots</url>
> >
> >   </repository>
> >
> > </repositories>
> >
> >
> >
> > - What's the support level of the Scala nightly packages?
> >
> > This package is currently experimental, so use at your own risk. While the
> > first nightly packages are already manually verified on several platforms,
> > and the automated publish pipeline runs unit tests and integration tests on
> > OSX and Ubuntu 14.04 before publishing, the testing is currently limited.
> >
> >
> >
> > - How can I help?
> >
> > We call on all MXNet Scala community to test out the new nightly packages.
> >
> >
> >
> > - I found a problem in using the package. What should I do?
> >
> > Please report this in the Github issue in [1].
> >
> >
> >
> > - What's next?
> >
> > There are several more things to be done:
> >
> > 1. We are adding more comprehensive support for GPU versions for different
> > CUDA versions.
> >
> > 2. We are adding more tests on more platforms. We are currently examining
> > the MXNet's Jenkins CI solution for supporting more platforms.
> >
> > 3. As we recognize that the build logic benefits the community, we plan to
> > open source the build scripts in the mxnet main repo. See progress at [5].
> >
> >
> >
> > Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this
> > happen. Thanks to Sheng Zha for sharing the pip build logic, and for
> > building the automated publishing pipelines. Special thanks to Qing and
> > Zach for sharing their expertise in Scala with Sheng and guiding and
> > supporting him locally for this milestone.
> >
> >
> >
> > [1] https://github.com/apache/incubator-mxnet/issues/8671
> >
> > [2]
> >
> > https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105
> >
> > [3]
> >
> > https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~
> >
> > [4] https://maven.apache.org/repository-management.html
> >
> > [5] https://github.com/apache/incubator-mxnet/projects/13
> >



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada