[DISCUSS][SYSTEMDS-3051] Native Backend guide for SystemDS

[Janardhan <ja...@gmail.com>]

Hi all,

I am planning to add Native backend guide. The document should
answer the following questions. Please, feel free to add new points
or contribute to discussion. thanks.

User guide

1. BLAS Installation instructions

1a. Intel MKL
Download Intel MKL from [1].
Version of MKL (2017 and higher), do we use MKL DNN primitives?

1b. OpenBLAS
Compiling OpenBLAS with OpenMP is recommended instead of apt-get/yum.
instructions are at [2].

2. How to provide the location of native libraries?

via commandline
-  Java: -Djava.library.path=/path/to/blas-n-other-dependencies
- Spark: --conf
spark.executorEnv.LD_LIBRARY_PATH=/path/to/blas-n-other-dependencies

This can also be set with `sysds.native.blas.directory`

3. How to enable native BLAS?

set sysds.native.blas[3] to either mkl, openblas

Troubleshooting
 - 1. Unable to load gomp
 - 2. Unable to load mkl_rt
 - 3. Unable to load openblas
 - 4. Using OpenBLAS without OpenMP can lead to performance issues
 - 5. Using MKL can lead to slow performance for convolution instruction

---
[1]
https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html
[2] https://github.com/xianyi/OpenBLAS#installation-from-source
[3]
https://github.com/apache/systemds/blob/master/conf/SystemDS-config.xml.template#L61

Thank you,
Janardhan

Re: [DISCUSS][SYSTEMDS-3051] Native Backend guide for SystemDS

[Mark Dokter <md...@know-center.at>]

On 02.07.21 14:22, Janardhan wrote:
> Hi all,
> 

Hi!

> I am planning to add Native backend guide. The document should
> answer the following questions. Please, feel free to add new points
> or contribute to discussion. thanks.
> 
> User guide
> 
> 1. BLAS Installation instructions
> 
> 1a. Intel MKL
> Download Intel MKL from [1].
> Version of MKL (2017 and higher), do we use MKL DNN primitives?
> 

Highest supported version is 2019.5. Everything above has API changes
which are not supported yet by our native dnn ops.

> 1b. OpenBLAS
> Compiling OpenBLAS with OpenMP is recommended instead of apt-get/yum.
> instructions are at [2].
> 

At least in Ubuntu 20.04 you don't need to compile OpenBLAS manually for
OpenMP support. Just remove the libopenblas0-pthread package and install
libopenblas0-openmp instead.

> 2. How to provide the location of native libraries?
> 
> via commandline
> -  Java: -Djava.library.path=/path/to/blas-n-other-dependencies
> - Spark: --conf
> spark.executorEnv.LD_LIBRARY_PATH=/path/to/blas-n-other-dependencies
> 
> This can also be set with `sysds.native.blas.directory`
> 

For Intel MKL the preferred method is to source their env setup script
(path adjustments to that script according to your installation):
source /opt/intel/bin/compilervars.sh intel64

> 3. How to enable native BLAS?
> 
> set sysds.native.blas[3] to either mkl, openblas
> 
> Troubleshooting
>  - 1. Unable to load gomp
>  - 2. Unable to load mkl_rt
>  - 3. Unable to load openblas
>  - 4. Using OpenBLAS without OpenMP can lead to performance issues
>  - 5. Using MKL can lead to slow performance for convolution instruction
> 

On very old Linux distributions (we have CentOS 7.x around on an older
cluster) there can be issues with loading libs because the gcc ABI
changed from gcc 4.x to 5 and above. In that case you'd have to either
fiddle with the patchelf utility or just quickly (?) compile yourself a
recent gcc and adjust your env vars (PATH, LD_LIBRARY_PATH).

> ---
> [1]
> https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html
> [2] https://github.com/xianyi/OpenBLAS#installation-from-source
> [3]
> https://github.com/apache/systemds/blob/master/conf/SystemDS-config.xml.template#L61
> 
> Thank you,
> Janardhan
> 


hth, Mark