Access OpenCL-installation from inside AsterixDB-instance

[Torsten Bergh Moss <to...@ig.ntnu.no>]

I've made two versions of my Machine Learning-based UDF, one using vanilla Java, and one using Aparapi, a java framework for executing native java-code on the GPU, to speed up the classification-part of the algorithm.


Both run perfectly on the local environment on my machine (java/maven, clicking "run" inside VSCode), and the vanilla UDF works perfectly inside of AsterixDB, however, the GPU-based version can't seem to locate my machine's OpenCL-installation from inside of AsterixDB, and therefore defaults back to using the Java Thread Pool for paralellization (instead of the GPU), which quickly runs out of threads for this task.


Has anybody done/experiences anything similar, or could point me in the direction of which actions to take next?


Thanks for all the help guys, this is the last piece of the puzzle for the project.


Best wishes,

Torsten Bergh Moss

Re: Access OpenCL-installation from inside AsterixDB-instance

[Ian Maxon <im...@uci.edu>]

Interesting. Yeah it probably has to do with the library path. Is the
library installed system-wide or is VSCode appending something like
LD_PRELOAD=/foo/bar/opencl/ to the environment to get it on the path?

On Tue, Nov 19, 2019 at 3:30 PM Torsten Bergh Moss
<to...@ig.ntnu.no> wrote:
>
> Of course, just let me clean up the code up a bit before putting it in a repo.
>
> The whole error message is
>
> SEVERE: Check your environment. Failed to load codegen native library or possibly failed to locate opencl native library (opencl.dll/opencl.so). Ensure that OpenCL is in your PATH (windows) or in LD_LIBRARY_PATH (linux).
>
> Could it be that something has to be done with regards to the PATH-configuration for Asterix?
>
> Best wishes,
> Torsten
>
> ________________________________________
> From: Ian Maxon <im...@uci.edu>
> Sent: Tuesday, November 19, 2019 7:34 PM
> To: dev@asterixdb.apache.org
> Subject: Re: Access OpenCL-installation from inside AsterixDB-instance
>
> Can you share the code? What hints are you getting that it can't find
> the OpenCL install? There shouldn't be too much different, I assume it
> is probably an environment variable or system property that needs to
> be accommodated.
>
> On Tue, Nov 19, 2019 at 9:48 AM Torsten Bergh Moss
> <to...@ig.ntnu.no> wrote:
> >
> > I've made two versions of my Machine Learning-based UDF, one using vanilla Java, and one using Aparapi, a java framework for executing native java-code on the GPU, to speed up the classification-part of the algorithm.
> >
> >
> > Both run perfectly on the local environment on my machine (java/maven, clicking "run" inside VSCode), and the vanilla UDF works perfectly inside of AsterixDB, however, the GPU-based version can't seem to locate my machine's OpenCL-installation from inside of AsterixDB, and therefore defaults back to using the Java Thread Pool for paralellization (instead of the GPU), which quickly runs out of threads for this task.
> >
> >
> > Has anybody done/experiences anything similar, or could point me in the direction of which actions to take next?
> >
> >
> > Thanks for all the help guys, this is the last piece of the puzzle for the project.
> >
> >
> > Best wishes,
> >
> > Torsten Bergh Moss

Re: Access OpenCL-installation from inside AsterixDB-instance

[Torsten Bergh Moss <to...@ig.ntnu.no>]

Of course, just let me clean up the code up a bit before putting it in a repo.

The whole error message is 

SEVERE: Check your environment. Failed to load codegen native library or possibly failed to locate opencl native library (opencl.dll/opencl.so). Ensure that OpenCL is in your PATH (windows) or in LD_LIBRARY_PATH (linux).

Could it be that something has to be done with regards to the PATH-configuration for Asterix?

Best wishes,
Torsten

________________________________________
From: Ian Maxon <im...@uci.edu>
Sent: Tuesday, November 19, 2019 7:34 PM
To: dev@asterixdb.apache.org
Subject: Re: Access OpenCL-installation from inside AsterixDB-instance

Can you share the code? What hints are you getting that it can't find
the OpenCL install? There shouldn't be too much different, I assume it
is probably an environment variable or system property that needs to
be accommodated.

On Tue, Nov 19, 2019 at 9:48 AM Torsten Bergh Moss
<to...@ig.ntnu.no> wrote:
>
> I've made two versions of my Machine Learning-based UDF, one using vanilla Java, and one using Aparapi, a java framework for executing native java-code on the GPU, to speed up the classification-part of the algorithm.
>
>
> Both run perfectly on the local environment on my machine (java/maven, clicking "run" inside VSCode), and the vanilla UDF works perfectly inside of AsterixDB, however, the GPU-based version can't seem to locate my machine's OpenCL-installation from inside of AsterixDB, and therefore defaults back to using the Java Thread Pool for paralellization (instead of the GPU), which quickly runs out of threads for this task.
>
>
> Has anybody done/experiences anything similar, or could point me in the direction of which actions to take next?
>
>
> Thanks for all the help guys, this is the last piece of the puzzle for the project.
>
>
> Best wishes,
>
> Torsten Bergh Moss

Re: Access OpenCL-installation from inside AsterixDB-instance

[Ian Maxon <im...@uci.edu>]

Can you share the code? What hints are you getting that it can't find
the OpenCL install? There shouldn't be too much different, I assume it
is probably an environment variable or system property that needs to
be accommodated.

On Tue, Nov 19, 2019 at 9:48 AM Torsten Bergh Moss
<to...@ig.ntnu.no> wrote:
>
> I've made two versions of my Machine Learning-based UDF, one using vanilla Java, and one using Aparapi, a java framework for executing native java-code on the GPU, to speed up the classification-part of the algorithm.
>
>
> Both run perfectly on the local environment on my machine (java/maven, clicking "run" inside VSCode), and the vanilla UDF works perfectly inside of AsterixDB, however, the GPU-based version can't seem to locate my machine's OpenCL-installation from inside of AsterixDB, and therefore defaults back to using the Java Thread Pool for paralellization (instead of the GPU), which quickly runs out of threads for this task.
>
>
> Has anybody done/experiences anything similar, or could point me in the direction of which actions to take next?
>
>
> Thanks for all the help guys, this is the last piece of the puzzle for the project.
>
>
> Best wishes,
>
> Torsten Bergh Moss