You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by Janardhan <ja...@apache.org> on 2021/05/05 00:00:00 UTC
[DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Hi all,
I am planning to add GPU docs like [1]. Essentially the document should
answer
the following questions. Please, feel free to add new points or contribute
to
the discussion. thanks.
1. What are the ways in which to run SystemDS with gpu?
For example,
with -gpu flag
> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar -f
test.dml -gpu
2. What are the CUDNN and CUDA versions supported?
For example, which SDK[2] version to download
3. About JCUDA, do we need to download anything or pom.xml will take care
of it?
4. How do the python users invoke gpu?
5. What is the default precision behaviour (single or double)?
6. Default memory allocation - Unified memory[3]?
For example, we use `cuda`[4]
--
[1] http://systemds.apache.org/docs/1.2.0/gpu
[2] https://developer.nvidia.com/cuda-downloads
[3]
https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
?
[4]
https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
[SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970
Thank you,
Janardhan
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by Mark Dokter <md...@know-center.at>.
On 14/05/2021 08:29, arnab phani wrote:
> GPU code generation and lineage-based reuse in GPU are still under
> development, and most probably will not be mentioned in the 2.1 release
> notes.
> Given that, the initial documentation from SystemML 1.2 should suffice for
> now.
> I will let Mark confirm that.
>
Yes, new features will be documented
* for users when they are mature enough to be used
* for developers when code is not in flux anymore.
> Regards,
> Arnab..
>
> On Fri, May 14, 2021 at 6:17 AM Janardhan <ja...@gmail.com>
> wrote:
>
>> Hi Mark,
>>
>> I have written down a GPU guide, with sufficient instructions[1]
>>
Thank you for your efforts!
>> In this doc, the python and spark-submit instructions are not yet provided.
>> We will mark this for the future.
>>
I see spark-submit and scala mentioned in that linked documentation web
page -> ?
>> As for the Developer guide is concerned, we need to document
>> internals of the GPU design too[2]. If the internals are not documented
>> explicitly, the feature might get dormant and difficult for other
>> contributors.
>>
>> 1. GPU prototyping
>> 2. Codegeneration
>> 3. Lineage feature
>> 4. Privacy feature if applicable.
>>
>> This process should not be treated as a counter productive one, but a
>> necessity.
>>
>> We could come to this discussion in the future. :)
>>
Yes, once we're there, we'll have discussion and documentation :)
>> [1] https://apache.github.io/systemds/site/gpu
>> [2]
>>
>> https://github.com/apache/systemds/blob/branch-1.2.0/docs/devdocs/gpu-backend.md#initial-prototype-for-gpu-backend
>>
>> Thank you,
>> Janardhan
>>
Thanks again, Janardhan, for your effort in documenting.
When I looked over the GPU Guide web page, I spotted a few things:
* our shipped SystemDS.ptx is compiled for compute capability 3.0 and up
iirc (the docs list "architecture 5.0...etc").
* ptx jit compilation can not be avoided as we don't have any cuda
binaries. The PTX get's compiled by the driver when loaded. That all
happens automatically - no nvcc is involved here.
* nvcc is used to get from cuda c++ code to ptx. That happens via cmake
at the moment. Manual invocation like in the docs is certainly possible.
Doing so requires putting the resulting PTX file in the appropriate
place for mvn package to find it of course.
* Having a quick start guide with all the necessary instructions is nice
but it's not our job to document how to install cuda imho. That gets
outdated quickly and yields unnecessary overhead (e.g. if we start
supporting CUDA 11, these instructions need to be changed).
* CUPTI is explicitly mentioned - why is that?
hth,
Mark
>> On Fri, May 7, 2021 at 3:42 AM Mark Dokter <md...@know-center.at> wrote:
>>
>>> On 05/05/2021 02:00, Janardhan wrote:
>>>> Hi all,
>>>>
>>>
>>> Hi! Thank you Janardhan, to take care of this.
>>>
>>>> I am planning to add GPU docs like [1]. Essentially the document should
>>>> answer
>>>> the following questions. Please, feel free to add new points or
>>> contribute
>>>> to
>>>> the discussion. thanks.
>>>>
>>>> 1. What are the ways in which to run SystemDS with gpu?
>>>> For example,
>>>> with -gpu flag
>>>>> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar
>>> -f
>>>> test.dml -gpu
>>>>
>>>
>>> Python is on the todo list now (see Sebastian Baunsgaard's response),
>>> JMLC should work if you set the config but I don't know if we have an
>>> example of that somewhere and for script execution from shell it's the
>>> -gpu flag. There's also a force option. About that spark-submit: Is
>>> running gpu instructions through spark supported (never tried it myself)?
>>>
>>>> 2. What are the CUDNN and CUDA versions supported?
>>>> For example, which SDK[2] version to download
>>>>
>>>
>>> At the moment we're tied to CUDA 10.2 and CUDNN 7.x due to API changes
>>> in CUDNN 8.x.
>>>
>>>
>>>> 3. About JCUDA, do we need to download anything or pom.xml will take
>> care
>>>> of it?
>>>>
>>>
>>> From source should be handled by mvn. If the binary release is used,
>>> everything that is needed should be in the extra jar.
>>>
>>>> 4. How do the python users invoke gpu?
>>>>
>>>
>>> See answer to 1. and Sebastians answer.
>>>
>>>> 5. What is the default precision behaviour (single or double)?
>>>>
>>>
>>> See Arnab's answer. The defaults can be viewed in the
>>> SystemDS-config.xml.template and in DMLConfig.java.
>>>
>>>> 6. Default memory allocation - Unified memory[3]?
>>>> For example, we use `cuda`[4]
>>>>
>>>
>>> See Arnab's response.
>>>
>>>> --
>>>> [1] http://systemds.apache.org/docs/1.2.0/gpu
>>>> [2] https://developer.nvidia.com/cuda-downloads
>>>> [3]
>>>>
>>>
>> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
>>>> ?
>>>> [4]
>>>>
>>>
>> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
>>>> [SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970
>>>>
>>>> Thank you,
>>>> Janardhan
>>>>
>>>
>>> hth, Mark
>>>
>>>
>>
>
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by arnab phani <ph...@gmail.com>.
GPU code generation and lineage-based reuse in GPU are still under
development, and most probably will not be mentioned in the 2.1 release
notes.
Given that, the initial documentation from SystemML 1.2 should suffice for
now.
I will let Mark confirm that.
Regards,
Arnab..
On Fri, May 14, 2021 at 6:17 AM Janardhan <ja...@gmail.com>
wrote:
> Hi Mark,
>
> I have written down a GPU guide, with sufficient instructions[1]
>
> In this doc, the python and spark-submit instructions are not yet provided.
> We will mark this for the future.
>
> As for the Developer guide is concerned, we need to document
> internals of the GPU design too[2]. If the internals are not documented
> explicitly, the feature might get dormant and difficult for other
> contributors.
>
> 1. GPU prototyping
> 2. Codegeneration
> 3. Lineage feature
> 4. Privacy feature if applicable.
>
> This process should not be treated as a counter productive one, but a
> necessity.
>
> We could come to this discussion in the future. :)
>
> [1] https://apache.github.io/systemds/site/gpu
> [2]
>
> https://github.com/apache/systemds/blob/branch-1.2.0/docs/devdocs/gpu-backend.md#initial-prototype-for-gpu-backend
>
> Thank you,
> Janardhan
>
> On Fri, May 7, 2021 at 3:42 AM Mark Dokter <md...@know-center.at> wrote:
>
> > On 05/05/2021 02:00, Janardhan wrote:
> > > Hi all,
> > >
> >
> > Hi! Thank you Janardhan, to take care of this.
> >
> > > I am planning to add GPU docs like [1]. Essentially the document should
> > > answer
> > > the following questions. Please, feel free to add new points or
> > contribute
> > > to
> > > the discussion. thanks.
> > >
> > > 1. What are the ways in which to run SystemDS with gpu?
> > > For example,
> > > with -gpu flag
> > >> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar
> > -f
> > > test.dml -gpu
> > >
> >
> > Python is on the todo list now (see Sebastian Baunsgaard's response),
> > JMLC should work if you set the config but I don't know if we have an
> > example of that somewhere and for script execution from shell it's the
> > -gpu flag. There's also a force option. About that spark-submit: Is
> > running gpu instructions through spark supported (never tried it myself)?
> >
> > > 2. What are the CUDNN and CUDA versions supported?
> > > For example, which SDK[2] version to download
> > >
> >
> > At the moment we're tied to CUDA 10.2 and CUDNN 7.x due to API changes
> > in CUDNN 8.x.
> >
> >
> > > 3. About JCUDA, do we need to download anything or pom.xml will take
> care
> > > of it?
> > >
> >
> > From source should be handled by mvn. If the binary release is used,
> > everything that is needed should be in the extra jar.
> >
> > > 4. How do the python users invoke gpu?
> > >
> >
> > See answer to 1. and Sebastians answer.
> >
> > > 5. What is the default precision behaviour (single or double)?
> > >
> >
> > See Arnab's answer. The defaults can be viewed in the
> > SystemDS-config.xml.template and in DMLConfig.java.
> >
> > > 6. Default memory allocation - Unified memory[3]?
> > > For example, we use `cuda`[4]
> > >
> >
> > See Arnab's response.
> >
> > > --
> > > [1] http://systemds.apache.org/docs/1.2.0/gpu
> > > [2] https://developer.nvidia.com/cuda-downloads
> > > [3]
> > >
> >
> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
> > > ?
> > > [4]
> > >
> >
> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
> > > [SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970
> > >
> > > Thank you,
> > > Janardhan
> > >
> >
> > hth, Mark
> >
> >
>
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by Janardhan <ja...@gmail.com>.
Hi Mark,
I have written down a GPU guide, with sufficient instructions[1]
In this doc, the python and spark-submit instructions are not yet provided.
We will mark this for the future.
As for the Developer guide is concerned, we need to document
internals of the GPU design too[2]. If the internals are not documented
explicitly, the feature might get dormant and difficult for other
contributors.
1. GPU prototyping
2. Codegeneration
3. Lineage feature
4. Privacy feature if applicable.
This process should not be treated as a counter productive one, but a
necessity.
We could come to this discussion in the future. :)
[1] https://apache.github.io/systemds/site/gpu
[2]
https://github.com/apache/systemds/blob/branch-1.2.0/docs/devdocs/gpu-backend.md#initial-prototype-for-gpu-backend
Thank you,
Janardhan
On Fri, May 7, 2021 at 3:42 AM Mark Dokter <md...@know-center.at> wrote:
> On 05/05/2021 02:00, Janardhan wrote:
> > Hi all,
> >
>
> Hi! Thank you Janardhan, to take care of this.
>
> > I am planning to add GPU docs like [1]. Essentially the document should
> > answer
> > the following questions. Please, feel free to add new points or
> contribute
> > to
> > the discussion. thanks.
> >
> > 1. What are the ways in which to run SystemDS with gpu?
> > For example,
> > with -gpu flag
> >> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar
> -f
> > test.dml -gpu
> >
>
> Python is on the todo list now (see Sebastian Baunsgaard's response),
> JMLC should work if you set the config but I don't know if we have an
> example of that somewhere and for script execution from shell it's the
> -gpu flag. There's also a force option. About that spark-submit: Is
> running gpu instructions through spark supported (never tried it myself)?
>
> > 2. What are the CUDNN and CUDA versions supported?
> > For example, which SDK[2] version to download
> >
>
> At the moment we're tied to CUDA 10.2 and CUDNN 7.x due to API changes
> in CUDNN 8.x.
>
>
> > 3. About JCUDA, do we need to download anything or pom.xml will take care
> > of it?
> >
>
> From source should be handled by mvn. If the binary release is used,
> everything that is needed should be in the extra jar.
>
> > 4. How do the python users invoke gpu?
> >
>
> See answer to 1. and Sebastians answer.
>
> > 5. What is the default precision behaviour (single or double)?
> >
>
> See Arnab's answer. The defaults can be viewed in the
> SystemDS-config.xml.template and in DMLConfig.java.
>
> > 6. Default memory allocation - Unified memory[3]?
> > For example, we use `cuda`[4]
> >
>
> See Arnab's response.
>
> > --
> > [1] http://systemds.apache.org/docs/1.2.0/gpu
> > [2] https://developer.nvidia.com/cuda-downloads
> > [3]
> >
> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
> > ?
> > [4]
> >
> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
> > [SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970
> >
> > Thank you,
> > Janardhan
> >
>
> hth, Mark
>
>
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by Mark Dokter <md...@know-center.at>.
On 05/05/2021 02:00, Janardhan wrote:
> Hi all,
>
Hi! Thank you Janardhan, to take care of this.
> I am planning to add GPU docs like [1]. Essentially the document should
> answer
> the following questions. Please, feel free to add new points or contribute
> to
> the discussion. thanks.
>
> 1. What are the ways in which to run SystemDS with gpu?
> For example,
> with -gpu flag
>> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar
-f
> test.dml -gpu
>
Python is on the todo list now (see Sebastian Baunsgaard's response),
JMLC should work if you set the config but I don't know if we have an
example of that somewhere and for script execution from shell it's the
-gpu flag. There's also a force option. About that spark-submit: Is
running gpu instructions through spark supported (never tried it myself)?
> 2. What are the CUDNN and CUDA versions supported?
> For example, which SDK[2] version to download
>
At the moment we're tied to CUDA 10.2 and CUDNN 7.x due to API changes
in CUDNN 8.x.
> 3. About JCUDA, do we need to download anything or pom.xml will take care
> of it?
>
From source should be handled by mvn. If the binary release is used,
everything that is needed should be in the extra jar.
> 4. How do the python users invoke gpu?
>
See answer to 1. and Sebastians answer.
> 5. What is the default precision behaviour (single or double)?
>
See Arnab's answer. The defaults can be viewed in the
SystemDS-config.xml.template and in DMLConfig.java.
> 6. Default memory allocation - Unified memory[3]?
> For example, we use `cuda`[4]
>
See Arnab's response.
> --
> [1] http://systemds.apache.org/docs/1.2.0/gpu
> [2] https://developer.nvidia.com/cuda-downloads
> [3]
> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
> ?
> [4]
> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
> [SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970
>
> Thank you,
> Janardhan
>
hth, Mark
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by "Baunsgaard, Sebastian" <ba...@tugraz.at.INVALID>.
based on that i can answer some as well
4. Python would simply need to add a gpu flag on SystemDSContext, that then specify to use gpu, this should not be complicated to add.
________________________________
From: arnab phani <ph...@gmail.com>
Sent: Thursday, May 6, 2021 11:53:46 AM
To: dev@systemds.apache.org
Subject: Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
I can answer a few of the questions. Mark may answer the rest.
1. Yes -gpu flag is the way to go. I am not aware of any other option.
3. NVIDIA CUDA Development Toolkit needs to be installed and set up as well
to be able to use gpu with SystemDS.
5. The default precision behavior is double.
6. The default memory allocator is regular cuda. SystemDS config provides
an option to use unified memory manager, however, I doubt how robust it is,
and even otherwise the default cuda allocator is guarded by our own gpu
memory manager, which is a better choice.
Regards,
Arnab..
On Wed, May 5, 2021 at 2:00 AM Janardhan <ja...@apache.org> wrote:
> Hi all,
>
> I am planning to add GPU docs like [1]. Essentially the document should
> answer
> the following questions. Please, feel free to add new points or contribute
> to
> the discussion. thanks.
>
> 1. What are the ways in which to run SystemDS with gpu?
> For example,
> with -gpu flag
> > spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar -f
> test.dml -gpu
>
> 2. What are the CUDNN and CUDA versions supported?
> For example, which SDK[2] version to download
>
> 3. About JCUDA, do we need to download anything or pom.xml will take care
> of it?
>
> 4. How do the python users invoke gpu?
>
> 5. What is the default precision behaviour (single or double)?
>
> 6. Default memory allocation - Unified memory[3]?
> For example, we use `cuda`[4]
>
> --
> [1] http://systemds.apache.org/docs/1.2.0/gpu
> [2] https://developer.nvidia.com/cuda-downloads
> [3]
>
> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
> ?
> [4]
>
> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
> [SYSTEMDS-2970
> <https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95[SYSTEMDS-2970>]
> https://issues.apache.org/jira/browse/SYSTEMDS-2970
>
> Thank you,
> Janardhan
>
Re: [DISCUSS][SYSTEMDS-2970] Using SystemDS with GPU docs
Posted by arnab phani <ph...@gmail.com>.
I can answer a few of the questions. Mark may answer the rest.
1. Yes -gpu flag is the way to go. I am not aware of any other option.
3. NVIDIA CUDA Development Toolkit needs to be installed and set up as well
to be able to use gpu with SystemDS.
5. The default precision behavior is double.
6. The default memory allocator is regular cuda. SystemDS config provides
an option to use unified memory manager, however, I doubt how robust it is,
and even otherwise the default cuda allocator is guarded by our own gpu
memory manager, which is a better choice.
Regards,
Arnab..
On Wed, May 5, 2021 at 2:00 AM Janardhan <ja...@apache.org> wrote:
> Hi all,
>
> I am planning to add GPU docs like [1]. Essentially the document should
> answer
> the following questions. Please, feel free to add new points or contribute
> to
> the discussion. thanks.
>
> 1. What are the ways in which to run SystemDS with gpu?
> For example,
> with -gpu flag
> > spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar -f
> test.dml -gpu
>
> 2. What are the CUDNN and CUDA versions supported?
> For example, which SDK[2] version to download
>
> 3. About JCUDA, do we need to download anything or pom.xml will take care
> of it?
>
> 4. How do the python users invoke gpu?
>
> 5. What is the default precision behaviour (single or double)?
>
> 6. Default memory allocation - Unified memory[3]?
> For example, we use `cuda`[4]
>
> --
> [1] http://systemds.apache.org/docs/1.2.0/gpu
> [2] https://developer.nvidia.com/cuda-downloads
> [3]
>
> https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory
> ?
> [4]
>
> https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95
> [SYSTEMDS-2970
> <https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95[SYSTEMDS-2970>]
> https://issues.apache.org/jira/browse/SYSTEMDS-2970
>
> Thank you,
> Janardhan
>