You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by Mark Dokter <md...@know-center.at> on 2020/11/02 08:10:40 UTC
Git submodules
Hi!
In my latest changeset about CUDA codegen [1] I will introduce a git
submodule to incorporate the external dependency jitify [2].
The interested but unaware of the submodule functionality of git, may
read up on it in the docs [3].
Apart from sneaking in advertising about my PR (will be merged Wednesday
EOD) I want to suggest making our code base more modular and use the
submodule feature for example on documentation and binaries.
Discuss.
Regards, Mark
[1] https://github.com/apache/systemds/pull/1092
[2] https://github.com/NVIDIA/jitify
[3] https://git-scm.com/book/en/v2/Git-Tools-Submodules
Re: Git submodules
Posted by Mark Dokter <md...@know-center.at>.
On 15/11/2020 01:02, Mark Dokter wrote:
>
>
> On 04/11/2020 12:25, Janardhan wrote:
>> Hi dev,
>>
>
> Hi!
>
Hello again.
A small follow up on this. Better late than never ;-)
>>> Should we introduce git submodules?
>>
>> The git submodules are normally not favoured by the git community[1][2].
>> A while ago, I tried to bring submodule functionality to `git`&`libgit2`
>> projects.
>>
>
> Thank you for the suggestion! I was not aware of the subtree
> functionality. I might try that :)
>
I had a go at git subtree and have decided against it as rebasing
becomes troublesome. When I tried rebasing after adding the external
dependency as a git subtree, git wanted to strip away the prefix
directory of the subtree, resulting in conflicts (you'd have to manually
resolve conflicts like Readme.md because everything was moved from
src/main/cuda/ext/jitify to the root of the repository).
There is of course a non trivial workaround for that, but I don't think
it's feasible to expect everybody to be aware of it.
The result is that we stay with a submodule (which I updated).
>> But, subtree is an option worth looking into. Since `jitify.hpp` can
>> just be
>> committed directly as a file or as a subtree. BSD-3 code can be used
>> Apache
>> (needs to be confirmed).
>>
>
> I actually did not consider legal issues when I started to use jitify.
> In that regard it might be better to stay with submodules for external
> references, so we don't ship them but the users have to pull them in by
> initializing the submodules themselves.
>
I guess we just need to make sure we include the license file in binary
releases.
Regards,
Mark
Re: Git submodules
Posted by Mark Dokter <md...@know-center.at>.
On 04/11/2020 12:25, Janardhan wrote:
> Hi dev,
>
Hi!
>> Should we introduce git submodules?
>
> The git submodules are normally not favoured by the git community[1][2].
> A while ago, I tried to bring submodule functionality to `git`&`libgit2`
> projects.
>
Thank you for the suggestion! I was not aware of the subtree
functionality. I might try that :)
> But, subtree is an option worth looking into. Since `jitify.hpp` can just be
> committed directly as a file or as a subtree. BSD-3 code can be used Apache
> (needs to be confirmed).
>
I actually did not consider legal issues when I started to use jitify.
In that regard it might be better to stay with submodules for external
references, so we don't ship them but the users have to pull them in by
initializing the submodules themselves.
>> Should we keep everything separate?
>
> Monorepo works fine, if we take sufficient care with commits.
>
Yes of course. The intention was to take some pressure off the
increasing repo size in terms of megabytes transferred and also reduce
the growth rate of our commit history.
>
> [1] https://github.com/git/git/pull/765#issuecomment-617749098
> [2] https://github.com/libgit2/libgit2/pull/5495#issuecomment-618989977
>
> Regards,
> Janardhan
>
Best regards,
Mark
> On Mon, Nov 2, 2020 at 5:12 PM Baunsgaard, Sebastian
> <ba...@tugraz.at.invalid> wrote:
>
>> Hi,
>>
>>
>> In general I'm a fan of separating it, if it is not to complicated and I'm
>> still able to find my files without the hassle of switching branches to see
>> the docs.
>>
>> I'm the one who pulled the docs back to master for SystemDS, because it
>> was put in a branch that no one ever looked at hopefully this will serve
>> well, if we decide to switch.
>>
>>
>> br
>>
>> Sebastian
>>
>> ________________________________
>> From: Mark Dokter <md...@know-center.at>
>> Sent: Monday, November 2, 2020 9:10:40 AM
>> To: dev@systemds.apache.org
>> Subject: Git submodules
>>
>> Hi!
>>
>> In my latest changeset about CUDA codegen [1] I will introduce a git
>> submodule to incorporate the external dependency jitify [2].
>> The interested but unaware of the submodule functionality of git, may
>> read up on it in the docs [3].
>> Apart from sneaking in advertising about my PR (will be merged Wednesday
>> EOD) I want to suggest making our code base more modular and use the
>> submodule feature for example on documentation and binaries.
>>
>> Discuss.
>>
>> Regards, Mark
>>
>> [1] https://github.com/apache/systemds/pull/1092
>> [2] https://github.com/NVIDIA/jitify
>> [3] https://git-scm.com/book/en/v2/Git-Tools-Submodules
>>
>>
>
--
Dipl.-Ing. Dr.techn. Mark Dokter, BSc
http://know-center.tugraz.at
Inffeldgasse 13, 8010 Graz
Tel. +43 316 873 30804
Re: Git submodules
Posted by Janardhan <ja...@apache.org>.
Hi dev,
> Should we introduce git submodules?
The git submodules are normally not favoured by the git community[1][2].
A while ago, I tried to bring submodule functionality to `git`&`libgit2`
projects.
But, subtree is an option worth looking into. Since `jitify.hpp` can just be
committed directly as a file or as a subtree. BSD-3 code can be used Apache
(needs to be confirmed).
> Should we keep everything separate?
Monorepo works fine, if we take sufficient care with commits.
[1] https://github.com/git/git/pull/765#issuecomment-617749098
[2] https://github.com/libgit2/libgit2/pull/5495#issuecomment-618989977
Regards,
Janardhan
On Mon, Nov 2, 2020 at 5:12 PM Baunsgaard, Sebastian
<ba...@tugraz.at.invalid> wrote:
> Hi,
>
>
> In general I'm a fan of separating it, if it is not to complicated and I'm
> still able to find my files without the hassle of switching branches to see
> the docs.
>
> I'm the one who pulled the docs back to master for SystemDS, because it
> was put in a branch that no one ever looked at hopefully this will serve
> well, if we decide to switch.
>
>
> br
>
> Sebastian
>
> ________________________________
> From: Mark Dokter <md...@know-center.at>
> Sent: Monday, November 2, 2020 9:10:40 AM
> To: dev@systemds.apache.org
> Subject: Git submodules
>
> Hi!
>
> In my latest changeset about CUDA codegen [1] I will introduce a git
> submodule to incorporate the external dependency jitify [2].
> The interested but unaware of the submodule functionality of git, may
> read up on it in the docs [3].
> Apart from sneaking in advertising about my PR (will be merged Wednesday
> EOD) I want to suggest making our code base more modular and use the
> submodule feature for example on documentation and binaries.
>
> Discuss.
>
> Regards, Mark
>
> [1] https://github.com/apache/systemds/pull/1092
> [2] https://github.com/NVIDIA/jitify
> [3] https://git-scm.com/book/en/v2/Git-Tools-Submodules
>
>
Re: Git submodules
Posted by "Baunsgaard, Sebastian" <ba...@tugraz.at.INVALID>.
Hi,
In general I'm a fan of separating it, if it is not to complicated and I'm still able to find my files without the hassle of switching branches to see the docs.
I'm the one who pulled the docs back to master for SystemDS, because it was put in a branch that no one ever looked at hopefully this will serve well, if we decide to switch.
br
Sebastian
________________________________
From: Mark Dokter <md...@know-center.at>
Sent: Monday, November 2, 2020 9:10:40 AM
To: dev@systemds.apache.org
Subject: Git submodules
Hi!
In my latest changeset about CUDA codegen [1] I will introduce a git
submodule to incorporate the external dependency jitify [2].
The interested but unaware of the submodule functionality of git, may
read up on it in the docs [3].
Apart from sneaking in advertising about my PR (will be merged Wednesday
EOD) I want to suggest making our code base more modular and use the
submodule feature for example on documentation and binaries.
Discuss.
Regards, Mark
[1] https://github.com/apache/systemds/pull/1092
[2] https://github.com/NVIDIA/jitify
[3] https://git-scm.com/book/en/v2/Git-Tools-Submodules