You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Luciano Resende <lu...@gmail.com> on 2016/02/09 20:11:12 UTC
Project folder structure
I would like to propose we follow more closely the project structure Spark
is using, where we clearly define a folder for samples, another for dev
utilities, etc instead of getting everything buried inside the /scripts
folder where it might be confusing/hard for users to find what they are
looking for.
Also, I would like to suggest creating a README.md on some of these folders
(e.g. samples) that describes what the contents of the folder are.
Thoughts ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Niketan Pansare <np...@us.ibm.com>.
I would advise not making significant project structure changes unless
absolutely required. Regarding release.sh, I think we can find a good place
for it without a major overhaul of the project structure. I don't have a
strong opinion on any particular location (be it dev or scripts or
something else) and am fine with the option you decide to go with :)
Thanks,
Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 02/09/2016 01:42 PM
Subject: Re: Project folder structure
On Tue, Feb 9, 2016 at 1:25 PM, Niketan Pansare <np...@us.ibm.com> wrote:
> -1 for changing the folder structure. I don't think the current setup is
> confusing for the end user and also agree with Matthias that our project
> structure does not have to be in sync with Spark.
> +1 for having README on user-facing folders like script or samples (which
> might demo JMLC/MLContext/MLPipeline/Python MLContext).
>
>
So, you are saying that moving
https://github.com/apache/incubator-systemml/blob/master/dev/release/release-build.sh
to scripts, will be very user friendly, particularly with all other ml
algorithms already there ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Luciano Resende <lu...@gmail.com>.
On Tue, Feb 9, 2016 at 1:25 PM, Niketan Pansare <np...@us.ibm.com> wrote:
> -1 for changing the folder structure. I don't think the current setup is
> confusing for the end user and also agree with Matthias that our project
> structure does not have to be in sync with Spark.
> +1 for having README on user-facing folders like script or samples (which
> might demo JMLC/MLContext/MLPipeline/Python MLContext).
>
>
So, you are saying that moving
https://github.com/apache/incubator-systemml/blob/master/dev/release/release-build.sh
to scripts, will be very user friendly, particularly with all other ml
algorithms already there ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Niketan Pansare <np...@us.ibm.com>.
-1 for changing the folder structure. I don't think the current setup is
confusing for the end user and also agree with Matthias that our project
structure does not have to be in sync with Spark.
+1 for having README on user-facing folders like script or samples (which
might demo JMLC/MLContext/MLPipeline/Python MLContext).
Thanks,
Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
From: Matthias Boehm/Almaden/IBM@IBMUS
To: dev@systemml.incubator.apache.org
Date: 02/09/2016 01:07 PM
Subject: Re: Project folder structure
-1
I don't see a compelling argument for this unnecessary change to a more
complex project structure just to follow Spark which is not directly
comparable - both in project size and content. For example, our algorithms
are at the same time a library of algorithms as well as samples for how to
write new algorithms. From my perspective, our major goal should be
"simplicity via minimality" not "simplicity via common structure" because
the latter would always require us to stay in sync.
Regards,
Matthias
Luciano Resende ---02/09/2016 11:11:24 AM---I would like to propose we
follow more closely the project structure Spark is using, where we clearl
From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 02/09/2016 11:11 AM
Subject: Project folder structure
I would like to propose we follow more closely the project structure Spark
is using, where we clearly define a folder for samples, another for dev
utilities, etc instead of getting everything buried inside the /scripts
folder where it might be confusing/hard for users to find what they are
looking for.
Also, I would like to suggest creating a README.md on some of these folders
(e.g. samples) that describes what the contents of the folder are.
Thoughts ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Deron Eriksson <de...@gmail.com>.
With regards to DML and PyDML, the test scripts are in src/test/scripts, so
something similar in main (src/main/scripts) may have some good points to
it. I've used Maven for a long time so personally I'm biased towards
Maven's "convention over configuration" paradigm. However, at the same time
it is nice to have the DML/PyDML scripts available at the root.
I'm a bit confused by the dev folder at the root of the project. If it only
contains a single script (dev/release/release-build.sh), maybe dev needs to
be deleted and the script needs to move elsewhere (the root of the project
itself or into another directory, but it doesn't seem like it would be a
good fit in the scripts directory with the DML). Is there some place under
src/ for release-build.sh? Would it fall under src/main/resources?
Somewhere else?
Perhaps we need some terminology to distinguish between DML/PyDML 'scripts'
and other 'scripts' such as release-build.sh.
Deron
On Tue, Feb 9, 2016 at 1:21 PM, Luciano Resende <lu...@gmail.com>
wrote:
> On Tue, Feb 9, 2016 at 12:43 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> > -1
> >
> > I don't see a compelling argument for this unnecessary change to a more
> > complex project structure just to follow Spark which is not directly
> > comparable - both in project size and content. For example, our
> algorithms
> > are at the same time a library of algorithms as well as samples for how
> to
> > write new algorithms. From my perspective, our major goal should be
> > "simplicity via minimality" not "simplicity via common structure" because
> > the latter would always require us to stay in sync.
> >
> > Regards,
> > Matthias
> >
> >
> I just don't see why it would make sense to add "notebooks" and "bash
> release scripts" all inside scripts which to me is currently filled with ML
> Algorithms in different stages or for different purposes.
>
> I am not keen on "simplicity via minimality" neither "simplicity via common
> structure"... I am keen on what makes sense I (and thus drive adoption) for
> someone that is first trying to look trough SystemML code, particularly
> folks that are already used to some best practices or with some other
> projects on the same area.
>
> --
> Luciano Resende
> http://people.apache.org/~lresende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
Re: Project folder structure
Posted by Luciano Resende <lu...@gmail.com>.
On Tue, Feb 9, 2016 at 12:43 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
> -1
>
> I don't see a compelling argument for this unnecessary change to a more
> complex project structure just to follow Spark which is not directly
> comparable - both in project size and content. For example, our algorithms
> are at the same time a library of algorithms as well as samples for how to
> write new algorithms. From my perspective, our major goal should be
> "simplicity via minimality" not "simplicity via common structure" because
> the latter would always require us to stay in sync.
>
> Regards,
> Matthias
>
>
I just don't see why it would make sense to add "notebooks" and "bash
release scripts" all inside scripts which to me is currently filled with ML
Algorithms in different stages or for different purposes.
I am not keen on "simplicity via minimality" neither "simplicity via common
structure"... I am keen on what makes sense I (and thus drive adoption) for
someone that is first trying to look trough SystemML code, particularly
folks that are already used to some best practices or with some other
projects on the same area.
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Matthias Boehm <mb...@us.ibm.com>.
-1
I don't see a compelling argument for this unnecessary change to a more
complex project structure just to follow Spark which is not directly
comparable - both in project size and content. For example, our algorithms
are at the same time a library of algorithms as well as samples for how to
write new algorithms. From my perspective, our major goal should be
"simplicity via minimality" not "simplicity via common structure" because
the latter would always require us to stay in sync.
Regards,
Matthias
From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 02/09/2016 11:11 AM
Subject: Project folder structure
I would like to propose we follow more closely the project structure Spark
is using, where we clearly define a folder for samples, another for dev
utilities, etc instead of getting everything buried inside the /scripts
folder where it might be confusing/hard for users to find what they are
looking for.
Also, I would like to suggest creating a README.md on some of these folders
(e.g. samples) that describes what the contents of the folder are.
Thoughts ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Project folder structure
Posted by Frederick R Reiss <fr...@us.ibm.com>.
Sounds like a good idea to me. In the spirit of Maven, the algorithms
really should go under src/main/dml and src/main/pydml. Perhaps we should
add some deep links to the top-level readme.md for important files that we
need to keep in nested directories?
Fred
From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 02/09/2016 11:11 AM
Subject: Project folder structure
I would like to propose we follow more closely the project structure Spark
is using, where we clearly define a folder for samples, another for dev
utilities, etc instead of getting everything buried inside the /scripts
folder where it might be confusing/hard for users to find what they are
looking for.
Also, I would like to suggest creating a README.md on some of these folders
(e.g. samples) that describes what the contents of the folder are.
Thoughts ?
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/