You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@predictionio.apache.org by takako shimamoto <ch...@gmail.com> on 2018/10/09 10:09:44 UTC
Re: [DISCUSS] Improve maintenance of the official templates
We might as well fold them into one single repo. When running git
clone, every template/example would be pulled down by default. At this
rate, it's too expensive to use our poor resource to maintain.
On Mon, Oct 1, 2018 at 7:32 AM Naoki Takezoe <ta...@gmail.com> wrote:
>
> I think that resurrection of "pio template" might be one idea.
>
> It would allow us flexible repository layout and also it can rewrite
> PredictionIO and Spark versions in build.sbt in templates at setup
> time. Though this isn't a standard way (e.g. "git clone" or "sbt
> new"), I think it both users and developers would be happy.
>
> 2018年9月25日(火) 17:28 Naoki Takezoe <ta...@gmail.com>:
> >
> > Yes. Folding templates into a single repository is ideal except that
> > users won't able to be use "git clone" to clone a individual template.
> > "svn" command supports it but using "svn" is not good idea. If we can
> > revive "pio template", we can implement partial download of git
> > repository in it. Anyway, still I don't have any solution without
> > problem.
> >
> > I created the following repository as a temporary solution personally.
> > There are some helper scripts for maintaining the official templates,
> > but this is obviously not a permanent solution.
> > https://github.com/takezoe/predictionio-templates-maintenance
> >
> > 2018年9月25日(火) 2:45 Donald Szeto <do...@apache.org>:
> > >
> > > How about folding them into one single repo with each template/examples as
> > > subprojects? Since we have retired the use of `pio template` command for a
> > > while, there is not really a hard requirement that each template has to be
> > > a git repo by itself.
> > >
> > > On Fri, Sep 21, 2018 at 10:59 PM takako shimamoto <ch...@gmail.com>
> > > wrote:
> > >
> > > > I don’t think giter8 is the best solution. There's room for
> > > > improvement. To improve workability in maintaining 7 official
> > > > templates and 12 examples, it's a good idea to put them together into
> > > > a repository. Isn't that just one of the options as a solution to
> > > > PIO-128?
> > > >
> > > > On Sat, Sep 22, 2018 at 9:10 AM Naoki Takezoe <ta...@gmail.com> wrote:
> > > > >
> > > > > Hi Donald,
> > > > >
> > > > > Thanks for your reply!
> > > > >
> > > > > > I think Giter8 is a nice idea, but my concern with it is its tight
> > > > coupling with sbt.
> > > > >
> > > > > Yes, giter8 templates changes user experience largely. This is my
> > > > > concern about it.
> > > > >
> > > > > > How about creating scripts that would make managing these templates
> > > > easier, despite they are in separate repos?
> > > > >
> > > > > Separated repositories are making situiation difficult. If all of them
> > > > > in a single repository, we can handle them easy. For example, we can
> > > > > commit and push modification for all templates at once. Even if
> > > > > repositories are separated, of course, it might be possible to create
> > > > > a script to automate these operation. But where should we put this
> > > > > script? But there is another problem. I don't know how users can clone
> > > > > a certain template if all templates are in a single repository.
> > > > >
> > > > > Also, we have to maintain examples in the PredictionIO repository.
> > > > > Currently, rhere are 12 example templates and need to be synchronized
> > > > > when original templates are updated.
> > > > >
> > > > > 2018年9月22日(土) 8:45 Donald Szeto <do...@apache.org>:
> > > > > >
> > > > > > Thanks for bringing this up, Naoki!
> > > > > >
> > > > > > I think Giter8 is a nice idea, but my concern with it is its tight
> > > > coupling
> > > > > > with sbt. It requires users to have access to sbt in their terminal
> > > > instead
> > > > > > of a simple git clone. How about creating scripts that would make
> > > > managing
> > > > > > these templates easier, despite they are in separate repos? I was
> > > > hoping
> > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-128 would help
> > > > with
> > > > > > that.
> > > > > >
> > > > > > A counter-argument to the above would be due to
> > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-81, users will
> > > > need
> > > > > > to have access to sbt to build official templates anyway (assuming we
> > > > > > continue using sbt as the build tool for official templates).
> > > > > >
> > > > > > I agree the quickstart can be moved to where the template is though.
> > > > > >
> > > > > > On Fri, Sep 21, 2018 at 1:02 AM Naoki Takezoe <ta...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I'd like to discuss about maintenance of the official templates.
> > > > > > >
> > > > > > > I updated all official templates for PredictionIO 0.13.0. There are 7
> > > > > > > templates for now and I think it's too expensive to use our poor
> > > > > > > resource to maintain. So I'd like to propose something to reduce
> > > > > > > maintenance cost of these templates.
> > > > > > >
> > > > > > > For example, porting them to giter8 templates is one idea. Takako has
> > > > > > > worked on it before
> > > > > > > (https://github.com/shimamoto/predictionio-template.g8). It makes
> > > > > > > possible to aggregate multiple templates to a single project, and
> > > > also
> > > > > > > help to reduce duplication. or at least, we might be able to
> > > > aggregate
> > > > > > > them to a single project simply.
> > > > > > >
> > > > > > > Further, we need to maintain the quickstart document in the
> > > > > > > PredictionIO repository as well. I wonder if we can move these docs
> > > > > > > (and examples as well?) to corresponding template repositories. It
> > > > > > > would make the synchronization of templates and docs much easier.
> > > > > > >
> > > > > > > Anyway, every release of PredictionIO causes such extra and
> > > > > > > unessential costs. I really want to reduce them. It would help to
> > > > make
> > > > > > > more frequent development and releases in the future. I'd like to
> > > > hear
> > > > > > > your opinion or ideas.
> > > > > > >
> > > > > > > --
> > > > > > > Naoki Takezoe
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Naoki Takezoe
> > > >
> >
> >
> >
> > --
> > Naoki Takezoe
>
>
>
> --
> Naoki Takezoe
Re: [DISCUSS] Improve maintenance of the official templates
Posted by Naoki Takezoe <ta...@gmail.com>.
Hi all,
Since there was no reaction to creating new repository to aggregate
the official templates I'm considering about incorporating the
official templates with main repository as suggested by Donald. My
work in progress pull request is here:
https://github.com/apache/predictionio/pull/500
My idea is not only incorporating the official templates to main
repository, but including the official templates and examples in the
distribution. Users can refer these examples or use templates instead
of cloning from GitHub. This will release us from trouble of templates
versioning.
Anyway, please take a look my pull request and let me know your thoughts.
.
On Sat, Oct 13, 2018 at 1:34 AM Naoki Takezoe <ta...@gmail.com> wrote:
>
> On Fri, Oct 12, 2018 at 9:06 AM Donald Szeto <do...@apache.org> wrote:
> >
> > That’s a pretty radical idea. It may send a wrong message of abandoning
> > templates though.
>
> Yes. In fact, I don't want to do it.
>
> > How about incorporating them into the main codebase as part of examples? We
> > can write tests around them to make sure they work with releases.
>
> Incorporating them into the main codebase is not good idea because
> some templates include data to be imported. The data is not big, but
> not much small either. It would make our core repository heavy.
>
> I think that creating a new repository and fold official templates
> there is better.
>
> --
> Naoki Takezoe
--
Naoki Takezoe
Re: [DISCUSS] Improve maintenance of the official templates
Posted by Naoki Takezoe <ta...@gmail.com>.
On Fri, Oct 12, 2018 at 9:06 AM Donald Szeto <do...@apache.org> wrote:
>
> That’s a pretty radical idea. It may send a wrong message of abandoning
> templates though.
Yes. In fact, I don't want to do it.
> How about incorporating them into the main codebase as part of examples? We
> can write tests around them to make sure they work with releases.
Incorporating them into the main codebase is not good idea because
some templates include data to be imported. The data is not big, but
not much small either. It would make our core repository heavy.
I think that creating a new repository and fold official templates
there is better.
--
Naoki Takezoe
Re: [DISCUSS] Improve maintenance of the official templates
Posted by Donald Szeto <do...@apache.org>.
That’s a pretty radical idea. It may send a wrong message of abandoning
templates though.
I am not a big fan of the template command. We got rid of it because it
caused more problems than being user-friendly.
How about incorporating them into the main codebase as part of examples? We
can write tests around them to make sure they work with releases.
On Thu, Oct 11, 2018 at 12:36 AM Naoki Takezoe <ta...@gmail.com> wrote:
> An another idea is making templates unofficial. It means moving
> template repositories to the outside of ASF and we don't guarantee
> maintenance for them.
>
> 2018年10月9日(火) 19:10 takako shimamoto <ch...@gmail.com>:
> >
> > We might as well fold them into one single repo. When running git
> > clone, every template/example would be pulled down by default. At this
> > rate, it's too expensive to use our poor resource to maintain.
> >
> > On Mon, Oct 1, 2018 at 7:32 AM Naoki Takezoe <ta...@gmail.com> wrote:
> > >
> > > I think that resurrection of "pio template" might be one idea.
> > >
> > > It would allow us flexible repository layout and also it can rewrite
> > > PredictionIO and Spark versions in build.sbt in templates at setup
> > > time. Though this isn't a standard way (e.g. "git clone" or "sbt
> > > new"), I think it both users and developers would be happy.
> > >
> > > 2018年9月25日(火) 17:28 Naoki Takezoe <ta...@gmail.com>:
> > > >
> > > > Yes. Folding templates into a single repository is ideal except that
> > > > users won't able to be use "git clone" to clone a individual
> template.
> > > > "svn" command supports it but using "svn" is not good idea. If we can
> > > > revive "pio template", we can implement partial download of git
> > > > repository in it. Anyway, still I don't have any solution without
> > > > problem.
> > > >
> > > > I created the following repository as a temporary solution
> personally.
> > > > There are some helper scripts for maintaining the official templates,
> > > > but this is obviously not a permanent solution.
> > > > https://github.com/takezoe/predictionio-templates-maintenance
> > > >
> > > > 2018年9月25日(火) 2:45 Donald Szeto <do...@apache.org>:
> > > > >
> > > > > How about folding them into one single repo with each
> template/examples as
> > > > > subprojects? Since we have retired the use of `pio template`
> command for a
> > > > > while, there is not really a hard requirement that each template
> has to be
> > > > > a git repo by itself.
> > > > >
> > > > > On Fri, Sep 21, 2018 at 10:59 PM takako shimamoto <
> chibochibo@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I don’t think giter8 is the best solution. There's room for
> > > > > > improvement. To improve workability in maintaining 7 official
> > > > > > templates and 12 examples, it's a good idea to put them together
> into
> > > > > > a repository. Isn't that just one of the options as a solution to
> > > > > > PIO-128?
> > > > > >
> > > > > > On Sat, Sep 22, 2018 at 9:10 AM Naoki Takezoe <ta...@gmail.com>
> wrote:
> > > > > > >
> > > > > > > Hi Donald,
> > > > > > >
> > > > > > > Thanks for your reply!
> > > > > > >
> > > > > > > > I think Giter8 is a nice idea, but my concern with it is its
> tight
> > > > > > coupling with sbt.
> > > > > > >
> > > > > > > Yes, giter8 templates changes user experience largely. This is
> my
> > > > > > > concern about it.
> > > > > > >
> > > > > > > > How about creating scripts that would make managing these
> templates
> > > > > > easier, despite they are in separate repos?
> > > > > > >
> > > > > > > Separated repositories are making situiation difficult. If all
> of them
> > > > > > > in a single repository, we can handle them easy. For example,
> we can
> > > > > > > commit and push modification for all templates at once. Even if
> > > > > > > repositories are separated, of course, it might be possible to
> create
> > > > > > > a script to automate these operation. But where should we put
> this
> > > > > > > script? But there is another problem. I don't know how users
> can clone
> > > > > > > a certain template if all templates are in a single repository.
> > > > > > >
> > > > > > > Also, we have to maintain examples in the PredictionIO
> repository.
> > > > > > > Currently, rhere are 12 example templates and need to be
> synchronized
> > > > > > > when original templates are updated.
> > > > > > >
> > > > > > > 2018年9月22日(土) 8:45 Donald Szeto <do...@apache.org>:
> > > > > > > >
> > > > > > > > Thanks for bringing this up, Naoki!
> > > > > > > >
> > > > > > > > I think Giter8 is a nice idea, but my concern with it is its
> tight
> > > > > > coupling
> > > > > > > > with sbt. It requires users to have access to sbt in their
> terminal
> > > > > > instead
> > > > > > > > of a simple git clone. How about creating scripts that would
> make
> > > > > > managing
> > > > > > > > these templates easier, despite they are in separate repos?
> I was
> > > > > > hoping
> > > > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-128
> would help
> > > > > > with
> > > > > > > > that.
> > > > > > > >
> > > > > > > > A counter-argument to the above would be due to
> > > > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-81,
> users will
> > > > > > need
> > > > > > > > to have access to sbt to build official templates anyway
> (assuming we
> > > > > > > > continue using sbt as the build tool for official templates).
> > > > > > > >
> > > > > > > > I agree the quickstart can be moved to where the template is
> though.
> > > > > > > >
> > > > > > > > On Fri, Sep 21, 2018 at 1:02 AM Naoki Takezoe <
> takezoe@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I'd like to discuss about maintenance of the official
> templates.
> > > > > > > > >
> > > > > > > > > I updated all official templates for PredictionIO 0.13.0.
> There are 7
> > > > > > > > > templates for now and I think it's too expensive to use
> our poor
> > > > > > > > > resource to maintain. So I'd like to propose something to
> reduce
> > > > > > > > > maintenance cost of these templates.
> > > > > > > > >
> > > > > > > > > For example, porting them to giter8 templates is one idea.
> Takako has
> > > > > > > > > worked on it before
> > > > > > > > > (https://github.com/shimamoto/predictionio-template.g8).
> It makes
> > > > > > > > > possible to aggregate multiple templates to a single
> project, and
> > > > > > also
> > > > > > > > > help to reduce duplication. or at least, we might be able
> to
> > > > > > aggregate
> > > > > > > > > them to a single project simply.
> > > > > > > > >
> > > > > > > > > Further, we need to maintain the quickstart document in the
> > > > > > > > > PredictionIO repository as well. I wonder if we can move
> these docs
> > > > > > > > > (and examples as well?) to corresponding template
> repositories. It
> > > > > > > > > would make the synchronization of templates and docs much
> easier.
> > > > > > > > >
> > > > > > > > > Anyway, every release of PredictionIO causes such extra and
> > > > > > > > > unessential costs. I really want to reduce them. It would
> help to
> > > > > > make
> > > > > > > > > more frequent development and releases in the future. I'd
> like to
> > > > > > hear
> > > > > > > > > your opinion or ideas.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Naoki Takezoe
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Naoki Takezoe
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Naoki Takezoe
> > >
> > >
> > >
> > > --
> > > Naoki Takezoe
>
>
>
> --
> Naoki Takezoe
>
Re: [DISCUSS] Improve maintenance of the official templates
Posted by Naoki Takezoe <ta...@gmail.com>.
An another idea is making templates unofficial. It means moving
template repositories to the outside of ASF and we don't guarantee
maintenance for them.
2018年10月9日(火) 19:10 takako shimamoto <ch...@gmail.com>:
>
> We might as well fold them into one single repo. When running git
> clone, every template/example would be pulled down by default. At this
> rate, it's too expensive to use our poor resource to maintain.
>
> On Mon, Oct 1, 2018 at 7:32 AM Naoki Takezoe <ta...@gmail.com> wrote:
> >
> > I think that resurrection of "pio template" might be one idea.
> >
> > It would allow us flexible repository layout and also it can rewrite
> > PredictionIO and Spark versions in build.sbt in templates at setup
> > time. Though this isn't a standard way (e.g. "git clone" or "sbt
> > new"), I think it both users and developers would be happy.
> >
> > 2018年9月25日(火) 17:28 Naoki Takezoe <ta...@gmail.com>:
> > >
> > > Yes. Folding templates into a single repository is ideal except that
> > > users won't able to be use "git clone" to clone a individual template.
> > > "svn" command supports it but using "svn" is not good idea. If we can
> > > revive "pio template", we can implement partial download of git
> > > repository in it. Anyway, still I don't have any solution without
> > > problem.
> > >
> > > I created the following repository as a temporary solution personally.
> > > There are some helper scripts for maintaining the official templates,
> > > but this is obviously not a permanent solution.
> > > https://github.com/takezoe/predictionio-templates-maintenance
> > >
> > > 2018年9月25日(火) 2:45 Donald Szeto <do...@apache.org>:
> > > >
> > > > How about folding them into one single repo with each template/examples as
> > > > subprojects? Since we have retired the use of `pio template` command for a
> > > > while, there is not really a hard requirement that each template has to be
> > > > a git repo by itself.
> > > >
> > > > On Fri, Sep 21, 2018 at 10:59 PM takako shimamoto <ch...@gmail.com>
> > > > wrote:
> > > >
> > > > > I don’t think giter8 is the best solution. There's room for
> > > > > improvement. To improve workability in maintaining 7 official
> > > > > templates and 12 examples, it's a good idea to put them together into
> > > > > a repository. Isn't that just one of the options as a solution to
> > > > > PIO-128?
> > > > >
> > > > > On Sat, Sep 22, 2018 at 9:10 AM Naoki Takezoe <ta...@gmail.com> wrote:
> > > > > >
> > > > > > Hi Donald,
> > > > > >
> > > > > > Thanks for your reply!
> > > > > >
> > > > > > > I think Giter8 is a nice idea, but my concern with it is its tight
> > > > > coupling with sbt.
> > > > > >
> > > > > > Yes, giter8 templates changes user experience largely. This is my
> > > > > > concern about it.
> > > > > >
> > > > > > > How about creating scripts that would make managing these templates
> > > > > easier, despite they are in separate repos?
> > > > > >
> > > > > > Separated repositories are making situiation difficult. If all of them
> > > > > > in a single repository, we can handle them easy. For example, we can
> > > > > > commit and push modification for all templates at once. Even if
> > > > > > repositories are separated, of course, it might be possible to create
> > > > > > a script to automate these operation. But where should we put this
> > > > > > script? But there is another problem. I don't know how users can clone
> > > > > > a certain template if all templates are in a single repository.
> > > > > >
> > > > > > Also, we have to maintain examples in the PredictionIO repository.
> > > > > > Currently, rhere are 12 example templates and need to be synchronized
> > > > > > when original templates are updated.
> > > > > >
> > > > > > 2018年9月22日(土) 8:45 Donald Szeto <do...@apache.org>:
> > > > > > >
> > > > > > > Thanks for bringing this up, Naoki!
> > > > > > >
> > > > > > > I think Giter8 is a nice idea, but my concern with it is its tight
> > > > > coupling
> > > > > > > with sbt. It requires users to have access to sbt in their terminal
> > > > > instead
> > > > > > > of a simple git clone. How about creating scripts that would make
> > > > > managing
> > > > > > > these templates easier, despite they are in separate repos? I was
> > > > > hoping
> > > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-128 would help
> > > > > with
> > > > > > > that.
> > > > > > >
> > > > > > > A counter-argument to the above would be due to
> > > > > > > https://issues.apache.org/jira/projects/PIO/issues/PIO-81, users will
> > > > > need
> > > > > > > to have access to sbt to build official templates anyway (assuming we
> > > > > > > continue using sbt as the build tool for official templates).
> > > > > > >
> > > > > > > I agree the quickstart can be moved to where the template is though.
> > > > > > >
> > > > > > > On Fri, Sep 21, 2018 at 1:02 AM Naoki Takezoe <ta...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I'd like to discuss about maintenance of the official templates.
> > > > > > > >
> > > > > > > > I updated all official templates for PredictionIO 0.13.0. There are 7
> > > > > > > > templates for now and I think it's too expensive to use our poor
> > > > > > > > resource to maintain. So I'd like to propose something to reduce
> > > > > > > > maintenance cost of these templates.
> > > > > > > >
> > > > > > > > For example, porting them to giter8 templates is one idea. Takako has
> > > > > > > > worked on it before
> > > > > > > > (https://github.com/shimamoto/predictionio-template.g8). It makes
> > > > > > > > possible to aggregate multiple templates to a single project, and
> > > > > also
> > > > > > > > help to reduce duplication. or at least, we might be able to
> > > > > aggregate
> > > > > > > > them to a single project simply.
> > > > > > > >
> > > > > > > > Further, we need to maintain the quickstart document in the
> > > > > > > > PredictionIO repository as well. I wonder if we can move these docs
> > > > > > > > (and examples as well?) to corresponding template repositories. It
> > > > > > > > would make the synchronization of templates and docs much easier.
> > > > > > > >
> > > > > > > > Anyway, every release of PredictionIO causes such extra and
> > > > > > > > unessential costs. I really want to reduce them. It would help to
> > > > > make
> > > > > > > > more frequent development and releases in the future. I'd like to
> > > > > hear
> > > > > > > > your opinion or ideas.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Naoki Takezoe
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Naoki Takezoe
> > > > >
> > >
> > >
> > >
> > > --
> > > Naoki Takezoe
> >
> >
> >
> > --
> > Naoki Takezoe
--
Naoki Takezoe