You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by "Eric Friedrich (efriedri)" <ef...@cisco.com> on 2017/05/01 00:49:50 UTC
Re: Goose installer script
Assuming we stick with goose, why not bundle goose source into the traffic ops RPM? This will pin the version for us and prevent users from needing to run go get
We are allowed to bundle code with the MIT license into our releases.
As for the go installation, what about modifying the RPM spec file to list GoLang as a dependency of the traffic ops RPM?
—Eric
> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com> wrote:
>
> They are, but makes the tooling easier if we are all in Golang
>
> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org> wrote:
>
>> I don't see why re-writing the APIs in something like golang would mean
>> that we also need to re-write the database admin script. I think those two
>> things are mutually exclusive, right?
>>
>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <de...@gmail.com>
>> wrote:
>>
>>> I had that thought, as well as there are more recent versions like
>>> https://github.com/mattes/migrate. The question becomes if we ever get
>>> around to rewriting TrafficOps APIs in golang, will the Perl version then
>>> become obsolete?
>>>
>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org> wrote:
>>>
>>>> Maybe it's time we take a look at what goose really buys us and
>> consider
>>>> writing our own database migration tool. We already have admin.pl, it
>>>> could probably fit in with that?
>>>>
>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>>>> efriedri@cisco.com> wrote:
>>>>
>>>>> Hey Dew-
>>>>> What calls this script?
>>>>>
>>>>> If its called from the Traffic Ops Spec file, then this will cause
>> some
>>>>> pain for those of us that need to install without internet access.
>>>>>
>>>>> —Eric
>>>>>
>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>> dewrich@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I'm working toward a more streamlined installation process for
>>> Traffic
>>>>> Ops
>>>>>> (internally) and publicly. Of course, the same hiccups that
>> everyone
>>>> else
>>>>>> runs into I am as well. Installation of Golang (proper version)
>> and
>>>>>> installation of Goose. Goose has been the most challenging for
>>> several
>>>>>> reasons. The maintainer hasn't made any real changes since 2015,
>> and
>>>> has
>>>>>> not "branched" his code to allow for explicit version download.
>> Per
>>>> his
>>>>>> installation instructions "go get bitbucket.org/liamstask/goose/
>>>>> cmd/goose"
>>>>>>
>>>>>> So I'm I'm proposing to write an installer script in bash to help
>>>>> automate
>>>>>> the Golang install as well as the Goose install. My only concern
>> (as
>>>>> well
>>>>>> as most of yours) is "go get" will grab the latest, but since no
>> real
>>>>>> changes have happened I'm left with no other option.
>>>>>>
>>>>>> Proposed:
>>>>>>
>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>>>>>>
>>>>>> - Install Golang (version 1.8.x)
>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> -Dew
>>>>>
>>>>>
>>>>
>>>
>>
Re: Goose installer script
Posted by Oren Shemesh <or...@qwilt.com>.
I second JvD, and I think the relevant goose version should be copied to a
location managed by the TC team.
This way, the will be no dependency of TC installs on an external,
uncontrollable source of data.
The issue was demonstrated a few months ago when the fancybox-2.1.5 source
code has moved.
On Mon, May 1, 2017 at 11:31 PM, Jan van Doorn <jv...@knutsel.com> wrote:
> I think we should just require the installing person to do a “go get
> liamstack/goose” (or whatever the syntax is), and move on. Put that in the
> installation instructions, like:
>
> Step 1: install Linux
> Step 2: go get goose
> Step 3: the rest.
>
> I know it’s not ideal, but… this seems to be becoming too hard.
>
> Rgds,
> JvD
>
> > On May 1, 2017, at 1:44 PM, Robert Butts <ro...@gmail.com>
> wrote:
> >
> > I just went thru all the recursive dependencies of Goose:
> >
> > https://bitbucket.org/liamstask/goose - MIT
> > https://github.com/kylelemons/go-gypsy - Apache
> > github.com/lib/pq - MIT
> > github.com/mattn/go-sqlite3 - MIT
> > github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3)
> > golang.org/x/net/context - BSD (included by go-sqlite3)
> > golang.org/x/net/html - BSD (included by goquery)
> > golang.org/x/net/html/atom - BSD (included by x/net/html)
> > github.com/andybalholm/cascadia - BSD 2-clause (included by goquery)
> >
> > All those licenses are permissible. So, we can vendor all those, either
> in
> > the Traffic Control, Traffic Ops, or Goose directory.
> >
> > -1 on Git Submodules. They're a pain. I vote simply vendoring them as
> > ordinary dirs, via `git clone` and then deleting the `.git` dir. It's
> > simple, and it works. It includes their respective license files by
> > default. Then we only have to include the proper LICENSE and NOTICE files
> > with the binaries we build.
> >
> > Not requiring Go to install is a separate issue. At a glance, it looks
> like
> > it's possible to rewrite that migration in SQL, but not trivial. Looks
> like
> > CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with.
> html
> > .
> >
> > +1 on setting the Go compiler as a dependency in the Traffic Ops RPM,
> until
> > Go is removed as a dependency. I know it's a pain since it isn't in the
> > standard CentOS packages, but it's less painful than the install failing.
> > We should also add instructions for adding Go to Yum in the
> documentation.
> >
> >
> > On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) <
> > efriedri@cisco.com> wrote:
> >
> >> I know licenses are the issue more than lines of code.
> >>
> >> I’m not a fan of submodules, but if licensing comes through, it might
> make
> >> sense to add goose and its deps as git submodules.
> >>
> >> One other thing to discuss :-)
> >>
> >>
> >>
> >>
> >>> On May 1, 2017, at 2:04 PM, Dan Kirkwood <da...@gmail.com> wrote:
> >>>
> >>> It's not a trivial problem. Yes -- we could include the source --
> >>> goose itself is actually fairly small and MIT licensed. Its
> >>> dependencies have licenses that would need to be vetted, so
> >>> vendoring is also not trivial -- no matter how many lines of code are
> >>> involved.
> >>>
> >>> We could potentially compile goose and distribute within the rpm --
> >>> I've also suggested that before. Unfortunately, we have a migration
> >>> written in go, which requires a go installation at run time. That
> >>> means the requirement of a go installation is not avoidable without
> >>> rewriting that as .sql.
> >>>
> >>> There are alternatives we could use that may not suffer from the same
> >> issues.
> >>>
> >>> I suggest we sit down together at the Summit and decide where to go
> with
> >> this..
> >>>
> >>> -dan
> >>>
> >>> On Mon, May 1, 2017 at 10:37 AM, Robert Butts <
> robert.o.butts@gmail.com>
> >> wrote:
> >>>> +1 on Vendoring. I don't see a difference if it's 375,000 lines or
> >>>> 10,000,000 lines. What does it matter if it's 375k lines in someone
> >> else's
> >>>> repo or our own? It does matter from a security standpoint. It means
> >> we're
> >>>> now vulnerable if their repo is compromised. We shouldn't be pulling
> >>>> _anything_ from the internet at install time.
> >>>>
> >>>> Question for the Apache Gurus: If we include the Goose source, can we
> >> also
> >>>> include a binary built from that source? I don't see a legal or
> >>>> philosophical reason we shouldn't be able to, if we include a hash of
> >> the
> >>>> binary and the LICENSE file. That lets us avoid requiring Go as a
> >>>> dependency, which is difficult since few package managers have a
> modern
> >> Go
> >>>> package. Goose is MIT,
> >>>> https://www.apache.org/dev/licensing-howto.html#binary suggests we
> >> can, yes?
> >>>>
> >>>>
> >>>> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com>
> wrote:
> >>>>
> >>>>> ughh.. I'd forgotten I'd done that in all this..
> >>>>>
> >>>>> Again -- catch-22.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <
> >> mtorluemke@apache.org>
> >>>>> wrote:
> >>>>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
> >>>>> Derek_Gelinas@comcast.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 on both of these.
> >>>>>>>
> >>>>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
> >>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>
> >>>>>>>> Assuming we stick with goose, why not bundle goose source into the
> >>>>>>> traffic ops RPM? This will pin the version for us and prevent users
> >> from
> >>>>>>> needing to run go get
> >>>>>>>
> >>>>>>
> >>>>>> Dan had put in a PR to add the Goose source:
> >>>>>> https://github.com/apache/incubator-trafficcontrol/pull/157
> >>>>>>
> >>>>>> We ended up closing it, as 375,000 lines felt a bit excessive...
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>>
> >>>>>>>> We are allowed to bundle code with the MIT license into our
> >> releases.
> >>>>>>>>
> >>>>>>>> As for the go installation, what about modifying the RPM spec file
> >> to
> >>>>>>> list GoLang as a dependency of the traffic ops RPM?
> >>>>>>>>
> >>>>>>>> —Eric
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <
> dewrich@gmail.com
> >>>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> They are, but makes the tooling easier if we are all in Golang
> >>>>>>>>>
> >>>>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <neuman@apache.org
> >
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I don't see why re-writing the APIs in something like golang
> would
> >>>>> mean
> >>>>>>>>>> that we also need to re-write the database admin script. I
> think
> >>>>>>> those two
> >>>>>>>>>> things are mutually exclusive, right?
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
> >>>>>>> dewrich@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I had that thought, as well as there are more recent versions
> >> like
> >>>>>>>>>>> https://github.com/mattes/migrate. The question becomes if we
> >>>>> ever
> >>>>>>> get
> >>>>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl
> >>>>> version
> >>>>>>> then
> >>>>>>>>>>> become obsolete?
> >>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <
> >> neuman@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Maybe it's time we take a look at what goose really buys us
> and
> >>>>>>>>>> consider
> >>>>>>>>>>>> writing our own database migration tool. We already have
> >>>>> admin.pl,
> >>>>>>> it
> >>>>>>>>>>>> could probably fit in with that?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hey Dew-
> >>>>>>>>>>>>> What calls this script?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will
> >>>>> cause
> >>>>>>>>>> some
> >>>>>>>>>>>>> pain for those of us that need to install without internet
> >>>>> access.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —Eric
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
> >>>>>>>>>> dewrich@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm working toward a more streamlined installation process
> for
> >>>>>>>>>>> Traffic
> >>>>>>>>>>>>> Ops
> >>>>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that
> >>>>>>>>>> everyone
> >>>>>>>>>>>> else
> >>>>>>>>>>>>>> runs into I am as well. Installation of Golang (proper
> >> version)
> >>>>>>>>>> and
> >>>>>>>>>>>>>> installation of Goose. Goose has been the most challenging
> >> for
> >>>>>>>>>>> several
> >>>>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since
> >>>>> 2015,
> >>>>>>>>>> and
> >>>>>>>>>>>> has
> >>>>>>>>>>>>>> not "branched" his code to allow for explicit version
> >> download.
> >>>>>>>>>> Per
> >>>>>>>>>>>> his
> >>>>>>>>>>>>>> installation instructions "go get
> >>>>> bitbucket.org/liamstask/goose/
> >>>>>>>>>>>>> cmd/goose"
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to
> >>>>> help
> >>>>>>>>>>>>> automate
> >>>>>>>>>>>>>> the Golang install as well as the Goose install. My only
> >>>>> concern
> >>>>>>>>>> (as
> >>>>>>>>>>>>> well
> >>>>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but
> since
> >> no
> >>>>>>>>>> real
> >>>>>>>>>>>>>> changes have happened I'm left with no other option.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Proposed:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Install Golang (version 1.8.x)
> >>>>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Dew
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>
> >>
>
>
--
*Oren Shemesh*
Qwilt | Work: +972-72-2221637| Mobile: +972-50-2281168 | orens@qwilt.com
<yo...@qwilt.com>
Re: Goose installer script
Posted by Jan van Doorn <jv...@knutsel.com>.
I think we should just require the installing person to do a “go get liamstack/goose” (or whatever the syntax is), and move on. Put that in the installation instructions, like:
Step 1: install Linux
Step 2: go get goose
Step 3: the rest.
I know it’s not ideal, but… this seems to be becoming too hard.
Rgds,
JvD
> On May 1, 2017, at 1:44 PM, Robert Butts <ro...@gmail.com> wrote:
>
> I just went thru all the recursive dependencies of Goose:
>
> https://bitbucket.org/liamstask/goose - MIT
> https://github.com/kylelemons/go-gypsy - Apache
> github.com/lib/pq - MIT
> github.com/mattn/go-sqlite3 - MIT
> github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3)
> golang.org/x/net/context - BSD (included by go-sqlite3)
> golang.org/x/net/html - BSD (included by goquery)
> golang.org/x/net/html/atom - BSD (included by x/net/html)
> github.com/andybalholm/cascadia - BSD 2-clause (included by goquery)
>
> All those licenses are permissible. So, we can vendor all those, either in
> the Traffic Control, Traffic Ops, or Goose directory.
>
> -1 on Git Submodules. They're a pain. I vote simply vendoring them as
> ordinary dirs, via `git clone` and then deleting the `.git` dir. It's
> simple, and it works. It includes their respective license files by
> default. Then we only have to include the proper LICENSE and NOTICE files
> with the binaries we build.
>
> Not requiring Go to install is a separate issue. At a glance, it looks like
> it's possible to rewrite that migration in SQL, but not trivial. Looks like
> CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with.html
> .
>
> +1 on setting the Go compiler as a dependency in the Traffic Ops RPM, until
> Go is removed as a dependency. I know it's a pain since it isn't in the
> standard CentOS packages, but it's less painful than the install failing.
> We should also add instructions for adding Go to Yum in the documentation.
>
>
> On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
>
>> I know licenses are the issue more than lines of code.
>>
>> I’m not a fan of submodules, but if licensing comes through, it might make
>> sense to add goose and its deps as git submodules.
>>
>> One other thing to discuss :-)
>>
>>
>>
>>
>>> On May 1, 2017, at 2:04 PM, Dan Kirkwood <da...@gmail.com> wrote:
>>>
>>> It's not a trivial problem. Yes -- we could include the source --
>>> goose itself is actually fairly small and MIT licensed. Its
>>> dependencies have licenses that would need to be vetted, so
>>> vendoring is also not trivial -- no matter how many lines of code are
>>> involved.
>>>
>>> We could potentially compile goose and distribute within the rpm --
>>> I've also suggested that before. Unfortunately, we have a migration
>>> written in go, which requires a go installation at run time. That
>>> means the requirement of a go installation is not avoidable without
>>> rewriting that as .sql.
>>>
>>> There are alternatives we could use that may not suffer from the same
>> issues.
>>>
>>> I suggest we sit down together at the Summit and decide where to go with
>> this..
>>>
>>> -dan
>>>
>>> On Mon, May 1, 2017 at 10:37 AM, Robert Butts <ro...@gmail.com>
>> wrote:
>>>> +1 on Vendoring. I don't see a difference if it's 375,000 lines or
>>>> 10,000,000 lines. What does it matter if it's 375k lines in someone
>> else's
>>>> repo or our own? It does matter from a security standpoint. It means
>> we're
>>>> now vulnerable if their repo is compromised. We shouldn't be pulling
>>>> _anything_ from the internet at install time.
>>>>
>>>> Question for the Apache Gurus: If we include the Goose source, can we
>> also
>>>> include a binary built from that source? I don't see a legal or
>>>> philosophical reason we shouldn't be able to, if we include a hash of
>> the
>>>> binary and the LICENSE file. That lets us avoid requiring Go as a
>>>> dependency, which is difficult since few package managers have a modern
>> Go
>>>> package. Goose is MIT,
>>>> https://www.apache.org/dev/licensing-howto.html#binary suggests we
>> can, yes?
>>>>
>>>>
>>>> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com> wrote:
>>>>
>>>>> ughh.. I'd forgotten I'd done that in all this..
>>>>>
>>>>> Again -- catch-22.
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <
>> mtorluemke@apache.org>
>>>>> wrote:
>>>>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
>>>>> Derek_Gelinas@comcast.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 on both of these.
>>>>>>>
>>>>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>
>>>>>>>> Assuming we stick with goose, why not bundle goose source into the
>>>>>>> traffic ops RPM? This will pin the version for us and prevent users
>> from
>>>>>>> needing to run go get
>>>>>>>
>>>>>>
>>>>>> Dan had put in a PR to add the Goose source:
>>>>>> https://github.com/apache/incubator-trafficcontrol/pull/157
>>>>>>
>>>>>> We ended up closing it, as 375,000 lines felt a bit excessive...
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>> We are allowed to bundle code with the MIT license into our
>> releases.
>>>>>>>>
>>>>>>>> As for the go installation, what about modifying the RPM spec file
>> to
>>>>>>> list GoLang as a dependency of the traffic ops RPM?
>>>>>>>>
>>>>>>>> —Eric
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <dewrich@gmail.com
>>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> They are, but makes the tooling easier if we are all in Golang
>>>>>>>>>
>>>>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I don't see why re-writing the APIs in something like golang would
>>>>> mean
>>>>>>>>>> that we also need to re-write the database admin script. I think
>>>>>>> those two
>>>>>>>>>> things are mutually exclusive, right?
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
>>>>>>> dewrich@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I had that thought, as well as there are more recent versions
>> like
>>>>>>>>>>> https://github.com/mattes/migrate. The question becomes if we
>>>>> ever
>>>>>>> get
>>>>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl
>>>>> version
>>>>>>> then
>>>>>>>>>>> become obsolete?
>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <
>> neuman@apache.org>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe it's time we take a look at what goose really buys us and
>>>>>>>>>> consider
>>>>>>>>>>>> writing our own database migration tool. We already have
>>>>> admin.pl,
>>>>>>> it
>>>>>>>>>>>> could probably fit in with that?
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Dew-
>>>>>>>>>>>>> What calls this script?
>>>>>>>>>>>>>
>>>>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will
>>>>> cause
>>>>>>>>>> some
>>>>>>>>>>>>> pain for those of us that need to install without internet
>>>>> access.
>>>>>>>>>>>>>
>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>>>>>>>>>> dewrich@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm working toward a more streamlined installation process for
>>>>>>>>>>> Traffic
>>>>>>>>>>>>> Ops
>>>>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that
>>>>>>>>>> everyone
>>>>>>>>>>>> else
>>>>>>>>>>>>>> runs into I am as well. Installation of Golang (proper
>> version)
>>>>>>>>>> and
>>>>>>>>>>>>>> installation of Goose. Goose has been the most challenging
>> for
>>>>>>>>>>> several
>>>>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since
>>>>> 2015,
>>>>>>>>>> and
>>>>>>>>>>>> has
>>>>>>>>>>>>>> not "branched" his code to allow for explicit version
>> download.
>>>>>>>>>> Per
>>>>>>>>>>>> his
>>>>>>>>>>>>>> installation instructions "go get
>>>>> bitbucket.org/liamstask/goose/
>>>>>>>>>>>>> cmd/goose"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to
>>>>> help
>>>>>>>>>>>>> automate
>>>>>>>>>>>>>> the Golang install as well as the Goose install. My only
>>>>> concern
>>>>>>>>>> (as
>>>>>>>>>>>>> well
>>>>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but since
>> no
>>>>>>>>>> real
>>>>>>>>>>>>>> changes have happened I'm left with no other option.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Proposed:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Install Golang (version 1.8.x)
>>>>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Dew
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>
>>
Re: Goose installer script
Posted by Robert Butts <ro...@gmail.com>.
I just went thru all the recursive dependencies of Goose:
https://bitbucket.org/liamstask/goose - MIT
https://github.com/kylelemons/go-gypsy - Apache
github.com/lib/pq - MIT
github.com/mattn/go-sqlite3 - MIT
github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3)
golang.org/x/net/context - BSD (included by go-sqlite3)
golang.org/x/net/html - BSD (included by goquery)
golang.org/x/net/html/atom - BSD (included by x/net/html)
github.com/andybalholm/cascadia - BSD 2-clause (included by goquery)
All those licenses are permissible. So, we can vendor all those, either in
the Traffic Control, Traffic Ops, or Goose directory.
-1 on Git Submodules. They're a pain. I vote simply vendoring them as
ordinary dirs, via `git clone` and then deleting the `.git` dir. It's
simple, and it works. It includes their respective license files by
default. Then we only have to include the proper LICENSE and NOTICE files
with the binaries we build.
Not requiring Go to install is a separate issue. At a glance, it looks like
it's possible to rewrite that migration in SQL, but not trivial. Looks like
CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with.html
.
+1 on setting the Go compiler as a dependency in the Traffic Ops RPM, until
Go is removed as a dependency. I know it's a pain since it isn't in the
standard CentOS packages, but it's less painful than the install failing.
We should also add instructions for adding Go to Yum in the documentation.
On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) <
efriedri@cisco.com> wrote:
> I know licenses are the issue more than lines of code.
>
> I’m not a fan of submodules, but if licensing comes through, it might make
> sense to add goose and its deps as git submodules.
>
> One other thing to discuss :-)
>
>
>
>
> > On May 1, 2017, at 2:04 PM, Dan Kirkwood <da...@gmail.com> wrote:
> >
> > It's not a trivial problem. Yes -- we could include the source --
> > goose itself is actually fairly small and MIT licensed. Its
> > dependencies have licenses that would need to be vetted, so
> > vendoring is also not trivial -- no matter how many lines of code are
> > involved.
> >
> > We could potentially compile goose and distribute within the rpm --
> > I've also suggested that before. Unfortunately, we have a migration
> > written in go, which requires a go installation at run time. That
> > means the requirement of a go installation is not avoidable without
> > rewriting that as .sql.
> >
> > There are alternatives we could use that may not suffer from the same
> issues.
> >
> > I suggest we sit down together at the Summit and decide where to go with
> this..
> >
> > -dan
> >
> > On Mon, May 1, 2017 at 10:37 AM, Robert Butts <ro...@gmail.com>
> wrote:
> >> +1 on Vendoring. I don't see a difference if it's 375,000 lines or
> >> 10,000,000 lines. What does it matter if it's 375k lines in someone
> else's
> >> repo or our own? It does matter from a security standpoint. It means
> we're
> >> now vulnerable if their repo is compromised. We shouldn't be pulling
> >> _anything_ from the internet at install time.
> >>
> >> Question for the Apache Gurus: If we include the Goose source, can we
> also
> >> include a binary built from that source? I don't see a legal or
> >> philosophical reason we shouldn't be able to, if we include a hash of
> the
> >> binary and the LICENSE file. That lets us avoid requiring Go as a
> >> dependency, which is difficult since few package managers have a modern
> Go
> >> package. Goose is MIT,
> >> https://www.apache.org/dev/licensing-howto.html#binary suggests we
> can, yes?
> >>
> >>
> >> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com> wrote:
> >>
> >>> ughh.. I'd forgotten I'd done that in all this..
> >>>
> >>> Again -- catch-22.
> >>>
> >>>
> >>>
> >>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <
> mtorluemke@apache.org>
> >>> wrote:
> >>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
> >>> Derek_Gelinas@comcast.com>
> >>>> wrote:
> >>>>
> >>>>> +1 on both of these.
> >>>>>
> >>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
> >>>>> efriedri@cisco.com> wrote:
> >>>>>>
> >>>>>> Assuming we stick with goose, why not bundle goose source into the
> >>>>> traffic ops RPM? This will pin the version for us and prevent users
> from
> >>>>> needing to run go get
> >>>>>
> >>>>
> >>>> Dan had put in a PR to add the Goose source:
> >>>> https://github.com/apache/incubator-trafficcontrol/pull/157
> >>>>
> >>>> We ended up closing it, as 375,000 lines felt a bit excessive...
> >>>>
> >>>>
> >>>>
> >>>>>>
> >>>>>> We are allowed to bundle code with the MIT license into our
> releases.
> >>>>>>
> >>>>>> As for the go installation, what about modifying the RPM spec file
> to
> >>>>> list GoLang as a dependency of the traffic ops RPM?
> >>>>>>
> >>>>>> —Eric
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <dewrich@gmail.com
> >
> >>>>> wrote:
> >>>>>>>
> >>>>>>> They are, but makes the tooling easier if we are all in Golang
> >>>>>>>
> >>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>> I don't see why re-writing the APIs in something like golang would
> >>> mean
> >>>>>>>> that we also need to re-write the database admin script. I think
> >>>>> those two
> >>>>>>>> things are mutually exclusive, right?
> >>>>>>>>
> >>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
> >>>>> dewrich@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I had that thought, as well as there are more recent versions
> like
> >>>>>>>>> https://github.com/mattes/migrate. The question becomes if we
> >>> ever
> >>>>> get
> >>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl
> >>> version
> >>>>> then
> >>>>>>>>> become obsolete?
> >>>>>>>>>
> >>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <
> neuman@apache.org>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Maybe it's time we take a look at what goose really buys us and
> >>>>>>>> consider
> >>>>>>>>>> writing our own database migration tool. We already have
> >>> admin.pl,
> >>>>> it
> >>>>>>>>>> could probably fit in with that?
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hey Dew-
> >>>>>>>>>>> What calls this script?
> >>>>>>>>>>>
> >>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will
> >>> cause
> >>>>>>>> some
> >>>>>>>>>>> pain for those of us that need to install without internet
> >>> access.
> >>>>>>>>>>>
> >>>>>>>>>>> —Eric
> >>>>>>>>>>>
> >>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
> >>>>>>>> dewrich@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm working toward a more streamlined installation process for
> >>>>>>>>> Traffic
> >>>>>>>>>>> Ops
> >>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that
> >>>>>>>> everyone
> >>>>>>>>>> else
> >>>>>>>>>>>> runs into I am as well. Installation of Golang (proper
> version)
> >>>>>>>> and
> >>>>>>>>>>>> installation of Goose. Goose has been the most challenging
> for
> >>>>>>>>> several
> >>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since
> >>> 2015,
> >>>>>>>> and
> >>>>>>>>>> has
> >>>>>>>>>>>> not "branched" his code to allow for explicit version
> download.
> >>>>>>>> Per
> >>>>>>>>>> his
> >>>>>>>>>>>> installation instructions "go get
> >>> bitbucket.org/liamstask/goose/
> >>>>>>>>>>> cmd/goose"
> >>>>>>>>>>>>
> >>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to
> >>> help
> >>>>>>>>>>> automate
> >>>>>>>>>>>> the Golang install as well as the Goose install. My only
> >>> concern
> >>>>>>>> (as
> >>>>>>>>>>> well
> >>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but since
> no
> >>>>>>>> real
> >>>>>>>>>>>> changes have happened I'm left with no other option.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Proposed:
> >>>>>>>>>>>>
> >>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Install Golang (version 1.8.x)
> >>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Dew
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>
>
>
Re: Goose installer script
Posted by "Eric Friedrich (efriedri)" <ef...@cisco.com>.
I know licenses are the issue more than lines of code.
I’m not a fan of submodules, but if licensing comes through, it might make sense to add goose and its deps as git submodules.
One other thing to discuss :-)
> On May 1, 2017, at 2:04 PM, Dan Kirkwood <da...@gmail.com> wrote:
>
> It's not a trivial problem. Yes -- we could include the source --
> goose itself is actually fairly small and MIT licensed. Its
> dependencies have licenses that would need to be vetted, so
> vendoring is also not trivial -- no matter how many lines of code are
> involved.
>
> We could potentially compile goose and distribute within the rpm --
> I've also suggested that before. Unfortunately, we have a migration
> written in go, which requires a go installation at run time. That
> means the requirement of a go installation is not avoidable without
> rewriting that as .sql.
>
> There are alternatives we could use that may not suffer from the same issues.
>
> I suggest we sit down together at the Summit and decide where to go with this..
>
> -dan
>
> On Mon, May 1, 2017 at 10:37 AM, Robert Butts <ro...@gmail.com> wrote:
>> +1 on Vendoring. I don't see a difference if it's 375,000 lines or
>> 10,000,000 lines. What does it matter if it's 375k lines in someone else's
>> repo or our own? It does matter from a security standpoint. It means we're
>> now vulnerable if their repo is compromised. We shouldn't be pulling
>> _anything_ from the internet at install time.
>>
>> Question for the Apache Gurus: If we include the Goose source, can we also
>> include a binary built from that source? I don't see a legal or
>> philosophical reason we shouldn't be able to, if we include a hash of the
>> binary and the LICENSE file. That lets us avoid requiring Go as a
>> dependency, which is difficult since few package managers have a modern Go
>> package. Goose is MIT,
>> https://www.apache.org/dev/licensing-howto.html#binary suggests we can, yes?
>>
>>
>> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com> wrote:
>>
>>> ughh.. I'd forgotten I'd done that in all this..
>>>
>>> Again -- catch-22.
>>>
>>>
>>>
>>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <mt...@apache.org>
>>> wrote:
>>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
>>> Derek_Gelinas@comcast.com>
>>>> wrote:
>>>>
>>>>> +1 on both of these.
>>>>>
>>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
>>>>> efriedri@cisco.com> wrote:
>>>>>>
>>>>>> Assuming we stick with goose, why not bundle goose source into the
>>>>> traffic ops RPM? This will pin the version for us and prevent users from
>>>>> needing to run go get
>>>>>
>>>>
>>>> Dan had put in a PR to add the Goose source:
>>>> https://github.com/apache/incubator-trafficcontrol/pull/157
>>>>
>>>> We ended up closing it, as 375,000 lines felt a bit excessive...
>>>>
>>>>
>>>>
>>>>>>
>>>>>> We are allowed to bundle code with the MIT license into our releases.
>>>>>>
>>>>>> As for the go installation, what about modifying the RPM spec file to
>>>>> list GoLang as a dependency of the traffic ops RPM?
>>>>>>
>>>>>> —Eric
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>> They are, but makes the tooling easier if we are all in Golang
>>>>>>>
>>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
>>>>> wrote:
>>>>>>>>
>>>>>>>> I don't see why re-writing the APIs in something like golang would
>>> mean
>>>>>>>> that we also need to re-write the database admin script. I think
>>>>> those two
>>>>>>>> things are mutually exclusive, right?
>>>>>>>>
>>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
>>>>> dewrich@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I had that thought, as well as there are more recent versions like
>>>>>>>>> https://github.com/mattes/migrate. The question becomes if we
>>> ever
>>>>> get
>>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl
>>> version
>>>>> then
>>>>>>>>> become obsolete?
>>>>>>>>>
>>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Maybe it's time we take a look at what goose really buys us and
>>>>>>>> consider
>>>>>>>>>> writing our own database migration tool. We already have
>>> admin.pl,
>>>>> it
>>>>>>>>>> could probably fit in with that?
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Dew-
>>>>>>>>>>> What calls this script?
>>>>>>>>>>>
>>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will
>>> cause
>>>>>>>> some
>>>>>>>>>>> pain for those of us that need to install without internet
>>> access.
>>>>>>>>>>>
>>>>>>>>>>> —Eric
>>>>>>>>>>>
>>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>>>>>>>> dewrich@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I'm working toward a more streamlined installation process for
>>>>>>>>> Traffic
>>>>>>>>>>> Ops
>>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that
>>>>>>>> everyone
>>>>>>>>>> else
>>>>>>>>>>>> runs into I am as well. Installation of Golang (proper version)
>>>>>>>> and
>>>>>>>>>>>> installation of Goose. Goose has been the most challenging for
>>>>>>>>> several
>>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since
>>> 2015,
>>>>>>>> and
>>>>>>>>>> has
>>>>>>>>>>>> not "branched" his code to allow for explicit version download.
>>>>>>>> Per
>>>>>>>>>> his
>>>>>>>>>>>> installation instructions "go get
>>> bitbucket.org/liamstask/goose/
>>>>>>>>>>> cmd/goose"
>>>>>>>>>>>>
>>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to
>>> help
>>>>>>>>>>> automate
>>>>>>>>>>>> the Golang install as well as the Goose install. My only
>>> concern
>>>>>>>> (as
>>>>>>>>>>> well
>>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but since no
>>>>>>>> real
>>>>>>>>>>>> changes have happened I'm left with no other option.
>>>>>>>>>>>>
>>>>>>>>>>>> Proposed:
>>>>>>>>>>>>
>>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>>>>>>>>>>>>
>>>>>>>>>>>> - Install Golang (version 1.8.x)
>>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> -Dew
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>
Re: Goose installer script
Posted by Dan Kirkwood <da...@gmail.com>.
It's not a trivial problem. Yes -- we could include the source --
goose itself is actually fairly small and MIT licensed. Its
dependencies have licenses that would need to be vetted, so
vendoring is also not trivial -- no matter how many lines of code are
involved.
We could potentially compile goose and distribute within the rpm --
I've also suggested that before. Unfortunately, we have a migration
written in go, which requires a go installation at run time. That
means the requirement of a go installation is not avoidable without
rewriting that as .sql.
There are alternatives we could use that may not suffer from the same issues.
I suggest we sit down together at the Summit and decide where to go with this..
-dan
On Mon, May 1, 2017 at 10:37 AM, Robert Butts <ro...@gmail.com> wrote:
> +1 on Vendoring. I don't see a difference if it's 375,000 lines or
> 10,000,000 lines. What does it matter if it's 375k lines in someone else's
> repo or our own? It does matter from a security standpoint. It means we're
> now vulnerable if their repo is compromised. We shouldn't be pulling
> _anything_ from the internet at install time.
>
> Question for the Apache Gurus: If we include the Goose source, can we also
> include a binary built from that source? I don't see a legal or
> philosophical reason we shouldn't be able to, if we include a hash of the
> binary and the LICENSE file. That lets us avoid requiring Go as a
> dependency, which is difficult since few package managers have a modern Go
> package. Goose is MIT,
> https://www.apache.org/dev/licensing-howto.html#binary suggests we can, yes?
>
>
> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com> wrote:
>
>> ughh.. I'd forgotten I'd done that in all this..
>>
>> Again -- catch-22.
>>
>>
>>
>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <mt...@apache.org>
>> wrote:
>> > On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
>> Derek_Gelinas@comcast.com>
>> > wrote:
>> >
>> >> +1 on both of these.
>> >>
>> >> > On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
>> >> efriedri@cisco.com> wrote:
>> >> >
>> >> > Assuming we stick with goose, why not bundle goose source into the
>> >> traffic ops RPM? This will pin the version for us and prevent users from
>> >> needing to run go get
>> >>
>> >
>> > Dan had put in a PR to add the Goose source:
>> > https://github.com/apache/incubator-trafficcontrol/pull/157
>> >
>> > We ended up closing it, as 375,000 lines felt a bit excessive...
>> >
>> >
>> >
>> >> >
>> >> > We are allowed to bundle code with the MIT license into our releases.
>> >> >
>> >> > As for the go installation, what about modifying the RPM spec file to
>> >> list GoLang as a dependency of the traffic ops RPM?
>> >> >
>> >> > —Eric
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com>
>> >> wrote:
>> >> >>
>> >> >> They are, but makes the tooling easier if we are all in Golang
>> >> >>
>> >> >>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
>> >> wrote:
>> >> >>>
>> >> >>> I don't see why re-writing the APIs in something like golang would
>> mean
>> >> >>> that we also need to re-write the database admin script. I think
>> >> those two
>> >> >>> things are mutually exclusive, right?
>> >> >>>
>> >> >>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
>> >> dewrich@gmail.com>
>> >> >>> wrote:
>> >> >>>
>> >> >>>> I had that thought, as well as there are more recent versions like
>> >> >>>> https://github.com/mattes/migrate. The question becomes if we
>> ever
>> >> get
>> >> >>>> around to rewriting TrafficOps APIs in golang, will the Perl
>> version
>> >> then
>> >> >>>> become obsolete?
>> >> >>>>
>> >> >>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org>
>> >> wrote:
>> >> >>>>>
>> >> >>>>> Maybe it's time we take a look at what goose really buys us and
>> >> >>> consider
>> >> >>>>> writing our own database migration tool. We already have
>> admin.pl,
>> >> it
>> >> >>>>> could probably fit in with that?
>> >> >>>>>
>> >> >>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>> >> >>>>> efriedri@cisco.com> wrote:
>> >> >>>>>
>> >> >>>>>> Hey Dew-
>> >> >>>>>> What calls this script?
>> >> >>>>>>
>> >> >>>>>> If its called from the Traffic Ops Spec file, then this will
>> cause
>> >> >>> some
>> >> >>>>>> pain for those of us that need to install without internet
>> access.
>> >> >>>>>>
>> >> >>>>>> —Eric
>> >> >>>>>>
>> >> >>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>> >> >>> dewrich@gmail.com>
>> >> >>>>>> wrote:
>> >> >>>>>>>
>> >> >>>>>>> I'm working toward a more streamlined installation process for
>> >> >>>> Traffic
>> >> >>>>>> Ops
>> >> >>>>>>> (internally) and publicly. Of course, the same hiccups that
>> >> >>> everyone
>> >> >>>>> else
>> >> >>>>>>> runs into I am as well. Installation of Golang (proper version)
>> >> >>> and
>> >> >>>>>>> installation of Goose. Goose has been the most challenging for
>> >> >>>> several
>> >> >>>>>>> reasons. The maintainer hasn't made any real changes since
>> 2015,
>> >> >>> and
>> >> >>>>> has
>> >> >>>>>>> not "branched" his code to allow for explicit version download.
>> >> >>> Per
>> >> >>>>> his
>> >> >>>>>>> installation instructions "go get
>> bitbucket.org/liamstask/goose/
>> >> >>>>>> cmd/goose"
>> >> >>>>>>>
>> >> >>>>>>> So I'm I'm proposing to write an installer script in bash to
>> help
>> >> >>>>>> automate
>> >> >>>>>>> the Golang install as well as the Goose install. My only
>> concern
>> >> >>> (as
>> >> >>>>>> well
>> >> >>>>>>> as most of yours) is "go get" will grab the latest, but since no
>> >> >>> real
>> >> >>>>>>> changes have happened I'm left with no other option.
>> >> >>>>>>>
>> >> >>>>>>> Proposed:
>> >> >>>>>>>
>> >> >>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>> >> >>>>>>>
>> >> >>>>>>> - Install Golang (version 1.8.x)
>> >> >>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>> >> >>>>>>>
>> >> >>>>>>> Thoughts?
>> >> >>>>>>>
>> >> >>>>>>> -Dew
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>
>> >> >>>
>> >> >
>> >>
>>
Re: Goose installer script
Posted by Robert Butts <ro...@gmail.com>.
+1 on Vendoring. I don't see a difference if it's 375,000 lines or
10,000,000 lines. What does it matter if it's 375k lines in someone else's
repo or our own? It does matter from a security standpoint. It means we're
now vulnerable if their repo is compromised. We shouldn't be pulling
_anything_ from the internet at install time.
Question for the Apache Gurus: If we include the Goose source, can we also
include a binary built from that source? I don't see a legal or
philosophical reason we shouldn't be able to, if we include a hash of the
binary and the LICENSE file. That lets us avoid requiring Go as a
dependency, which is difficult since few package managers have a modern Go
package. Goose is MIT,
https://www.apache.org/dev/licensing-howto.html#binary suggests we can, yes?
On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <da...@gmail.com> wrote:
> ughh.. I'd forgotten I'd done that in all this..
>
> Again -- catch-22.
>
>
>
> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <mt...@apache.org>
> wrote:
> > On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <
> Derek_Gelinas@comcast.com>
> > wrote:
> >
> >> +1 on both of these.
> >>
> >> > On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
> >> efriedri@cisco.com> wrote:
> >> >
> >> > Assuming we stick with goose, why not bundle goose source into the
> >> traffic ops RPM? This will pin the version for us and prevent users from
> >> needing to run go get
> >>
> >
> > Dan had put in a PR to add the Goose source:
> > https://github.com/apache/incubator-trafficcontrol/pull/157
> >
> > We ended up closing it, as 375,000 lines felt a bit excessive...
> >
> >
> >
> >> >
> >> > We are allowed to bundle code with the MIT license into our releases.
> >> >
> >> > As for the go installation, what about modifying the RPM spec file to
> >> list GoLang as a dependency of the traffic ops RPM?
> >> >
> >> > —Eric
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com>
> >> wrote:
> >> >>
> >> >> They are, but makes the tooling easier if we are all in Golang
> >> >>
> >> >>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
> >> wrote:
> >> >>>
> >> >>> I don't see why re-writing the APIs in something like golang would
> mean
> >> >>> that we also need to re-write the database admin script. I think
> >> those two
> >> >>> things are mutually exclusive, right?
> >> >>>
> >> >>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
> >> dewrich@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>> I had that thought, as well as there are more recent versions like
> >> >>>> https://github.com/mattes/migrate. The question becomes if we
> ever
> >> get
> >> >>>> around to rewriting TrafficOps APIs in golang, will the Perl
> version
> >> then
> >> >>>> become obsolete?
> >> >>>>
> >> >>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org>
> >> wrote:
> >> >>>>>
> >> >>>>> Maybe it's time we take a look at what goose really buys us and
> >> >>> consider
> >> >>>>> writing our own database migration tool. We already have
> admin.pl,
> >> it
> >> >>>>> could probably fit in with that?
> >> >>>>>
> >> >>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
> >> >>>>> efriedri@cisco.com> wrote:
> >> >>>>>
> >> >>>>>> Hey Dew-
> >> >>>>>> What calls this script?
> >> >>>>>>
> >> >>>>>> If its called from the Traffic Ops Spec file, then this will
> cause
> >> >>> some
> >> >>>>>> pain for those of us that need to install without internet
> access.
> >> >>>>>>
> >> >>>>>> —Eric
> >> >>>>>>
> >> >>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
> >> >>> dewrich@gmail.com>
> >> >>>>>> wrote:
> >> >>>>>>>
> >> >>>>>>> I'm working toward a more streamlined installation process for
> >> >>>> Traffic
> >> >>>>>> Ops
> >> >>>>>>> (internally) and publicly. Of course, the same hiccups that
> >> >>> everyone
> >> >>>>> else
> >> >>>>>>> runs into I am as well. Installation of Golang (proper version)
> >> >>> and
> >> >>>>>>> installation of Goose. Goose has been the most challenging for
> >> >>>> several
> >> >>>>>>> reasons. The maintainer hasn't made any real changes since
> 2015,
> >> >>> and
> >> >>>>> has
> >> >>>>>>> not "branched" his code to allow for explicit version download.
> >> >>> Per
> >> >>>>> his
> >> >>>>>>> installation instructions "go get
> bitbucket.org/liamstask/goose/
> >> >>>>>> cmd/goose"
> >> >>>>>>>
> >> >>>>>>> So I'm I'm proposing to write an installer script in bash to
> help
> >> >>>>>> automate
> >> >>>>>>> the Golang install as well as the Goose install. My only
> concern
> >> >>> (as
> >> >>>>>> well
> >> >>>>>>> as most of yours) is "go get" will grab the latest, but since no
> >> >>> real
> >> >>>>>>> changes have happened I'm left with no other option.
> >> >>>>>>>
> >> >>>>>>> Proposed:
> >> >>>>>>>
> >> >>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
> >> >>>>>>>
> >> >>>>>>> - Install Golang (version 1.8.x)
> >> >>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
> >> >>>>>>>
> >> >>>>>>> Thoughts?
> >> >>>>>>>
> >> >>>>>>> -Dew
> >> >>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >
> >>
>
Re: Goose installer script
Posted by Dan Kirkwood <da...@gmail.com>.
ughh.. I'd forgotten I'd done that in all this..
Again -- catch-22.
On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke <mt...@apache.org> wrote:
> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <De...@comcast.com>
> wrote:
>
>> +1 on both of these.
>>
>> > On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
>> efriedri@cisco.com> wrote:
>> >
>> > Assuming we stick with goose, why not bundle goose source into the
>> traffic ops RPM? This will pin the version for us and prevent users from
>> needing to run go get
>>
>
> Dan had put in a PR to add the Goose source:
> https://github.com/apache/incubator-trafficcontrol/pull/157
>
> We ended up closing it, as 375,000 lines felt a bit excessive...
>
>
>
>> >
>> > We are allowed to bundle code with the MIT license into our releases.
>> >
>> > As for the go installation, what about modifying the RPM spec file to
>> list GoLang as a dependency of the traffic ops RPM?
>> >
>> > —Eric
>> >
>> >
>> >
>> >
>> >
>> >> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com>
>> wrote:
>> >>
>> >> They are, but makes the tooling easier if we are all in Golang
>> >>
>> >>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
>> wrote:
>> >>>
>> >>> I don't see why re-writing the APIs in something like golang would mean
>> >>> that we also need to re-write the database admin script. I think
>> those two
>> >>> things are mutually exclusive, right?
>> >>>
>> >>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
>> dewrich@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> I had that thought, as well as there are more recent versions like
>> >>>> https://github.com/mattes/migrate. The question becomes if we ever
>> get
>> >>>> around to rewriting TrafficOps APIs in golang, will the Perl version
>> then
>> >>>> become obsolete?
>> >>>>
>> >>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org>
>> wrote:
>> >>>>>
>> >>>>> Maybe it's time we take a look at what goose really buys us and
>> >>> consider
>> >>>>> writing our own database migration tool. We already have admin.pl,
>> it
>> >>>>> could probably fit in with that?
>> >>>>>
>> >>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>> >>>>> efriedri@cisco.com> wrote:
>> >>>>>
>> >>>>>> Hey Dew-
>> >>>>>> What calls this script?
>> >>>>>>
>> >>>>>> If its called from the Traffic Ops Spec file, then this will cause
>> >>> some
>> >>>>>> pain for those of us that need to install without internet access.
>> >>>>>>
>> >>>>>> —Eric
>> >>>>>>
>> >>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>> >>> dewrich@gmail.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> I'm working toward a more streamlined installation process for
>> >>>> Traffic
>> >>>>>> Ops
>> >>>>>>> (internally) and publicly. Of course, the same hiccups that
>> >>> everyone
>> >>>>> else
>> >>>>>>> runs into I am as well. Installation of Golang (proper version)
>> >>> and
>> >>>>>>> installation of Goose. Goose has been the most challenging for
>> >>>> several
>> >>>>>>> reasons. The maintainer hasn't made any real changes since 2015,
>> >>> and
>> >>>>> has
>> >>>>>>> not "branched" his code to allow for explicit version download.
>> >>> Per
>> >>>>> his
>> >>>>>>> installation instructions "go get bitbucket.org/liamstask/goose/
>> >>>>>> cmd/goose"
>> >>>>>>>
>> >>>>>>> So I'm I'm proposing to write an installer script in bash to help
>> >>>>>> automate
>> >>>>>>> the Golang install as well as the Goose install. My only concern
>> >>> (as
>> >>>>>> well
>> >>>>>>> as most of yours) is "go get" will grab the latest, but since no
>> >>> real
>> >>>>>>> changes have happened I'm left with no other option.
>> >>>>>>>
>> >>>>>>> Proposed:
>> >>>>>>>
>> >>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>> >>>>>>>
>> >>>>>>> - Install Golang (version 1.8.x)
>> >>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>> >>>>>>>
>> >>>>>>> Thoughts?
>> >>>>>>>
>> >>>>>>> -Dew
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >
>>
Re: Goose installer script
Posted by Mark Torluemke <mt...@apache.org>.
On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek <De...@comcast.com>
wrote:
> +1 on both of these.
>
> > On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
> >
> > Assuming we stick with goose, why not bundle goose source into the
> traffic ops RPM? This will pin the version for us and prevent users from
> needing to run go get
>
Dan had put in a PR to add the Goose source:
https://github.com/apache/incubator-trafficcontrol/pull/157
We ended up closing it, as 375,000 lines felt a bit excessive...
> >
> > We are allowed to bundle code with the MIT license into our releases.
> >
> > As for the go installation, what about modifying the RPM spec file to
> list GoLang as a dependency of the traffic ops RPM?
> >
> > —Eric
> >
> >
> >
> >
> >
> >> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com>
> wrote:
> >>
> >> They are, but makes the tooling easier if we are all in Golang
> >>
> >>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org>
> wrote:
> >>>
> >>> I don't see why re-writing the APIs in something like golang would mean
> >>> that we also need to re-write the database admin script. I think
> those two
> >>> things are mutually exclusive, right?
> >>>
> >>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <
> dewrich@gmail.com>
> >>> wrote:
> >>>
> >>>> I had that thought, as well as there are more recent versions like
> >>>> https://github.com/mattes/migrate. The question becomes if we ever
> get
> >>>> around to rewriting TrafficOps APIs in golang, will the Perl version
> then
> >>>> become obsolete?
> >>>>
> >>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org>
> wrote:
> >>>>>
> >>>>> Maybe it's time we take a look at what goose really buys us and
> >>> consider
> >>>>> writing our own database migration tool. We already have admin.pl,
> it
> >>>>> could probably fit in with that?
> >>>>>
> >>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
> >>>>> efriedri@cisco.com> wrote:
> >>>>>
> >>>>>> Hey Dew-
> >>>>>> What calls this script?
> >>>>>>
> >>>>>> If its called from the Traffic Ops Spec file, then this will cause
> >>> some
> >>>>>> pain for those of us that need to install without internet access.
> >>>>>>
> >>>>>> —Eric
> >>>>>>
> >>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
> >>> dewrich@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> I'm working toward a more streamlined installation process for
> >>>> Traffic
> >>>>>> Ops
> >>>>>>> (internally) and publicly. Of course, the same hiccups that
> >>> everyone
> >>>>> else
> >>>>>>> runs into I am as well. Installation of Golang (proper version)
> >>> and
> >>>>>>> installation of Goose. Goose has been the most challenging for
> >>>> several
> >>>>>>> reasons. The maintainer hasn't made any real changes since 2015,
> >>> and
> >>>>> has
> >>>>>>> not "branched" his code to allow for explicit version download.
> >>> Per
> >>>>> his
> >>>>>>> installation instructions "go get bitbucket.org/liamstask/goose/
> >>>>>> cmd/goose"
> >>>>>>>
> >>>>>>> So I'm I'm proposing to write an installer script in bash to help
> >>>>>> automate
> >>>>>>> the Golang install as well as the Goose install. My only concern
> >>> (as
> >>>>>> well
> >>>>>>> as most of yours) is "go get" will grab the latest, but since no
> >>> real
> >>>>>>> changes have happened I'm left with no other option.
> >>>>>>>
> >>>>>>> Proposed:
> >>>>>>>
> >>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
> >>>>>>>
> >>>>>>> - Install Golang (version 1.8.x)
> >>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
> >>>>>>>
> >>>>>>> Thoughts?
> >>>>>>>
> >>>>>>> -Dew
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >
>
Re: Goose installer script
Posted by "Gelinas, Derek" <De...@comcast.com>.
+1 on both of these.
> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) <ef...@cisco.com> wrote:
>
> Assuming we stick with goose, why not bundle goose source into the traffic ops RPM? This will pin the version for us and prevent users from needing to run go get
>
> We are allowed to bundle code with the MIT license into our releases.
>
> As for the go installation, what about modifying the RPM spec file to list GoLang as a dependency of the traffic ops RPM?
>
> —Eric
>
>
>
>
>
>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <de...@gmail.com> wrote:
>>
>> They are, but makes the tooling easier if we are all in Golang
>>
>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <ne...@apache.org> wrote:
>>>
>>> I don't see why re-writing the APIs in something like golang would mean
>>> that we also need to re-write the database admin script. I think those two
>>> things are mutually exclusive, right?
>>>
>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson <de...@gmail.com>
>>> wrote:
>>>
>>>> I had that thought, as well as there are more recent versions like
>>>> https://github.com/mattes/migrate. The question becomes if we ever get
>>>> around to rewriting TrafficOps APIs in golang, will the Perl version then
>>>> become obsolete?
>>>>
>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman <ne...@apache.org> wrote:
>>>>>
>>>>> Maybe it's time we take a look at what goose really buys us and
>>> consider
>>>>> writing our own database migration tool. We already have admin.pl, it
>>>>> could probably fit in with that?
>>>>>
>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) <
>>>>> efriedri@cisco.com> wrote:
>>>>>
>>>>>> Hey Dew-
>>>>>> What calls this script?
>>>>>>
>>>>>> If its called from the Traffic Ops Spec file, then this will cause
>>> some
>>>>>> pain for those of us that need to install without internet access.
>>>>>>
>>>>>> —Eric
>>>>>>
>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson <
>>> dewrich@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> I'm working toward a more streamlined installation process for
>>>> Traffic
>>>>>> Ops
>>>>>>> (internally) and publicly. Of course, the same hiccups that
>>> everyone
>>>>> else
>>>>>>> runs into I am as well. Installation of Golang (proper version)
>>> and
>>>>>>> installation of Goose. Goose has been the most challenging for
>>>> several
>>>>>>> reasons. The maintainer hasn't made any real changes since 2015,
>>> and
>>>>> has
>>>>>>> not "branched" his code to allow for explicit version download.
>>> Per
>>>>> his
>>>>>>> installation instructions "go get bitbucket.org/liamstask/goose/
>>>>>> cmd/goose"
>>>>>>>
>>>>>>> So I'm I'm proposing to write an installer script in bash to help
>>>>>> automate
>>>>>>> the Golang install as well as the Goose install. My only concern
>>> (as
>>>>>> well
>>>>>>> as most of yours) is "go get" will grab the latest, but since no
>>> real
>>>>>>> changes have happened I'm left with no other option.
>>>>>>>
>>>>>>> Proposed:
>>>>>>>
>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh
>>>>>>>
>>>>>>> - Install Golang (version 1.8.x)
>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> -Dew
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>