You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Michael Miklavcic <mi...@gmail.com> on 2017/10/30 19:00:41 UTC
Re: [DISCUSS] Using Yarn package manager for metron-alerts
Would love to revive this - I think this could help drastically reduce our
build times for metron-interface, which locally just took me 9 minutes in
non-parallel mode with -DskipTests set. This is a really good suggestion
even just for the offline install and version locking, as pointed out by
Nick.
Best,
Mike
On Thu, Aug 17, 2017 at 8:12 AM, Ryan Merriman <me...@gmail.com> wrote:
> Thanks for this Raghu. You make a pretty compelling argument. I'm +1 on
> moving to yarn.
>
> Ryan
>
> On Wed, Aug 16, 2017 at 3:51 PM, Nick Allen <ni...@nickallen.org> wrote:
>
> > It is also my understanding that
> > there is no hard cut-over to yarn
> > .
> > After we
> > introduce the yarn.lock
> >
> > ,
> > as a developer you can choose to continue to use npm or switch to yarn.
> >
> > Other developers on the project can keep using npm, so you don’t need to
> > > get everyone on your project to convert at the same time. The
> developers
> > > using yarn will all get exactly the same configuration as each other,
> and
> > > the developers using npm may get slightly different configurations,
> which
> > > is the intended behavior of npm.
> >
> >
> > https://yarnpkg.com/lang/en/docs/migrating-from-npm/
> >
> >
> > Oh, and I just switched metron-alerts projects to yarn (as a test) and
> > performed an offline install. It was stupid simple.
> >
> >
> >
> >
> > On Wed, Aug 16, 2017 at 4:12 PM Nick Allen <ni...@nickallen.org> wrote:
> >
> > > Thanks for laying this all out for us, Raghu. Based on the built-in
> > > support for offline installs and version locking, I think this is a
> great
> > > suggestion. (However unfortunate the namespace collision might be.)
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 16, 2017 at 8:51 AM RaghuMitra Kandikonda <
> > > raghumitra.ksv@gmail.com> wrote:
> > >
> > >> I would like to start a discussion around using 'yarn' for managing
> > >> dependencies for metron-alerts instead of 'npm'.
> > >>
> > >> This article beautifully summarizes the need of yarn and npm.
> > >> (https://code.facebook.com/posts/1840075619545360)
> > >>
> > >> If you have read the above article you can skip the next two sections
> > >> and jump to 'Additional advantages of Yarn'
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >> Why do we need a new package manager ?.
> > >>
> > >> While 'npm' does a good job for downloading all the required
> > >> dependencies. npm always tries to download the latest and greatest
> > >> versions of all these dependencies. This would create a problem in
> > >> replicating the same build every time we build. Having hard coded
> > >> versions in the package.json seems like a possible solution but this
> > >> will prevent us from knowing that a library has been updated. In JS
> > >> world the version updates are very frequent and we might be missing on
> > >> some of the latest updates and some of these updates might be related
> > >> to security or a cool feature we would like to have in our code base.
> > >> Ex: Angular made 10 releases in last two months, bootstrap made 2
> > >> releases in last two months.
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >> What is Yarn ?.
> > >>
> > >> Yarn is a new age package manager that can (needs to) be installed
> > >> over npm (or bower). Yarn resolves issues around versioning and
> > >> non-determinism of JS dependencies by using lock files and an install
> > >> algorithm that is deterministic and reliable. These lock files lock
> > >> the installed dependencies to a specific version and ensure that every
> > >> install results in the exact same file structure in node_modules
> > >> across all machines. This kind of a locking mechanism is not available
> > >> with vanilla node.
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >> Additional advantages of Yarn ?.
> > >>
> > >> 1.Yarn helps us to check licenses of all the frameworks we are using.
> > >> (This feature is built in)
> > >> 2.It will reduce the build time of UI for dev as well as in Travis as
> > >> all the dependencies are cached inside '~/.config/yarn/global'
> > >> 3.We can do an offline install of UI as we can zip the dependencies
> > >> and supply it to Yarn instead of downloading from the internet
> > >> 4.Yarn is already integrated with Travis
> > >> (https://blog.travis-ci.com/2016-11-21-travis-ci-now-supports-yarn)
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >> How to migrate ?.
> > >>
> > >> A yarn.lock file can be created from existing package.json file and
> > >> this file would be checked in.
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >> How does the process change ?.
> > >>
> > >> 1.All the developers would use 'npm install' so that they can get the
> > >> latest versions of the dependencies.
> > >> 2.The build would use 'yarn install'. ( This change would be made in
> > >> metron-alerts pom.xml file )
> > >> 3.When the dev notices that a new version of the library is available
> > >> we can test it thoroughly and update yarn.lock file
> > >>
> > >> ============================================================
> > >> ============================================================
> > >> ===============
> > >>
> > >> I am not aware of any other package manager that can do this for us, I
> > >> can explore others if you have a suggestion.
> > >>
> > >>
> > >> -Raghu Mitra
> > >>
> > >
> >
>
Re: [DISCUSS] Using Yarn package manager for metron-alerts
Posted by Otto Fowler <ot...@gmail.com>.
NPM supports this natively now as well it seems:
https://docs.npmjs.com/files/package-locks
On November 1, 2017 at 12:46:44, Nick Allen (nick@nickallen.org) wrote:
I'm still all for it. It is a backwards compatible change and would have
significant benefits. I think it is just a matter of someone getting
cycles to do it.
On Mon, Oct 30, 2017 at 3:00 PM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Would love to revive this - I think this could help drastically reduce
our
> build times for metron-interface, which locally just took me 9 minutes in
> non-parallel mode with -DskipTests set. This is a really good suggestion
> even just for the offline install and version locking, as pointed out by
> Nick.
>
> Best,
> Mike
>
> On Thu, Aug 17, 2017 at 8:12 AM, Ryan Merriman <me...@gmail.com>
> wrote:
>
> > Thanks for this Raghu. You make a pretty compelling argument. I'm +1 on
> > moving to yarn.
> >
> > Ryan
> >
> > On Wed, Aug 16, 2017 at 3:51 PM, Nick Allen <ni...@nickallen.org> wrote:
> >
> > > It is also my understanding that
> > > there is no hard cut-over to yarn
> > > .
> > > After we
> > > introduce the yarn.lock
> > >
> > > ,
> > > as a developer you can choose to continue to use npm or switch to
yarn.
> > >
> > > Other developers on the project can keep using npm, so you don’t need
> to
> > > > get everyone on your project to convert at the same time. The
> > developers
> > > > using yarn will all get exactly the same configuration as each
other,
> > and
> > > > the developers using npm may get slightly different configurations,
> > which
> > > > is the intended behavior of npm.
> > >
> > >
> > > https://yarnpkg.com/lang/en/docs/migrating-from-npm/
> > >
> > >
> > > Oh, and I just switched metron-alerts projects to yarn (as a test)
and
> > > performed an offline install. It was stupid simple.
> > >
> > >
> > >
> > >
> > > On Wed, Aug 16, 2017 at 4:12 PM Nick Allen <ni...@nickallen.org>
wrote:
> > >
> > > > Thanks for laying this all out for us, Raghu. Based on the built-in
> > > > support for offline installs and version locking, I think this is a
> > great
> > > > suggestion. (However unfortunate the namespace collision might be.)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 16, 2017 at 8:51 AM RaghuMitra Kandikonda <
> > > > raghumitra.ksv@gmail.com> wrote:
> > > >
> > > >> I would like to start a discussion around using 'yarn' for
managing
> > > >> dependencies for metron-alerts instead of 'npm'.
> > > >>
> > > >> This article beautifully summarizes the need of yarn and npm.
> > > >> (https://code.facebook.com/posts/1840075619545360)
> > > >>
> > > >> If you have read the above article you can skip the next two
> sections
> > > >> and jump to 'Additional advantages of Yarn'
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> Why do we need a new package manager ?.
> > > >>
> > > >> While 'npm' does a good job for downloading all the required
> > > >> dependencies. npm always tries to download the latest and greatest
> > > >> versions of all these dependencies. This would create a problem in
> > > >> replicating the same build every time we build. Having hard coded
> > > >> versions in the package.json seems like a possible solution but
this
> > > >> will prevent us from knowing that a library has been updated. In
JS
> > > >> world the version updates are very frequent and we might be
missing
> on
> > > >> some of the latest updates and some of these updates might be
> related
> > > >> to security or a cool feature we would like to have in our code
> base.
> > > >> Ex: Angular made 10 releases in last two months, bootstrap made 2
> > > >> releases in last two months.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> What is Yarn ?.
> > > >>
> > > >> Yarn is a new age package manager that can (needs to) be installed
> > > >> over npm (or bower). Yarn resolves issues around versioning and
> > > >> non-determinism of JS dependencies by using lock files and an
> install
> > > >> algorithm that is deterministic and reliable. These lock files
lock
> > > >> the installed dependencies to a specific version and ensure that
> every
> > > >> install results in the exact same file structure in node_modules
> > > >> across all machines. This kind of a locking mechanism is not
> available
> > > >> with vanilla node.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> Additional advantages of Yarn ?.
> > > >>
> > > >> 1.Yarn helps us to check licenses of all the frameworks we are
> using.
> > > >> (This feature is built in)
> > > >> 2.It will reduce the build time of UI for dev as well as in Travis
> as
> > > >> all the dependencies are cached inside '~/.config/yarn/global'
> > > >> 3.We can do an offline install of UI as we can zip the
dependencies
> > > >> and supply it to Yarn instead of downloading from the internet
> > > >> 4.Yarn is already integrated with Travis
> > > >> (https://blog.travis-ci.com/2016-11-21-travis-ci-now-supports-yarn)
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> How to migrate ?.
> > > >>
> > > >> A yarn.lock file can be created from existing package.json file
and
> > > >> this file would be checked in.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> How does the process change ?.
> > > >>
> > > >> 1.All the developers would use 'npm install' so that they can get
> the
> > > >> latest versions of the dependencies.
> > > >> 2.The build would use 'yarn install'. ( This change would be made
in
> > > >> metron-alerts pom.xml file )
> > > >> 3.When the dev notices that a new version of the library is
> available
> > > >> we can test it thoroughly and update yarn.lock file
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >>
> > > >> I am not aware of any other package manager that can do this for
> us, I
> > > >> can explore others if you have a suggestion.
> > > >>
> > > >>
> > > >> -Raghu Mitra
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Using Yarn package manager for metron-alerts
Posted by Nick Allen <ni...@nickallen.org>.
I'm still all for it. It is a backwards compatible change and would have
significant benefits. I think it is just a matter of someone getting
cycles to do it.
On Mon, Oct 30, 2017 at 3:00 PM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Would love to revive this - I think this could help drastically reduce our
> build times for metron-interface, which locally just took me 9 minutes in
> non-parallel mode with -DskipTests set. This is a really good suggestion
> even just for the offline install and version locking, as pointed out by
> Nick.
>
> Best,
> Mike
>
> On Thu, Aug 17, 2017 at 8:12 AM, Ryan Merriman <me...@gmail.com>
> wrote:
>
> > Thanks for this Raghu. You make a pretty compelling argument. I'm +1 on
> > moving to yarn.
> >
> > Ryan
> >
> > On Wed, Aug 16, 2017 at 3:51 PM, Nick Allen <ni...@nickallen.org> wrote:
> >
> > > It is also my understanding that
> > > there is no hard cut-over to yarn
> > > .
> > > After we
> > > introduce the yarn.lock
> > >
> > > ,
> > > as a developer you can choose to continue to use npm or switch to yarn.
> > >
> > > Other developers on the project can keep using npm, so you don’t need
> to
> > > > get everyone on your project to convert at the same time. The
> > developers
> > > > using yarn will all get exactly the same configuration as each other,
> > and
> > > > the developers using npm may get slightly different configurations,
> > which
> > > > is the intended behavior of npm.
> > >
> > >
> > > https://yarnpkg.com/lang/en/docs/migrating-from-npm/
> > >
> > >
> > > Oh, and I just switched metron-alerts projects to yarn (as a test) and
> > > performed an offline install. It was stupid simple.
> > >
> > >
> > >
> > >
> > > On Wed, Aug 16, 2017 at 4:12 PM Nick Allen <ni...@nickallen.org> wrote:
> > >
> > > > Thanks for laying this all out for us, Raghu. Based on the built-in
> > > > support for offline installs and version locking, I think this is a
> > great
> > > > suggestion. (However unfortunate the namespace collision might be.)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 16, 2017 at 8:51 AM RaghuMitra Kandikonda <
> > > > raghumitra.ksv@gmail.com> wrote:
> > > >
> > > >> I would like to start a discussion around using 'yarn' for managing
> > > >> dependencies for metron-alerts instead of 'npm'.
> > > >>
> > > >> This article beautifully summarizes the need of yarn and npm.
> > > >> (https://code.facebook.com/posts/1840075619545360)
> > > >>
> > > >> If you have read the above article you can skip the next two
> sections
> > > >> and jump to 'Additional advantages of Yarn'
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> Why do we need a new package manager ?.
> > > >>
> > > >> While 'npm' does a good job for downloading all the required
> > > >> dependencies. npm always tries to download the latest and greatest
> > > >> versions of all these dependencies. This would create a problem in
> > > >> replicating the same build every time we build. Having hard coded
> > > >> versions in the package.json seems like a possible solution but this
> > > >> will prevent us from knowing that a library has been updated. In JS
> > > >> world the version updates are very frequent and we might be missing
> on
> > > >> some of the latest updates and some of these updates might be
> related
> > > >> to security or a cool feature we would like to have in our code
> base.
> > > >> Ex: Angular made 10 releases in last two months, bootstrap made 2
> > > >> releases in last two months.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> What is Yarn ?.
> > > >>
> > > >> Yarn is a new age package manager that can (needs to) be installed
> > > >> over npm (or bower). Yarn resolves issues around versioning and
> > > >> non-determinism of JS dependencies by using lock files and an
> install
> > > >> algorithm that is deterministic and reliable. These lock files lock
> > > >> the installed dependencies to a specific version and ensure that
> every
> > > >> install results in the exact same file structure in node_modules
> > > >> across all machines. This kind of a locking mechanism is not
> available
> > > >> with vanilla node.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> Additional advantages of Yarn ?.
> > > >>
> > > >> 1.Yarn helps us to check licenses of all the frameworks we are
> using.
> > > >> (This feature is built in)
> > > >> 2.It will reduce the build time of UI for dev as well as in Travis
> as
> > > >> all the dependencies are cached inside '~/.config/yarn/global'
> > > >> 3.We can do an offline install of UI as we can zip the dependencies
> > > >> and supply it to Yarn instead of downloading from the internet
> > > >> 4.Yarn is already integrated with Travis
> > > >> (https://blog.travis-ci.com/2016-11-21-travis-ci-now-supports-yarn)
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> How to migrate ?.
> > > >>
> > > >> A yarn.lock file can be created from existing package.json file and
> > > >> this file would be checked in.
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >> How does the process change ?.
> > > >>
> > > >> 1.All the developers would use 'npm install' so that they can get
> the
> > > >> latest versions of the dependencies.
> > > >> 2.The build would use 'yarn install'. ( This change would be made in
> > > >> metron-alerts pom.xml file )
> > > >> 3.When the dev notices that a new version of the library is
> available
> > > >> we can test it thoroughly and update yarn.lock file
> > > >>
> > > >> ============================================================
> > > >> ============================================================
> > > >> ===============
> > > >>
> > > >> I am not aware of any other package manager that can do this for
> us, I
> > > >> can explore others if you have a suggestion.
> > > >>
> > > >>
> > > >> -Raghu Mitra
> > > >>
> > > >
> > >
> >
>