You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Allen Wittenauer <aw...@altiscale.com> on 2016/02/25 18:16:07 UTC
node.js and more as dependencies
Hey folks.
Have any of you looked at YARN-3368? Is adding node.js+a bunch of other stuff as dependencies just for the UI a good idea? Doesn’t that seem significantly heavyweight? How hard is this going to be operationally to manage?
Re: node.js and more as dependencies
Posted by Steve Loughran <st...@hortonworks.com>.
> On 2 Mar 2016, at 05:55, Haohui Mai <ri...@gmail.com> wrote:
>
> +1 on adding npm / gulp / mocha to be part of the build dependencies
> (not runtime). The main benefit is modularity -- for example, in the
> HDFS UI we manually duplicate the navigation bars and footer. We don't
> have unit tests for them due to the lack of infrastructure.
>
> In my opinion Introducing npm will effectively bridge the gap.
Still need a plan for testing any web UI with .js in the mix; it's a lot harder than for static HTML.
HtmlUnit should be able to handle it: it uses the JVM's own Javascript engine to interpret the code —has anyone looked at using this to test the new UIs?
Re: node.js and more as dependencies
Posted by Steve Loughran <st...@hortonworks.com>.
> On 2 Mar 2016, at 05:55, Haohui Mai <ri...@gmail.com> wrote:
>
> +1 on adding npm / gulp / mocha to be part of the build dependencies
> (not runtime). The main benefit is modularity -- for example, in the
> HDFS UI we manually duplicate the navigation bars and footer. We don't
> have unit tests for them due to the lack of infrastructure.
>
> In my opinion Introducing npm will effectively bridge the gap.
Still need a plan for testing any web UI with .js in the mix; it's a lot harder than for static HTML.
HtmlUnit should be able to handle it: it uses the JVM's own Javascript engine to interpret the code —has anyone looked at using this to test the new UIs?
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
@Haihui/@Colin,
Thanks for suggestions, I agree that there're pros and cons of lightweight
framework, Ember and other frameworks like React.js. And since new
frameworks are continuously released, we can spend years to finalize a
framework :).
I would like to revisit choices of frameworks after we have more
contributors. Now it's a side project to several YARN contributors, most of
us are pretty new to JS and understand Ember better than others. I wanna to
get some basic runnable works done before make it projection-ready.
In between, I will try to make code not to be framework-specific as much as
possible to alleviate painful of switching frameworks.
@Steve,
Unit testing for visible part will be hard. First, it changes a lot while
developing, we can spend hours write a test in HtmlUnit, but a few lines of
layout code changes could lead to rewrite entire test.
I think we need tests of invisible part, like how to translate REST
response to models. And smoke tests of visible part, like basic layout,
table rendering, error handling, etc. And manually test is required to
complex UI logic, like SVG chart rendering, etc.
Thanks,
Wangda
On Wed, Mar 2, 2016 at 11:22 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Frameworks are a double-edged sword. On one hand, developers are
> somewhat more productive when using a framework (at least, a good
> one). On the other hand, frameworks increase the barrier to entry for
> new contributors, because they have to learn the idioms of the
> framework before they can contribute anything meaningful.
>
> There is also a huge amount of churn in Javascript frameworks.
> There's no guarantee that 2015's framework will still be popular in
> 2017. On a related note, the "why ember" page you linked to says that
> "Now that Ember has reached 1.0 the code samples below are no longer
> correct and the expressed opinions may no longer be accurate." Oops.
>
> In HTrace, we went with a more lightweight framework, backbone.js, and
> a small set of libraries like moment.js and jquery. In general,
> libraries are easier to handle than frameworks, since they have a
> pretty narrowly defined scope. If you have problems with one you can
> easily swap it out for another, or just write your own functions to do
> what you need. With frameworks, you don't always have that option.
> (I would point out that the sunburst visualization you linked to is
> part of the d3 library, not the ember framework. It could just as
> easily be used with dust, backbone, or any other framework as with
> ember.)
>
> Anyway, the choice of framework and library should be up to the
> developers doing the coding and maintenance. If you think that ember
> will be helpful, then sure, that's fine.
>
> Colin
>
> On Tue, Mar 1, 2016 at 9:55 PM, Haohui Mai <ri...@gmail.com> wrote:
> > +1 on adding npm / gulp / mocha to be part of the build dependencies
> > (not runtime). The main benefit is modularity -- for example, in the
> > HDFS UI we manually duplicate the navigation bars and footer. We don't
> > have unit tests for them due to the lack of infrastructure.
> >
> > In my opinion Introducing npm will effectively bridge the gap.
> >
> > However, I'm not entirely convinced by the Ember.js argument -- I
> > understand it might provide better integration with Ambari, but there
> > are clear trends that the industry is moving to a more reactive UI
> > design. I think the decision of using ember.js exclusively might worth
> > revisiting. To me it makes more sense to move both HDFS / YARN towards
> > React and go from there.
> >
> > ~Haohui
> >
> > On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
> >> Hi Colin,
> >>
> >> Thanks for comment, I think your concerns are all valid but also
> arguable:
> >>
> >> First, YARN UI is different from HDFS UI, it is much more complex:
> >> 1. We have many data models, such as app/container/system/node, etc. UI
> of
> >> HDFS is more like a file explorer.
> >> 2. We plan to add more rich data visualization to YARN UI to make admin
> can
> >> easier identify what happened. For example, using sunburst map
> >> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
> >> cluster/queue/user/app, etc.
> >> 3. We need to get data from different sources with different format. For
> >> example, application's running container information stores at RM and
> >> finished container information stores at Timeline server. We need to get
> >> data from both daemon, normalize these data (because REST API is
> different)
> >> and aggregate them.
> >>
> >> Ember.js could simplify what we should do a lot:
> >> - It has great data store design so we can easily normalize data model
> from
> >> different sources with different formats (adapter)
> >> - It can easily bind data model with view, so any changes to data store
> >> (like application status updated) can trigger page re-rendering without
> any
> >> additional works.
> >> - Besides binding data with view, it can also bind data to other
> computed
> >> properties. For example, if property of a model relies on another model,
> >> all properties/models will be updated altogether.
> >> - Integrates bower/broccoli/watchman to help with package
> >> management/build/development.
> >> - For other benefits, please refer to Why Ember?
> >> <http://mutewinter.github.io/why_ember/#/slides> slides.
> >>
> >> The plan of nextgen YARN UI is not only inherit and prettify existing
> YARN
> >> UI. I hope it can let users can get deep insight of what happens in
> their
> >> cluster. As you said, a simple JS framework can also achieve what we
> wanna
> >> to do, but using well designed framework can avoid reinvent the wheel.
> >>
> >> Regarding to your concerns about JS compilation/compaction, I think it
> is
> >> not conflict with open source: In source folder (git repository), all
> code
> >> are readable. Compilation/compaction code only exists in released code.
> I
> >> agree that we don't need obfuscation at all, but source code compaction
> >> could increase performance a lot, we could have heavy rendering tasks,
> such
> >> as visualization from statuses of 10K+ applications. Just like Java
> code of
> >> Hadoop, no user will try to get source code from a running cluster :).
> >>
> >> I will make sure integration to Maven is as less as possible, we should
> >> only need one single sub module, and limit all changes in that module
> only.
> >>
> >> Please let me know if you have any other concerns.
> >>
> >> Thanks,
> >> Wangda
> >>
> >> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org>
> wrote:
> >>
> >>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
> >>>
> >>> The main use-cases for "JS builds" seem to be if you want to minimize
> >>> or obfuscate your JS. Given that this is open source code,
> >>> obfuscation seems unnecessary. Given that it's a low-traffic
> >>> management interface, minimizing the JS seems like a premature
> >>> optimization.
> >>>
> >>> The HDFS user interface is based on dust.js, and it just requires JS
> >>> files to be copied into the correct location.
> >>>
> >>> Perhaps there are advantages to ember.js that I am missing. But
> >>> there's also a big advantage to not having to manage a node.js build
> >>> system separate from Maven and CMake. What do you think?
> >>>
> >>> best,
> >>> Colin
> >>>
> >>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com>
> wrote:
> >>> > Hi Allen,
> >>> >
> >>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS
> Package
> >>> > Manager) to manage packages.
> >>> >
> >>> > One thing to clarify is: npm dependency is only required by build
> stage
> >>> (JS
> >>> > build is stitching source files and renaming variables). After JS
> build
> >>> > completes, there's no dependency of Node.JS any more. Server such as
> RM
> >>> > only needs to run a HTTP server to host JS files, and browser will
> take
> >>> > care of page rendering, just like HDFS/Spark/Mesos UI.
> >>> >
> >>> > There're a couple of other Apache projects are using Ember.JS, such
> as
> >>> > Tez/Ambari. Ember.JS can help front-end developers easier manage
> models,
> >>> > pages, events and packages.
> >>> >
> >>> > Thanks,
> >>> > Wangda
> >>> >
> >>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
> >>> wrote:
> >>> >
> >>> >>
> >>> >> Hey folks.
> >>> >>
> >>> >> Have any of you looked at YARN-3368? Is adding node.js+a
> bunch
> >>> of
> >>> >> other stuff as dependencies just for the UI a good idea? Doesn’t
> that
> >>> seem
> >>> >> significantly heavyweight? How hard is this going to be
> operationally
> >>> to
> >>> >> manage?
> >>>
>
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
@Haihui/@Colin,
Thanks for suggestions, I agree that there're pros and cons of lightweight
framework, Ember and other frameworks like React.js. And since new
frameworks are continuously released, we can spend years to finalize a
framework :).
I would like to revisit choices of frameworks after we have more
contributors. Now it's a side project to several YARN contributors, most of
us are pretty new to JS and understand Ember better than others. I wanna to
get some basic runnable works done before make it projection-ready.
In between, I will try to make code not to be framework-specific as much as
possible to alleviate painful of switching frameworks.
@Steve,
Unit testing for visible part will be hard. First, it changes a lot while
developing, we can spend hours write a test in HtmlUnit, but a few lines of
layout code changes could lead to rewrite entire test.
I think we need tests of invisible part, like how to translate REST
response to models. And smoke tests of visible part, like basic layout,
table rendering, error handling, etc. And manually test is required to
complex UI logic, like SVG chart rendering, etc.
Thanks,
Wangda
On Wed, Mar 2, 2016 at 11:22 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Frameworks are a double-edged sword. On one hand, developers are
> somewhat more productive when using a framework (at least, a good
> one). On the other hand, frameworks increase the barrier to entry for
> new contributors, because they have to learn the idioms of the
> framework before they can contribute anything meaningful.
>
> There is also a huge amount of churn in Javascript frameworks.
> There's no guarantee that 2015's framework will still be popular in
> 2017. On a related note, the "why ember" page you linked to says that
> "Now that Ember has reached 1.0 the code samples below are no longer
> correct and the expressed opinions may no longer be accurate." Oops.
>
> In HTrace, we went with a more lightweight framework, backbone.js, and
> a small set of libraries like moment.js and jquery. In general,
> libraries are easier to handle than frameworks, since they have a
> pretty narrowly defined scope. If you have problems with one you can
> easily swap it out for another, or just write your own functions to do
> what you need. With frameworks, you don't always have that option.
> (I would point out that the sunburst visualization you linked to is
> part of the d3 library, not the ember framework. It could just as
> easily be used with dust, backbone, or any other framework as with
> ember.)
>
> Anyway, the choice of framework and library should be up to the
> developers doing the coding and maintenance. If you think that ember
> will be helpful, then sure, that's fine.
>
> Colin
>
> On Tue, Mar 1, 2016 at 9:55 PM, Haohui Mai <ri...@gmail.com> wrote:
> > +1 on adding npm / gulp / mocha to be part of the build dependencies
> > (not runtime). The main benefit is modularity -- for example, in the
> > HDFS UI we manually duplicate the navigation bars and footer. We don't
> > have unit tests for them due to the lack of infrastructure.
> >
> > In my opinion Introducing npm will effectively bridge the gap.
> >
> > However, I'm not entirely convinced by the Ember.js argument -- I
> > understand it might provide better integration with Ambari, but there
> > are clear trends that the industry is moving to a more reactive UI
> > design. I think the decision of using ember.js exclusively might worth
> > revisiting. To me it makes more sense to move both HDFS / YARN towards
> > React and go from there.
> >
> > ~Haohui
> >
> > On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
> >> Hi Colin,
> >>
> >> Thanks for comment, I think your concerns are all valid but also
> arguable:
> >>
> >> First, YARN UI is different from HDFS UI, it is much more complex:
> >> 1. We have many data models, such as app/container/system/node, etc. UI
> of
> >> HDFS is more like a file explorer.
> >> 2. We plan to add more rich data visualization to YARN UI to make admin
> can
> >> easier identify what happened. For example, using sunburst map
> >> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
> >> cluster/queue/user/app, etc.
> >> 3. We need to get data from different sources with different format. For
> >> example, application's running container information stores at RM and
> >> finished container information stores at Timeline server. We need to get
> >> data from both daemon, normalize these data (because REST API is
> different)
> >> and aggregate them.
> >>
> >> Ember.js could simplify what we should do a lot:
> >> - It has great data store design so we can easily normalize data model
> from
> >> different sources with different formats (adapter)
> >> - It can easily bind data model with view, so any changes to data store
> >> (like application status updated) can trigger page re-rendering without
> any
> >> additional works.
> >> - Besides binding data with view, it can also bind data to other
> computed
> >> properties. For example, if property of a model relies on another model,
> >> all properties/models will be updated altogether.
> >> - Integrates bower/broccoli/watchman to help with package
> >> management/build/development.
> >> - For other benefits, please refer to Why Ember?
> >> <http://mutewinter.github.io/why_ember/#/slides> slides.
> >>
> >> The plan of nextgen YARN UI is not only inherit and prettify existing
> YARN
> >> UI. I hope it can let users can get deep insight of what happens in
> their
> >> cluster. As you said, a simple JS framework can also achieve what we
> wanna
> >> to do, but using well designed framework can avoid reinvent the wheel.
> >>
> >> Regarding to your concerns about JS compilation/compaction, I think it
> is
> >> not conflict with open source: In source folder (git repository), all
> code
> >> are readable. Compilation/compaction code only exists in released code.
> I
> >> agree that we don't need obfuscation at all, but source code compaction
> >> could increase performance a lot, we could have heavy rendering tasks,
> such
> >> as visualization from statuses of 10K+ applications. Just like Java
> code of
> >> Hadoop, no user will try to get source code from a running cluster :).
> >>
> >> I will make sure integration to Maven is as less as possible, we should
> >> only need one single sub module, and limit all changes in that module
> only.
> >>
> >> Please let me know if you have any other concerns.
> >>
> >> Thanks,
> >> Wangda
> >>
> >> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org>
> wrote:
> >>
> >>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
> >>>
> >>> The main use-cases for "JS builds" seem to be if you want to minimize
> >>> or obfuscate your JS. Given that this is open source code,
> >>> obfuscation seems unnecessary. Given that it's a low-traffic
> >>> management interface, minimizing the JS seems like a premature
> >>> optimization.
> >>>
> >>> The HDFS user interface is based on dust.js, and it just requires JS
> >>> files to be copied into the correct location.
> >>>
> >>> Perhaps there are advantages to ember.js that I am missing. But
> >>> there's also a big advantage to not having to manage a node.js build
> >>> system separate from Maven and CMake. What do you think?
> >>>
> >>> best,
> >>> Colin
> >>>
> >>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com>
> wrote:
> >>> > Hi Allen,
> >>> >
> >>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS
> Package
> >>> > Manager) to manage packages.
> >>> >
> >>> > One thing to clarify is: npm dependency is only required by build
> stage
> >>> (JS
> >>> > build is stitching source files and renaming variables). After JS
> build
> >>> > completes, there's no dependency of Node.JS any more. Server such as
> RM
> >>> > only needs to run a HTTP server to host JS files, and browser will
> take
> >>> > care of page rendering, just like HDFS/Spark/Mesos UI.
> >>> >
> >>> > There're a couple of other Apache projects are using Ember.JS, such
> as
> >>> > Tez/Ambari. Ember.JS can help front-end developers easier manage
> models,
> >>> > pages, events and packages.
> >>> >
> >>> > Thanks,
> >>> > Wangda
> >>> >
> >>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
> >>> wrote:
> >>> >
> >>> >>
> >>> >> Hey folks.
> >>> >>
> >>> >> Have any of you looked at YARN-3368? Is adding node.js+a
> bunch
> >>> of
> >>> >> other stuff as dependencies just for the UI a good idea? Doesn’t
> that
> >>> seem
> >>> >> significantly heavyweight? How hard is this going to be
> operationally
> >>> to
> >>> >> manage?
> >>>
>
Re: node.js and more as dependencies
Posted by "Colin P. McCabe" <cm...@apache.org>.
Frameworks are a double-edged sword. On one hand, developers are
somewhat more productive when using a framework (at least, a good
one). On the other hand, frameworks increase the barrier to entry for
new contributors, because they have to learn the idioms of the
framework before they can contribute anything meaningful.
There is also a huge amount of churn in Javascript frameworks.
There's no guarantee that 2015's framework will still be popular in
2017. On a related note, the "why ember" page you linked to says that
"Now that Ember has reached 1.0 the code samples below are no longer
correct and the expressed opinions may no longer be accurate." Oops.
In HTrace, we went with a more lightweight framework, backbone.js, and
a small set of libraries like moment.js and jquery. In general,
libraries are easier to handle than frameworks, since they have a
pretty narrowly defined scope. If you have problems with one you can
easily swap it out for another, or just write your own functions to do
what you need. With frameworks, you don't always have that option.
(I would point out that the sunburst visualization you linked to is
part of the d3 library, not the ember framework. It could just as
easily be used with dust, backbone, or any other framework as with
ember.)
Anyway, the choice of framework and library should be up to the
developers doing the coding and maintenance. If you think that ember
will be helpful, then sure, that's fine.
Colin
On Tue, Mar 1, 2016 at 9:55 PM, Haohui Mai <ri...@gmail.com> wrote:
> +1 on adding npm / gulp / mocha to be part of the build dependencies
> (not runtime). The main benefit is modularity -- for example, in the
> HDFS UI we manually duplicate the navigation bars and footer. We don't
> have unit tests for them due to the lack of infrastructure.
>
> In my opinion Introducing npm will effectively bridge the gap.
>
> However, I'm not entirely convinced by the Ember.js argument -- I
> understand it might provide better integration with Ambari, but there
> are clear trends that the industry is moving to a more reactive UI
> design. I think the decision of using ember.js exclusively might worth
> revisiting. To me it makes more sense to move both HDFS / YARN towards
> React and go from there.
>
> ~Haohui
>
> On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
>> Hi Colin,
>>
>> Thanks for comment, I think your concerns are all valid but also arguable:
>>
>> First, YARN UI is different from HDFS UI, it is much more complex:
>> 1. We have many data models, such as app/container/system/node, etc. UI of
>> HDFS is more like a file explorer.
>> 2. We plan to add more rich data visualization to YARN UI to make admin can
>> easier identify what happened. For example, using sunburst map
>> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
>> cluster/queue/user/app, etc.
>> 3. We need to get data from different sources with different format. For
>> example, application's running container information stores at RM and
>> finished container information stores at Timeline server. We need to get
>> data from both daemon, normalize these data (because REST API is different)
>> and aggregate them.
>>
>> Ember.js could simplify what we should do a lot:
>> - It has great data store design so we can easily normalize data model from
>> different sources with different formats (adapter)
>> - It can easily bind data model with view, so any changes to data store
>> (like application status updated) can trigger page re-rendering without any
>> additional works.
>> - Besides binding data with view, it can also bind data to other computed
>> properties. For example, if property of a model relies on another model,
>> all properties/models will be updated altogether.
>> - Integrates bower/broccoli/watchman to help with package
>> management/build/development.
>> - For other benefits, please refer to Why Ember?
>> <http://mutewinter.github.io/why_ember/#/slides> slides.
>>
>> The plan of nextgen YARN UI is not only inherit and prettify existing YARN
>> UI. I hope it can let users can get deep insight of what happens in their
>> cluster. As you said, a simple JS framework can also achieve what we wanna
>> to do, but using well designed framework can avoid reinvent the wheel.
>>
>> Regarding to your concerns about JS compilation/compaction, I think it is
>> not conflict with open source: In source folder (git repository), all code
>> are readable. Compilation/compaction code only exists in released code. I
>> agree that we don't need obfuscation at all, but source code compaction
>> could increase performance a lot, we could have heavy rendering tasks, such
>> as visualization from statuses of 10K+ applications. Just like Java code of
>> Hadoop, no user will try to get source code from a running cluster :).
>>
>> I will make sure integration to Maven is as less as possible, we should
>> only need one single sub module, and limit all changes in that module only.
>>
>> Please let me know if you have any other concerns.
>>
>> Thanks,
>> Wangda
>>
>> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
>>
>>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>>>
>>> The main use-cases for "JS builds" seem to be if you want to minimize
>>> or obfuscate your JS. Given that this is open source code,
>>> obfuscation seems unnecessary. Given that it's a low-traffic
>>> management interface, minimizing the JS seems like a premature
>>> optimization.
>>>
>>> The HDFS user interface is based on dust.js, and it just requires JS
>>> files to be copied into the correct location.
>>>
>>> Perhaps there are advantages to ember.js that I am missing. But
>>> there's also a big advantage to not having to manage a node.js build
>>> system separate from Maven and CMake. What do you think?
>>>
>>> best,
>>> Colin
>>>
>>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
>>> > Hi Allen,
>>> >
>>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
>>> > Manager) to manage packages.
>>> >
>>> > One thing to clarify is: npm dependency is only required by build stage
>>> (JS
>>> > build is stitching source files and renaming variables). After JS build
>>> > completes, there's no dependency of Node.JS any more. Server such as RM
>>> > only needs to run a HTTP server to host JS files, and browser will take
>>> > care of page rendering, just like HDFS/Spark/Mesos UI.
>>> >
>>> > There're a couple of other Apache projects are using Ember.JS, such as
>>> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
>>> > pages, events and packages.
>>> >
>>> > Thanks,
>>> > Wangda
>>> >
>>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
>>> wrote:
>>> >
>>> >>
>>> >> Hey folks.
>>> >>
>>> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
>>> of
>>> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
>>> seem
>>> >> significantly heavyweight? How hard is this going to be operationally
>>> to
>>> >> manage?
>>>
Re: node.js and more as dependencies
Posted by "Colin P. McCabe" <cm...@apache.org>.
Frameworks are a double-edged sword. On one hand, developers are
somewhat more productive when using a framework (at least, a good
one). On the other hand, frameworks increase the barrier to entry for
new contributors, because they have to learn the idioms of the
framework before they can contribute anything meaningful.
There is also a huge amount of churn in Javascript frameworks.
There's no guarantee that 2015's framework will still be popular in
2017. On a related note, the "why ember" page you linked to says that
"Now that Ember has reached 1.0 the code samples below are no longer
correct and the expressed opinions may no longer be accurate." Oops.
In HTrace, we went with a more lightweight framework, backbone.js, and
a small set of libraries like moment.js and jquery. In general,
libraries are easier to handle than frameworks, since they have a
pretty narrowly defined scope. If you have problems with one you can
easily swap it out for another, or just write your own functions to do
what you need. With frameworks, you don't always have that option.
(I would point out that the sunburst visualization you linked to is
part of the d3 library, not the ember framework. It could just as
easily be used with dust, backbone, or any other framework as with
ember.)
Anyway, the choice of framework and library should be up to the
developers doing the coding and maintenance. If you think that ember
will be helpful, then sure, that's fine.
Colin
On Tue, Mar 1, 2016 at 9:55 PM, Haohui Mai <ri...@gmail.com> wrote:
> +1 on adding npm / gulp / mocha to be part of the build dependencies
> (not runtime). The main benefit is modularity -- for example, in the
> HDFS UI we manually duplicate the navigation bars and footer. We don't
> have unit tests for them due to the lack of infrastructure.
>
> In my opinion Introducing npm will effectively bridge the gap.
>
> However, I'm not entirely convinced by the Ember.js argument -- I
> understand it might provide better integration with Ambari, but there
> are clear trends that the industry is moving to a more reactive UI
> design. I think the decision of using ember.js exclusively might worth
> revisiting. To me it makes more sense to move both HDFS / YARN towards
> React and go from there.
>
> ~Haohui
>
> On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
>> Hi Colin,
>>
>> Thanks for comment, I think your concerns are all valid but also arguable:
>>
>> First, YARN UI is different from HDFS UI, it is much more complex:
>> 1. We have many data models, such as app/container/system/node, etc. UI of
>> HDFS is more like a file explorer.
>> 2. We plan to add more rich data visualization to YARN UI to make admin can
>> easier identify what happened. For example, using sunburst map
>> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
>> cluster/queue/user/app, etc.
>> 3. We need to get data from different sources with different format. For
>> example, application's running container information stores at RM and
>> finished container information stores at Timeline server. We need to get
>> data from both daemon, normalize these data (because REST API is different)
>> and aggregate them.
>>
>> Ember.js could simplify what we should do a lot:
>> - It has great data store design so we can easily normalize data model from
>> different sources with different formats (adapter)
>> - It can easily bind data model with view, so any changes to data store
>> (like application status updated) can trigger page re-rendering without any
>> additional works.
>> - Besides binding data with view, it can also bind data to other computed
>> properties. For example, if property of a model relies on another model,
>> all properties/models will be updated altogether.
>> - Integrates bower/broccoli/watchman to help with package
>> management/build/development.
>> - For other benefits, please refer to Why Ember?
>> <http://mutewinter.github.io/why_ember/#/slides> slides.
>>
>> The plan of nextgen YARN UI is not only inherit and prettify existing YARN
>> UI. I hope it can let users can get deep insight of what happens in their
>> cluster. As you said, a simple JS framework can also achieve what we wanna
>> to do, but using well designed framework can avoid reinvent the wheel.
>>
>> Regarding to your concerns about JS compilation/compaction, I think it is
>> not conflict with open source: In source folder (git repository), all code
>> are readable. Compilation/compaction code only exists in released code. I
>> agree that we don't need obfuscation at all, but source code compaction
>> could increase performance a lot, we could have heavy rendering tasks, such
>> as visualization from statuses of 10K+ applications. Just like Java code of
>> Hadoop, no user will try to get source code from a running cluster :).
>>
>> I will make sure integration to Maven is as less as possible, we should
>> only need one single sub module, and limit all changes in that module only.
>>
>> Please let me know if you have any other concerns.
>>
>> Thanks,
>> Wangda
>>
>> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
>>
>>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>>>
>>> The main use-cases for "JS builds" seem to be if you want to minimize
>>> or obfuscate your JS. Given that this is open source code,
>>> obfuscation seems unnecessary. Given that it's a low-traffic
>>> management interface, minimizing the JS seems like a premature
>>> optimization.
>>>
>>> The HDFS user interface is based on dust.js, and it just requires JS
>>> files to be copied into the correct location.
>>>
>>> Perhaps there are advantages to ember.js that I am missing. But
>>> there's also a big advantage to not having to manage a node.js build
>>> system separate from Maven and CMake. What do you think?
>>>
>>> best,
>>> Colin
>>>
>>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
>>> > Hi Allen,
>>> >
>>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
>>> > Manager) to manage packages.
>>> >
>>> > One thing to clarify is: npm dependency is only required by build stage
>>> (JS
>>> > build is stitching source files and renaming variables). After JS build
>>> > completes, there's no dependency of Node.JS any more. Server such as RM
>>> > only needs to run a HTTP server to host JS files, and browser will take
>>> > care of page rendering, just like HDFS/Spark/Mesos UI.
>>> >
>>> > There're a couple of other Apache projects are using Ember.JS, such as
>>> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
>>> > pages, events and packages.
>>> >
>>> > Thanks,
>>> > Wangda
>>> >
>>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
>>> wrote:
>>> >
>>> >>
>>> >> Hey folks.
>>> >>
>>> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
>>> of
>>> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
>>> seem
>>> >> significantly heavyweight? How hard is this going to be operationally
>>> to
>>> >> manage?
>>>
Re: node.js and more as dependencies
Posted by Haohui Mai <ri...@gmail.com>.
+1 on adding npm / gulp / mocha to be part of the build dependencies
(not runtime). The main benefit is modularity -- for example, in the
HDFS UI we manually duplicate the navigation bars and footer. We don't
have unit tests for them due to the lack of infrastructure.
In my opinion Introducing npm will effectively bridge the gap.
However, I'm not entirely convinced by the Ember.js argument -- I
understand it might provide better integration with Ambari, but there
are clear trends that the industry is moving to a more reactive UI
design. I think the decision of using ember.js exclusively might worth
revisiting. To me it makes more sense to move both HDFS / YARN towards
React and go from there.
~Haohui
On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
> Hi Colin,
>
> Thanks for comment, I think your concerns are all valid but also arguable:
>
> First, YARN UI is different from HDFS UI, it is much more complex:
> 1. We have many data models, such as app/container/system/node, etc. UI of
> HDFS is more like a file explorer.
> 2. We plan to add more rich data visualization to YARN UI to make admin can
> easier identify what happened. For example, using sunburst map
> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
> cluster/queue/user/app, etc.
> 3. We need to get data from different sources with different format. For
> example, application's running container information stores at RM and
> finished container information stores at Timeline server. We need to get
> data from both daemon, normalize these data (because REST API is different)
> and aggregate them.
>
> Ember.js could simplify what we should do a lot:
> - It has great data store design so we can easily normalize data model from
> different sources with different formats (adapter)
> - It can easily bind data model with view, so any changes to data store
> (like application status updated) can trigger page re-rendering without any
> additional works.
> - Besides binding data with view, it can also bind data to other computed
> properties. For example, if property of a model relies on another model,
> all properties/models will be updated altogether.
> - Integrates bower/broccoli/watchman to help with package
> management/build/development.
> - For other benefits, please refer to Why Ember?
> <http://mutewinter.github.io/why_ember/#/slides> slides.
>
> The plan of nextgen YARN UI is not only inherit and prettify existing YARN
> UI. I hope it can let users can get deep insight of what happens in their
> cluster. As you said, a simple JS framework can also achieve what we wanna
> to do, but using well designed framework can avoid reinvent the wheel.
>
> Regarding to your concerns about JS compilation/compaction, I think it is
> not conflict with open source: In source folder (git repository), all code
> are readable. Compilation/compaction code only exists in released code. I
> agree that we don't need obfuscation at all, but source code compaction
> could increase performance a lot, we could have heavy rendering tasks, such
> as visualization from statuses of 10K+ applications. Just like Java code of
> Hadoop, no user will try to get source code from a running cluster :).
>
> I will make sure integration to Maven is as less as possible, we should
> only need one single sub module, and limit all changes in that module only.
>
> Please let me know if you have any other concerns.
>
> Thanks,
> Wangda
>
> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
>
>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>>
>> The main use-cases for "JS builds" seem to be if you want to minimize
>> or obfuscate your JS. Given that this is open source code,
>> obfuscation seems unnecessary. Given that it's a low-traffic
>> management interface, minimizing the JS seems like a premature
>> optimization.
>>
>> The HDFS user interface is based on dust.js, and it just requires JS
>> files to be copied into the correct location.
>>
>> Perhaps there are advantages to ember.js that I am missing. But
>> there's also a big advantage to not having to manage a node.js build
>> system separate from Maven and CMake. What do you think?
>>
>> best,
>> Colin
>>
>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
>> > Hi Allen,
>> >
>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
>> > Manager) to manage packages.
>> >
>> > One thing to clarify is: npm dependency is only required by build stage
>> (JS
>> > build is stitching source files and renaming variables). After JS build
>> > completes, there's no dependency of Node.JS any more. Server such as RM
>> > only needs to run a HTTP server to host JS files, and browser will take
>> > care of page rendering, just like HDFS/Spark/Mesos UI.
>> >
>> > There're a couple of other Apache projects are using Ember.JS, such as
>> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
>> > pages, events and packages.
>> >
>> > Thanks,
>> > Wangda
>> >
>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
>> wrote:
>> >
>> >>
>> >> Hey folks.
>> >>
>> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
>> of
>> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
>> seem
>> >> significantly heavyweight? How hard is this going to be operationally
>> to
>> >> manage?
>>
Re: node.js and more as dependencies
Posted by Haohui Mai <ri...@gmail.com>.
+1 on adding npm / gulp / mocha to be part of the build dependencies
(not runtime). The main benefit is modularity -- for example, in the
HDFS UI we manually duplicate the navigation bars and footer. We don't
have unit tests for them due to the lack of infrastructure.
In my opinion Introducing npm will effectively bridge the gap.
However, I'm not entirely convinced by the Ember.js argument -- I
understand it might provide better integration with Ambari, but there
are clear trends that the industry is moving to a more reactive UI
design. I think the decision of using ember.js exclusively might worth
revisiting. To me it makes more sense to move both HDFS / YARN towards
React and go from there.
~Haohui
On Mon, Feb 29, 2016 at 5:14 PM, Wangda Tan <wh...@gmail.com> wrote:
> Hi Colin,
>
> Thanks for comment, I think your concerns are all valid but also arguable:
>
> First, YARN UI is different from HDFS UI, it is much more complex:
> 1. We have many data models, such as app/container/system/node, etc. UI of
> HDFS is more like a file explorer.
> 2. We plan to add more rich data visualization to YARN UI to make admin can
> easier identify what happened. For example, using sunburst map
> <http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
> cluster/queue/user/app, etc.
> 3. We need to get data from different sources with different format. For
> example, application's running container information stores at RM and
> finished container information stores at Timeline server. We need to get
> data from both daemon, normalize these data (because REST API is different)
> and aggregate them.
>
> Ember.js could simplify what we should do a lot:
> - It has great data store design so we can easily normalize data model from
> different sources with different formats (adapter)
> - It can easily bind data model with view, so any changes to data store
> (like application status updated) can trigger page re-rendering without any
> additional works.
> - Besides binding data with view, it can also bind data to other computed
> properties. For example, if property of a model relies on another model,
> all properties/models will be updated altogether.
> - Integrates bower/broccoli/watchman to help with package
> management/build/development.
> - For other benefits, please refer to Why Ember?
> <http://mutewinter.github.io/why_ember/#/slides> slides.
>
> The plan of nextgen YARN UI is not only inherit and prettify existing YARN
> UI. I hope it can let users can get deep insight of what happens in their
> cluster. As you said, a simple JS framework can also achieve what we wanna
> to do, but using well designed framework can avoid reinvent the wheel.
>
> Regarding to your concerns about JS compilation/compaction, I think it is
> not conflict with open source: In source folder (git repository), all code
> are readable. Compilation/compaction code only exists in released code. I
> agree that we don't need obfuscation at all, but source code compaction
> could increase performance a lot, we could have heavy rendering tasks, such
> as visualization from statuses of 10K+ applications. Just like Java code of
> Hadoop, no user will try to get source code from a running cluster :).
>
> I will make sure integration to Maven is as less as possible, we should
> only need one single sub module, and limit all changes in that module only.
>
> Please let me know if you have any other concerns.
>
> Thanks,
> Wangda
>
> On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
>
>> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>>
>> The main use-cases for "JS builds" seem to be if you want to minimize
>> or obfuscate your JS. Given that this is open source code,
>> obfuscation seems unnecessary. Given that it's a low-traffic
>> management interface, minimizing the JS seems like a premature
>> optimization.
>>
>> The HDFS user interface is based on dust.js, and it just requires JS
>> files to be copied into the correct location.
>>
>> Perhaps there are advantages to ember.js that I am missing. But
>> there's also a big advantage to not having to manage a node.js build
>> system separate from Maven and CMake. What do you think?
>>
>> best,
>> Colin
>>
>> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
>> > Hi Allen,
>> >
>> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
>> > Manager) to manage packages.
>> >
>> > One thing to clarify is: npm dependency is only required by build stage
>> (JS
>> > build is stitching source files and renaming variables). After JS build
>> > completes, there's no dependency of Node.JS any more. Server such as RM
>> > only needs to run a HTTP server to host JS files, and browser will take
>> > care of page rendering, just like HDFS/Spark/Mesos UI.
>> >
>> > There're a couple of other Apache projects are using Ember.JS, such as
>> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
>> > pages, events and packages.
>> >
>> > Thanks,
>> > Wangda
>> >
>> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
>> wrote:
>> >
>> >>
>> >> Hey folks.
>> >>
>> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
>> of
>> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
>> seem
>> >> significantly heavyweight? How hard is this going to be operationally
>> to
>> >> manage?
>>
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
Hi Colin,
Thanks for comment, I think your concerns are all valid but also arguable:
First, YARN UI is different from HDFS UI, it is much more complex:
1. We have many data models, such as app/container/system/node, etc. UI of
HDFS is more like a file explorer.
2. We plan to add more rich data visualization to YARN UI to make admin can
easier identify what happened. For example, using sunburst map
<http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
cluster/queue/user/app, etc.
3. We need to get data from different sources with different format. For
example, application's running container information stores at RM and
finished container information stores at Timeline server. We need to get
data from both daemon, normalize these data (because REST API is different)
and aggregate them.
Ember.js could simplify what we should do a lot:
- It has great data store design so we can easily normalize data model from
different sources with different formats (adapter)
- It can easily bind data model with view, so any changes to data store
(like application status updated) can trigger page re-rendering without any
additional works.
- Besides binding data with view, it can also bind data to other computed
properties. For example, if property of a model relies on another model,
all properties/models will be updated altogether.
- Integrates bower/broccoli/watchman to help with package
management/build/development.
- For other benefits, please refer to Why Ember?
<http://mutewinter.github.io/why_ember/#/slides> slides.
The plan of nextgen YARN UI is not only inherit and prettify existing YARN
UI. I hope it can let users can get deep insight of what happens in their
cluster. As you said, a simple JS framework can also achieve what we wanna
to do, but using well designed framework can avoid reinvent the wheel.
Regarding to your concerns about JS compilation/compaction, I think it is
not conflict with open source: In source folder (git repository), all code
are readable. Compilation/compaction code only exists in released code. I
agree that we don't need obfuscation at all, but source code compaction
could increase performance a lot, we could have heavy rendering tasks, such
as visualization from statuses of 10K+ applications. Just like Java code of
Hadoop, no user will try to get source code from a running cluster :).
I will make sure integration to Maven is as less as possible, we should
only need one single sub module, and limit all changes in that module only.
Please let me know if you have any other concerns.
Thanks,
Wangda
On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>
> The main use-cases for "JS builds" seem to be if you want to minimize
> or obfuscate your JS. Given that this is open source code,
> obfuscation seems unnecessary. Given that it's a low-traffic
> management interface, minimizing the JS seems like a premature
> optimization.
>
> The HDFS user interface is based on dust.js, and it just requires JS
> files to be copied into the correct location.
>
> Perhaps there are advantages to ember.js that I am missing. But
> there's also a big advantage to not having to manage a node.js build
> system separate from Maven and CMake. What do you think?
>
> best,
> Colin
>
> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
> > Hi Allen,
> >
> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
> > Manager) to manage packages.
> >
> > One thing to clarify is: npm dependency is only required by build stage
> (JS
> > build is stitching source files and renaming variables). After JS build
> > completes, there's no dependency of Node.JS any more. Server such as RM
> > only needs to run a HTTP server to host JS files, and browser will take
> > care of page rendering, just like HDFS/Spark/Mesos UI.
> >
> > There're a couple of other Apache projects are using Ember.JS, such as
> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
> > pages, events and packages.
> >
> > Thanks,
> > Wangda
> >
> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
> wrote:
> >
> >>
> >> Hey folks.
> >>
> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
> of
> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
> seem
> >> significantly heavyweight? How hard is this going to be operationally
> to
> >> manage?
>
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
Hi Colin,
Thanks for comment, I think your concerns are all valid but also arguable:
First, YARN UI is different from HDFS UI, it is much more complex:
1. We have many data models, such as app/container/system/node, etc. UI of
HDFS is more like a file explorer.
2. We plan to add more rich data visualization to YARN UI to make admin can
easier identify what happened. For example, using sunburst map
<http://bl.ocks.org/kerryrodden/7090426> to render usage breakdown of
cluster/queue/user/app, etc.
3. We need to get data from different sources with different format. For
example, application's running container information stores at RM and
finished container information stores at Timeline server. We need to get
data from both daemon, normalize these data (because REST API is different)
and aggregate them.
Ember.js could simplify what we should do a lot:
- It has great data store design so we can easily normalize data model from
different sources with different formats (adapter)
- It can easily bind data model with view, so any changes to data store
(like application status updated) can trigger page re-rendering without any
additional works.
- Besides binding data with view, it can also bind data to other computed
properties. For example, if property of a model relies on another model,
all properties/models will be updated altogether.
- Integrates bower/broccoli/watchman to help with package
management/build/development.
- For other benefits, please refer to Why Ember?
<http://mutewinter.github.io/why_ember/#/slides> slides.
The plan of nextgen YARN UI is not only inherit and prettify existing YARN
UI. I hope it can let users can get deep insight of what happens in their
cluster. As you said, a simple JS framework can also achieve what we wanna
to do, but using well designed framework can avoid reinvent the wheel.
Regarding to your concerns about JS compilation/compaction, I think it is
not conflict with open source: In source folder (git repository), all code
are readable. Compilation/compaction code only exists in released code. I
agree that we don't need obfuscation at all, but source code compaction
could increase performance a lot, we could have heavy rendering tasks, such
as visualization from statuses of 10K+ applications. Just like Java code of
Hadoop, no user will try to get source code from a running cluster :).
I will make sure integration to Maven is as less as possible, we should
only need one single sub module, and limit all changes in that module only.
Please let me know if you have any other concerns.
Thanks,
Wangda
On Mon, Feb 29, 2016 at 8:51 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Hmm. Devil's advocate here: Do we really need to have a "JS build"?
>
> The main use-cases for "JS builds" seem to be if you want to minimize
> or obfuscate your JS. Given that this is open source code,
> obfuscation seems unnecessary. Given that it's a low-traffic
> management interface, minimizing the JS seems like a premature
> optimization.
>
> The HDFS user interface is based on dust.js, and it just requires JS
> files to be copied into the correct location.
>
> Perhaps there are advantages to ember.js that I am missing. But
> there's also a big advantage to not having to manage a node.js build
> system separate from Maven and CMake. What do you think?
>
> best,
> Colin
>
> On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
> > Hi Allen,
> >
> > YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
> > Manager) to manage packages.
> >
> > One thing to clarify is: npm dependency is only required by build stage
> (JS
> > build is stitching source files and renaming variables). After JS build
> > completes, there's no dependency of Node.JS any more. Server such as RM
> > only needs to run a HTTP server to host JS files, and browser will take
> > care of page rendering, just like HDFS/Spark/Mesos UI.
> >
> > There're a couple of other Apache projects are using Ember.JS, such as
> > Tez/Ambari. Ember.JS can help front-end developers easier manage models,
> > pages, events and packages.
> >
> > Thanks,
> > Wangda
> >
> > On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com>
> wrote:
> >
> >>
> >> Hey folks.
> >>
> >> Have any of you looked at YARN-3368? Is adding node.js+a bunch
> of
> >> other stuff as dependencies just for the UI a good idea? Doesn’t that
> seem
> >> significantly heavyweight? How hard is this going to be operationally
> to
> >> manage?
>
Re: node.js and more as dependencies
Posted by "Colin P. McCabe" <cm...@apache.org>.
Hmm. Devil's advocate here: Do we really need to have a "JS build"?
The main use-cases for "JS builds" seem to be if you want to minimize
or obfuscate your JS. Given that this is open source code,
obfuscation seems unnecessary. Given that it's a low-traffic
management interface, minimizing the JS seems like a premature
optimization.
The HDFS user interface is based on dust.js, and it just requires JS
files to be copied into the correct location.
Perhaps there are advantages to ember.js that I am missing. But
there's also a big advantage to not having to manage a node.js build
system separate from Maven and CMake. What do you think?
best,
Colin
On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
> Hi Allen,
>
> YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
> Manager) to manage packages.
>
> One thing to clarify is: npm dependency is only required by build stage (JS
> build is stitching source files and renaming variables). After JS build
> completes, there's no dependency of Node.JS any more. Server such as RM
> only needs to run a HTTP server to host JS files, and browser will take
> care of page rendering, just like HDFS/Spark/Mesos UI.
>
> There're a couple of other Apache projects are using Ember.JS, such as
> Tez/Ambari. Ember.JS can help front-end developers easier manage models,
> pages, events and packages.
>
> Thanks,
> Wangda
>
> On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Hey folks.
>>
>> Have any of you looked at YARN-3368? Is adding node.js+a bunch of
>> other stuff as dependencies just for the UI a good idea? Doesn’t that seem
>> significantly heavyweight? How hard is this going to be operationally to
>> manage?
Re: node.js and more as dependencies
Posted by "Colin P. McCabe" <cm...@apache.org>.
Hmm. Devil's advocate here: Do we really need to have a "JS build"?
The main use-cases for "JS builds" seem to be if you want to minimize
or obfuscate your JS. Given that this is open source code,
obfuscation seems unnecessary. Given that it's a low-traffic
management interface, minimizing the JS seems like a premature
optimization.
The HDFS user interface is based on dust.js, and it just requires JS
files to be copied into the correct location.
Perhaps there are advantages to ember.js that I am missing. But
there's also a big advantage to not having to manage a node.js build
system separate from Maven and CMake. What do you think?
best,
Colin
On Thu, Feb 25, 2016 at 11:18 AM, Wangda Tan <wh...@gmail.com> wrote:
> Hi Allen,
>
> YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
> Manager) to manage packages.
>
> One thing to clarify is: npm dependency is only required by build stage (JS
> build is stitching source files and renaming variables). After JS build
> completes, there's no dependency of Node.JS any more. Server such as RM
> only needs to run a HTTP server to host JS files, and browser will take
> care of page rendering, just like HDFS/Spark/Mesos UI.
>
> There're a couple of other Apache projects are using Ember.JS, such as
> Tez/Ambari. Ember.JS can help front-end developers easier manage models,
> pages, events and packages.
>
> Thanks,
> Wangda
>
> On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Hey folks.
>>
>> Have any of you looked at YARN-3368? Is adding node.js+a bunch of
>> other stuff as dependencies just for the UI a good idea? Doesn’t that seem
>> significantly heavyweight? How hard is this going to be operationally to
>> manage?
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
Hi Allen,
YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
Manager) to manage packages.
One thing to clarify is: npm dependency is only required by build stage (JS
build is stitching source files and renaming variables). After JS build
completes, there's no dependency of Node.JS any more. Server such as RM
only needs to run a HTTP server to host JS files, and browser will take
care of page rendering, just like HDFS/Spark/Mesos UI.
There're a couple of other Apache projects are using Ember.JS, such as
Tez/Ambari. Ember.JS can help front-end developers easier manage models,
pages, events and packages.
Thanks,
Wangda
On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Hey folks.
>
> Have any of you looked at YARN-3368? Is adding node.js+a bunch of
> other stuff as dependencies just for the UI a good idea? Doesn’t that seem
> significantly heavyweight? How hard is this going to be operationally to
> manage?
Re: node.js and more as dependencies
Posted by Wangda Tan <wh...@gmail.com>.
Hi Allen,
YARN-3368 is using Ember.JS and Ember.JS depends on npm (Node.JS Package
Manager) to manage packages.
One thing to clarify is: npm dependency is only required by build stage (JS
build is stitching source files and renaming variables). After JS build
completes, there's no dependency of Node.JS any more. Server such as RM
only needs to run a HTTP server to host JS files, and browser will take
care of page rendering, just like HDFS/Spark/Mesos UI.
There're a couple of other Apache projects are using Ember.JS, such as
Tez/Ambari. Ember.JS can help front-end developers easier manage models,
pages, events and packages.
Thanks,
Wangda
On Thu, Feb 25, 2016 at 9:16 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Hey folks.
>
> Have any of you looked at YARN-3368? Is adding node.js+a bunch of
> other stuff as dependencies just for the UI a good idea? Doesn’t that seem
> significantly heavyweight? How hard is this going to be operationally to
> manage?