You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Haohui Mai <hm...@hortonworks.com> on 2014/09/12 00:29:13 UTC
[DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Hi all,
In HDFS, the code that relates to webhdfs and UIs has diverged quite a bit
between branch-2 and trunk. This complicates the processes of maintaining
webhdfs in several jiras (e.g., HDFS-5946)
Most of the divergences come from removing the JSP UI and the related clean
ups that are committed to trunk.
Though I myself have to take the blame of creating these divergences -- I
wonder, is it acceptable to resolve these divergences, by committing these
patches that remove JSP UI to branch-2? This would save us the pains of
providing two patches when fixing issues in webhdfs.
Note that the new HTML5-based UI is fairly mature, as it has been around
since 2.2, and it has been the default UI since 2.3. Compatibility,
however, might be a concern -- according to the compatibility guide, Hadoop
claims no compatibility for web UIs, thus we might be able to get away with
that.
Thoughts?
Regards,
Haohui
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Haohui Mai <hm...@hortonworks.com>.
bq. If it cost nothing, I'd be in favour, but if the ongoing maintenance is
high then I think that it has to be retired at some time. All that's
being argued
about is "when".
There are several types of costs associated with the JSP UIs. For
example, fixing
security issues (HDFS-4901), and fixing webhdfs issues twice due to the
divergences between trunk and branch-2. I have been one of the people
maintaining JSP UI and webhdfs since 2.2, personally I think it is nice to
get rid of the extra costs when it is appropriate.
I think this is the right time to retire the old UI to reduce the costs of
maintenance. The new UI has been around since 2.3 and it
has stabilized through several releases.
~Haohui
On Thu, Oct 2, 2014 at 10:01 AM, Steve Loughran <st...@hortonworks.com>
wrote:
> On 1 October 2014 17:25, Allen Wittenauer <aw...@altiscale.com> wrote:
> >
> > This is also an interesting precedent: Are we declaring that UIs are
> defacto non-stable? Does this mean we can break the output of, e.g., count
> or ls or whatever because they don't have stabilities associated with
> them? What line is to be drawn here? IMO, difficulty in moving code is
> NOT a sufficient reason to throw out the compatibility guarantees. It is
> only an indicator that we as a community are taking too long getting out
> releases.
> >
> > If we do this change, then it's pretty much open season on all
> of other UI-incompatible changes sitting in trunk (and there are a
> bunch….). Are we really ready to open the floodgates?
>
>
>
>
> http://hadoop.apache.org/docs/r2.5.1/hadoop-project-dist/hadoop-common/Compatibility.html#Web_UI
>
> Web UI
> ======
> Web UI, particularly the content and layout of web pages, changes
> could potentially interfere with attempts to screen scrape the web
> pages for information.
>
> Policy
> =====
>
> Web pages are not meant to be scraped and hence incompatible changes
> to them are allowed at any time. Users are expected to use REST APIs
> to get any information.
>
>
> we're saying "REST is stable; UI flexible"
>
> w.r.t browser simulation; HtmlUnit even handles javascript in its
> inside-JUnit4 test runs, negotiated connections &c. It can assess
> basic functionality. Real browser testing can be done with Selenium
>
> the risk that AW is pointing out on the NN UI is that it has been
> around and stable long enough that it predates webhdfs, and because
> of its JSP-free view, has been amenable to scraping. Some people
> probably are still using it. Does that mean we should keep it? If it
> cost nothing, I'd be in favour, but if the ongoing maintenance is high
> then I think that it has to be retired at some time. All that's being
> argued about is "when".
>
> (BTW, this pushes for a REST-first policy on all future web work. Do
> the stable API then make the human-friendly view a thin layer in front
> of it)
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Steve Loughran <st...@hortonworks.com>.
On 1 October 2014 17:25, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> This is also an interesting precedent: Are we declaring that UIs are defacto non-stable? Does this mean we can break the output of, e.g., count or ls or whatever because they don't have stabilities associated with them? What line is to be drawn here? IMO, difficulty in moving code is NOT a sufficient reason to throw out the compatibility guarantees. It is only an indicator that we as a community are taking too long getting out releases.
>
> If we do this change, then it's pretty much open season on all of other UI-incompatible changes sitting in trunk (and there are a bunch….). Are we really ready to open the floodgates?
http://hadoop.apache.org/docs/r2.5.1/hadoop-project-dist/hadoop-common/Compatibility.html#Web_UI
Web UI
======
Web UI, particularly the content and layout of web pages, changes
could potentially interfere with attempts to screen scrape the web
pages for information.
Policy
=====
Web pages are not meant to be scraped and hence incompatible changes
to them are allowed at any time. Users are expected to use REST APIs
to get any information.
we're saying "REST is stable; UI flexible"
w.r.t browser simulation; HtmlUnit even handles javascript in its
inside-JUnit4 test runs, negotiated connections &c. It can assess
basic functionality. Real browser testing can be done with Selenium
the risk that AW is pointing out on the NN UI is that it has been
around and stable long enough that it predates webhdfs, and because
of its JSP-free view, has been amenable to scraping. Some people
probably are still using it. Does that mean we should keep it? If it
cost nothing, I'd be in favour, but if the ongoing maintenance is high
then I think that it has to be retired at some time. All that's being
argued about is "when".
(BTW, this pushes for a REST-first policy on all future web work. Do
the stable API then make the human-friendly view a thin layer in front
of it)
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by Allen Wittenauer <aw...@altiscale.com>.
On Oct 1, 2014, at 6:19 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>
> bq. Are we declaring that UIs are defacto non-stable?
>
> UI has been declared non-stable explicitly.
... then game over.
Go forth and break it.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Andrew Wang <an...@cloudera.com>.
>
> IMO the scripting work you've done is a good example -- personally I really
> appreciate the work and I think it should be merged into branch-2 to
> benefit mainstream users when it's appropriate. Merging the work into
> branch-2 and starting the discussion of 3.x release, should be independent.
>
>
This actually is addressed in the compat guidelines, the shell script
rewrite is incompatible and thus shouldn't land in branch-2.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Haohui Mai <hm...@hortonworks.com>.
bq. Besides that obvious problem, how does one simulate HDFS browsing
(especially going through the web auth--which is the key part!) via JMX or
REST? Will it provide the guarantees that I know that users who are using
browsers are functional?
It can be tested in the same way that the UI is tested today. The test
needs to make sure JMX / webhdfs have authenticate the connections, which
is well covered today. In fact, parts of the JSP UI can take a separate
code path for authentication (e.g. HDFS-1017).
One motivation of the new UI work is to unify these code paths and to say
goodbye to these bugs from an architectural prospective.
bq. Are we declaring that UIs are defacto non-stable?
UI has been declared non-stable explicitly. Just to quote
Hadoop Compatibility Policy (
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
):
Web UI: Web UI, particularly the content and layout of web pages, changes
could potentially interfere with attempts to screen scrape the web pages
for information.
Policy: Web pages are not meant to be scraped and hence incompatible
changes to them are allowed at any time. Users are expected to use REST
APIs to get any information.
By the way, maybe I've missed some of your points, I'm somewhat confused by
your messages. I agree that it might be good to start the discussion of
3.x, the issue of removing the old UI from branch-2, however, is
orthogonal. UI is non-stable according to Hadoop Compatibility Policy, and
the new UI has been the default UI since 2.3.
Even for people that have workflows based upon scraping the Web UIs (I
doubt there are any), they should be scraping the new UI instead of the old
one by now.
I failed to understand the reasoning behind bundling this issue with the
3.x release -- though some work that has dramatic changes / refactoring is
not immediately merged to branch-2, these changes are actually quite good
and should be merged into branch-2 when it is stabilized.
IMO the scripting work you've done is a good example -- personally I really
appreciate the work and I think it should be merged into branch-2 to
benefit mainstream users when it's appropriate. Merging the work into
branch-2 and starting the discussion of 3.x release, should be independent.
~Haohui
On Wed, Oct 1, 2014 at 5:25 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> > Webui equivalent data has been exposed as jmx http APIs for a long time
>
> Not true. HDFS-5342 was allegedly committed to branch-2 less than
> a year ago.
>
> Besides that obvious problem, how does one simulate HDFS browsing
> (especially going through the web auth--which is the key part!) via JMX or
> REST? Will it provide the guarantees that I know that users who are using
> browsers are functional?
>
> This is also an interesting precedent: Are we declaring that UIs
> are defacto non-stable? Does this mean we can break the output of, e.g.,
> count or ls or whatever because they don't have stabilities associated with
> them? What line is to be drawn here? IMO, difficulty in moving code is
> NOT a sufficient reason to throw out the compatibility guarantees. It is
> only an indicator that we as a community are taking too long getting out
> releases.
>
> If we do this change, then it's pretty much open season on all of
> other UI-incompatible changes sitting in trunk (and there are a bunch….).
> Are we really ready to open the floodgates?
>
>
> On Oct 1, 2014, at 4:05 PM, Suresh Srinivas <su...@hortonworks.com>
> wrote:
>
> > Webui equivalent data has been exposed as jmx http APIs for a long time
> (I think from 0.20 release, almost 4 years ago). We have made many jsp
> changes that we have made that should have broken these applications many
> times. I believe this is a frivolous objection for a functionality that as
> a community we have never committed to keeping stable or backward
> compatible.
> >
> > Again I think given there is a clear and easier path for such uses to
> move to, I am +1 on this change.
> >
> > Sent from phone
> >
> >> On Oct 1, 2014, at 3:44 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
> >>
> >>
> >> OK, then consider this an official objection. Yes, people do have
> workflows based upon scraping the Web UIs, especially workflows that
> simulate what interfaces users actually use.
> >>
> >>> On Oct 1, 2014, at 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Thanks for the discussions. If there is no objection, I plan to merge
> the
> >>> related jiras from trunk to branch-2 in the coming days.
> >>>
> >>> ~Haohui
> >>>
> >>> On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <
> suresh@hortonworks.com>
> >>> wrote:
> >>>
> >>>> Sorry for responding very late. I am +1 on removing the JSP UI in
> branch-2
> >>>> for the following reasons:
> >>>> 1. The new web UI added has all the functionality of JSP UI.
> >>>> 2. For folks who might have used web UI to scrape the HDFS status, we
> have
> >>>> build a http JMX interface a long time ago.
> >>>>
> >>>> Given we do not guarantee the web UI as a stable and backward
> compatible
> >>>> interface, removing this should be okay.
> >>>>
> >>>>
> >>>>
> >>>>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com>
> wrote:
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> In HDFS, the code that relates to webhdfs and UIs has diverged quite
> a
> >>>> bit
> >>>>> between branch-2 and trunk. This complicates the processes of
> maintaining
> >>>>> webhdfs in several jiras (e.g., HDFS-5946)
> >>>>>
> >>>>> Most of the divergences come from removing the JSP UI and the related
> >>>> clean
> >>>>> ups that are committed to trunk.
> >>>>>
> >>>>> Though I myself have to take the blame of creating these divergences
> -- I
> >>>>> wonder, is it acceptable to resolve these divergences, by committing
> >>>> these
> >>>>> patches that remove JSP UI to branch-2? This would save us the pains
> of
> >>>>> providing two patches when fixing issues in webhdfs.
> >>>>>
> >>>>> Note that the new HTML5-based UI is fairly mature, as it has been
> around
> >>>>> since 2.2, and it has been the default UI since 2.3. Compatibility,
> >>>>> however, might be a concern -- according to the compatibility guide,
> >>>> Hadoop
> >>>>> claims no compatibility for web UIs, thus we might be able to get
> away
> >>>> with
> >>>>> that.
> >>>>>
> >>>>> Thoughts?
> >>>>>
> >>>>> Regards,
> >>>>> Haohui
> >>>>>
> >>>>> --
> >>>>> CONFIDENTIALITY NOTICE
> >>>>> NOTICE: This message is intended for the use of the individual or
> entity
> >>>> to
> >>>>> which it is addressed and may contain information that is
> confidential,
> >>>>> privileged and exempt from disclosure under applicable law. If the
> reader
> >>>>> of this message is not the intended recipient, you are hereby
> notified
> >>>> that
> >>>>> any printing, copying, dissemination, distribution, disclosure or
> >>>>> forwarding of this communication is strictly prohibited. If you have
> >>>>> received this communication in error, please contact the sender
> >>>> immediately
> >>>>> and delete it from your system. Thank You.
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> http://hortonworks.com/download/
> >>>>
> >>>> --
> >>>> CONFIDENTIALITY NOTICE
> >>>> NOTICE: This message is intended for the use of the individual or
> entity to
> >>>> which it is addressed and may contain information that is
> confidential,
> >>>> privileged and exempt from disclosure under applicable law. If the
> reader
> >>>> of this message is not the intended recipient, you are hereby
> notified that
> >>>> any printing, copying, dissemination, distribution, disclosure or
> >>>> forwarding of this communication is strictly prohibited. If you have
> >>>> received this communication in error, please contact the sender
> immediately
> >>>> and delete it from your system. Thank You.
> >>>
> >>> --
> >>> CONFIDENTIALITY NOTICE
> >>> NOTICE: This message is intended for the use of the individual or
> entity to
> >>> which it is addressed and may contain information that is confidential,
> >>> privileged and exempt from disclosure under applicable law. If the
> reader
> >>> of this message is not the intended recipient, you are hereby notified
> that
> >>> any printing, copying, dissemination, distribution, disclosure or
> >>> forwarding of this communication is strictly prohibited. If you have
> >>> received this communication in error, please contact the sender
> immediately
> >>> and delete it from your system. Thank You.
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by Allen Wittenauer <aw...@altiscale.com>.
> Webui equivalent data has been exposed as jmx http APIs for a long time
Not true. HDFS-5342 was allegedly committed to branch-2 less than a year ago.
Besides that obvious problem, how does one simulate HDFS browsing (especially going through the web auth--which is the key part!) via JMX or REST? Will it provide the guarantees that I know that users who are using browsers are functional?
This is also an interesting precedent: Are we declaring that UIs are defacto non-stable? Does this mean we can break the output of, e.g., count or ls or whatever because they don't have stabilities associated with them? What line is to be drawn here? IMO, difficulty in moving code is NOT a sufficient reason to throw out the compatibility guarantees. It is only an indicator that we as a community are taking too long getting out releases.
If we do this change, then it's pretty much open season on all of other UI-incompatible changes sitting in trunk (and there are a bunch….). Are we really ready to open the floodgates?
On Oct 1, 2014, at 4:05 PM, Suresh Srinivas <su...@hortonworks.com> wrote:
> Webui equivalent data has been exposed as jmx http APIs for a long time (I think from 0.20 release, almost 4 years ago). We have made many jsp changes that we have made that should have broken these applications many times. I believe this is a frivolous objection for a functionality that as a community we have never committed to keeping stable or backward compatible.
>
> Again I think given there is a clear and easier path for such uses to move to, I am +1 on this change.
>
> Sent from phone
>
>> On Oct 1, 2014, at 3:44 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>>
>>
>> OK, then consider this an official objection. Yes, people do have workflows based upon scraping the Web UIs, especially workflows that simulate what interfaces users actually use.
>>
>>> On Oct 1, 2014, at 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>>
>>> Hi,
>>>
>>> Thanks for the discussions. If there is no objection, I plan to merge the
>>> related jiras from trunk to branch-2 in the coming days.
>>>
>>> ~Haohui
>>>
>>> On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <su...@hortonworks.com>
>>> wrote:
>>>
>>>> Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
>>>> for the following reasons:
>>>> 1. The new web UI added has all the functionality of JSP UI.
>>>> 2. For folks who might have used web UI to scrape the HDFS status, we have
>>>> build a http JMX interface a long time ago.
>>>>
>>>> Given we do not guarantee the web UI as a stable and backward compatible
>>>> interface, removing this should be okay.
>>>>
>>>>
>>>>
>>>>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> In HDFS, the code that relates to webhdfs and UIs has diverged quite a
>>>> bit
>>>>> between branch-2 and trunk. This complicates the processes of maintaining
>>>>> webhdfs in several jiras (e.g., HDFS-5946)
>>>>>
>>>>> Most of the divergences come from removing the JSP UI and the related
>>>> clean
>>>>> ups that are committed to trunk.
>>>>>
>>>>> Though I myself have to take the blame of creating these divergences -- I
>>>>> wonder, is it acceptable to resolve these divergences, by committing
>>>> these
>>>>> patches that remove JSP UI to branch-2? This would save us the pains of
>>>>> providing two patches when fixing issues in webhdfs.
>>>>>
>>>>> Note that the new HTML5-based UI is fairly mature, as it has been around
>>>>> since 2.2, and it has been the default UI since 2.3. Compatibility,
>>>>> however, might be a concern -- according to the compatibility guide,
>>>> Hadoop
>>>>> claims no compatibility for web UIs, thus we might be able to get away
>>>> with
>>>>> that.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Haohui
>>>>>
>>>>> --
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or entity
>>>> to
>>>>> which it is addressed and may contain information that is confidential,
>>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>> and delete it from your system. Thank You.
>>>>
>>>>
>>>>
>>>> --
>>>> http://hortonworks.com/download/
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender immediately
>>>> and delete it from your system. Thank You.
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Jitendra Pandey <ji...@hortonworks.com>.
+1 for removing the old UI.
On Wed, Oct 1, 2014 at 4:05 PM, Suresh Srinivas <su...@hortonworks.com>
wrote:
> Webui equivalent data has been exposed as jmx http APIs for a long time (I
> think from 0.20 release, almost 4 years ago). We have made many jsp changes
> that we have made that should have broken these applications many times. I
> believe this is a frivolous objection for a functionality that as a
> community we have never committed to keeping stable or backward compatible.
>
> Again I think given there is a clear and easier path for such uses to move
> to, I am +1 on this change.
>
> Sent from phone
>
> > On Oct 1, 2014, at 3:44 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
> >
> >
> > OK, then consider this an official objection. Yes, people do have
> workflows based upon scraping the Web UIs, especially workflows that
> simulate what interfaces users actually use.
> >
> >> On Oct 1, 2014, at 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
> >>
> >> Hi,
> >>
> >> Thanks for the discussions. If there is no objection, I plan to merge
> the
> >> related jiras from trunk to branch-2 in the coming days.
> >>
> >> ~Haohui
> >>
> >> On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <suresh@hortonworks.com
> >
> >> wrote:
> >>
> >>> Sorry for responding very late. I am +1 on removing the JSP UI in
> branch-2
> >>> for the following reasons:
> >>> 1. The new web UI added has all the functionality of JSP UI.
> >>> 2. For folks who might have used web UI to scrape the HDFS status, we
> have
> >>> build a http JMX interface a long time ago.
> >>>
> >>> Given we do not guarantee the web UI as a stable and backward
> compatible
> >>> interface, removing this should be okay.
> >>>
> >>>
> >>>
> >>>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com>
> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> In HDFS, the code that relates to webhdfs and UIs has diverged quite a
> >>> bit
> >>>> between branch-2 and trunk. This complicates the processes of
> maintaining
> >>>> webhdfs in several jiras (e.g., HDFS-5946)
> >>>>
> >>>> Most of the divergences come from removing the JSP UI and the related
> >>> clean
> >>>> ups that are committed to trunk.
> >>>>
> >>>> Though I myself have to take the blame of creating these divergences
> -- I
> >>>> wonder, is it acceptable to resolve these divergences, by committing
> >>> these
> >>>> patches that remove JSP UI to branch-2? This would save us the pains
> of
> >>>> providing two patches when fixing issues in webhdfs.
> >>>>
> >>>> Note that the new HTML5-based UI is fairly mature, as it has been
> around
> >>>> since 2.2, and it has been the default UI since 2.3. Compatibility,
> >>>> however, might be a concern -- according to the compatibility guide,
> >>> Hadoop
> >>>> claims no compatibility for web UIs, thus we might be able to get away
> >>> with
> >>>> that.
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> Regards,
> >>>> Haohui
> >>>>
> >>>> --
> >>>> CONFIDENTIALITY NOTICE
> >>>> NOTICE: This message is intended for the use of the individual or
> entity
> >>> to
> >>>> which it is addressed and may contain information that is
> confidential,
> >>>> privileged and exempt from disclosure under applicable law. If the
> reader
> >>>> of this message is not the intended recipient, you are hereby notified
> >>> that
> >>>> any printing, copying, dissemination, distribution, disclosure or
> >>>> forwarding of this communication is strictly prohibited. If you have
> >>>> received this communication in error, please contact the sender
> >>> immediately
> >>>> and delete it from your system. Thank You.
> >>>
> >>>
> >>>
> >>> --
> >>> http://hortonworks.com/download/
> >>>
> >>> --
> >>> CONFIDENTIALITY NOTICE
> >>> NOTICE: This message is intended for the use of the individual or
> entity to
> >>> which it is addressed and may contain information that is confidential,
> >>> privileged and exempt from disclosure under applicable law. If the
> reader
> >>> of this message is not the intended recipient, you are hereby notified
> that
> >>> any printing, copying, dissemination, distribution, disclosure or
> >>> forwarding of this communication is strictly prohibited. If you have
> >>> received this communication in error, please contact the sender
> immediately
> >>> and delete it from your system. Thank You.
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
<http://hortonworks.com/download/>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
Webui equivalent data has been exposed as jmx http APIs for a long time (I think from 0.20 release, almost 4 years ago). We have made many jsp changes that we have made that should have broken these applications many times. I believe this is a frivolous objection for a functionality that as a community we have never committed to keeping stable or backward compatible.
Again I think given there is a clear and easier path for such uses to move to, I am +1 on this change.
Sent from phone
> On Oct 1, 2014, at 3:44 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
> OK, then consider this an official objection. Yes, people do have workflows based upon scraping the Web UIs, especially workflows that simulate what interfaces users actually use.
>
>> On Oct 1, 2014, at 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>
>> Hi,
>>
>> Thanks for the discussions. If there is no objection, I plan to merge the
>> related jiras from trunk to branch-2 in the coming days.
>>
>> ~Haohui
>>
>> On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>>> Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
>>> for the following reasons:
>>> 1. The new web UI added has all the functionality of JSP UI.
>>> 2. For folks who might have used web UI to scrape the HDFS status, we have
>>> build a http JMX interface a long time ago.
>>>
>>> Given we do not guarantee the web UI as a stable and backward compatible
>>> interface, removing this should be okay.
>>>
>>>
>>>
>>>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> In HDFS, the code that relates to webhdfs and UIs has diverged quite a
>>> bit
>>>> between branch-2 and trunk. This complicates the processes of maintaining
>>>> webhdfs in several jiras (e.g., HDFS-5946)
>>>>
>>>> Most of the divergences come from removing the JSP UI and the related
>>> clean
>>>> ups that are committed to trunk.
>>>>
>>>> Though I myself have to take the blame of creating these divergences -- I
>>>> wonder, is it acceptable to resolve these divergences, by committing
>>> these
>>>> patches that remove JSP UI to branch-2? This would save us the pains of
>>>> providing two patches when fixing issues in webhdfs.
>>>>
>>>> Note that the new HTML5-based UI is fairly mature, as it has been around
>>>> since 2.2, and it has been the default UI since 2.3. Compatibility,
>>>> however, might be a concern -- according to the compatibility guide,
>>> Hadoop
>>>> claims no compatibility for web UIs, thus we might be able to get away
>>> with
>>>> that.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards,
>>>> Haohui
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>
>>>
>>>
>>> --
>>> http://hortonworks.com/download/
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by Allen Wittenauer <aw...@altiscale.com>.
OK, then consider this an official objection. Yes, people do have workflows based upon scraping the Web UIs, especially workflows that simulate what interfaces users actually use.
On Oct 1, 2014, at 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
> Hi,
>
> Thanks for the discussions. If there is no objection, I plan to merge the
> related jiras from trunk to branch-2 in the coming days.
>
> ~Haohui
>
> On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <su...@hortonworks.com>
> wrote:
>
>> Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
>> for the following reasons:
>> 1. The new web UI added has all the functionality of JSP UI.
>> 2. For folks who might have used web UI to scrape the HDFS status, we have
>> build a http JMX interface a long time ago.
>>
>> Given we do not guarantee the web UI as a stable and backward compatible
>> interface, removing this should be okay.
>>
>>
>>
>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>
>>> Hi all,
>>>
>>> In HDFS, the code that relates to webhdfs and UIs has diverged quite a
>> bit
>>> between branch-2 and trunk. This complicates the processes of maintaining
>>> webhdfs in several jiras (e.g., HDFS-5946)
>>>
>>> Most of the divergences come from removing the JSP UI and the related
>> clean
>>> ups that are committed to trunk.
>>>
>>> Though I myself have to take the blame of creating these divergences -- I
>>> wonder, is it acceptable to resolve these divergences, by committing
>> these
>>> patches that remove JSP UI to branch-2? This would save us the pains of
>>> providing two patches when fixing issues in webhdfs.
>>>
>>> Note that the new HTML5-based UI is fairly mature, as it has been around
>>> since 2.2, and it has been the default UI since 2.3. Compatibility,
>>> however, might be a concern -- according to the compatibility guide,
>> Hadoop
>>> claims no compatibility for web UIs, thus we might be able to get away
>> with
>>> that.
>>>
>>> Thoughts?
>>>
>>> Regards,
>>> Haohui
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>>>
>>
>>
>>
>> --
>> http://hortonworks.com/download/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com.INVALID>.
+1 on removing the old JSP UI. Thanks.
Tsz-Wo
On Wednesday, October 1, 2014 3:34 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>
>
>Hi,
>
>Thanks for the discussions. If there is no objection, I plan to merge the
>related jiras from trunk to branch-2 in the coming days.
>
>~Haohui
>
>On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <su...@hortonworks.com>
>wrote:
>
>> Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
>> for the following reasons:
>> 1. The new web UI added has all the functionality of JSP UI.
>> 2. For folks who might have used web UI to scrape the HDFS status, we have
>> build a http JMX interface a long time ago.
>>
>> Given we do not guarantee the web UI as a stable and backward compatible
>> interface, removing this should be okay.
>>
>>
>>
>> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>>
>> > Hi all,
>> >
>> > In HDFS, the code that relates to webhdfs and UIs has diverged quite a
>> bit
>> > between branch-2 and trunk. This complicates the processes of maintaining
>> > webhdfs in several jiras (e.g., HDFS-5946)
>> >
>> > Most of the divergences come from removing the JSP UI and the related
>> clean
>> > ups that are committed to trunk.
>> >
>> > Though I myself have to take the blame of creating these divergences -- I
>> > wonder, is it acceptable to resolve these divergences, by committing
>> these
>> > patches that remove JSP UI to branch-2? This would save us the pains of
>> > providing two patches when fixing issues in webhdfs.
>> >
>> > Note that the new HTML5-based UI is fairly mature, as it has been around
>> > since 2.2, and it has been the default UI since 2.3. Compatibility,
>> > however, might be a concern -- according to the compatibility guide,
>> Hadoop
>> > claims no compatibility for web UIs, thus we might be able to get away
>> with
>> > that.
>> >
>> > Thoughts?
>> >
>> > Regards,
>> > Haohui
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>>
>>
>> --
>> http://hortonworks.com/download/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>--
>CONFIDENTIALITY NOTICE
>NOTICE: This message is intended for the use of the individual or entity to
>which it is addressed and may contain information that is confidential,
>privileged and exempt from disclosure under applicable law. If the reader
>of this message is not the intended recipient, you are hereby notified that
>any printing, copying, dissemination, distribution, disclosure or
>forwarding of this communication is strictly prohibited. If you have
>received this communication in error, please contact the sender immediately
>and delete it from your system. Thank You.
>
>
>
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Haohui Mai <hm...@hortonworks.com>.
Hi,
Thanks for the discussions. If there is no objection, I plan to merge the
related jiras from trunk to branch-2 in the coming days.
~Haohui
On Wed, Oct 1, 2014 at 2:35 PM, Suresh Srinivas <su...@hortonworks.com>
wrote:
> Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
> for the following reasons:
> 1. The new web UI added has all the functionality of JSP UI.
> 2. For folks who might have used web UI to scrape the HDFS status, we have
> build a http JMX interface a long time ago.
>
> Given we do not guarantee the web UI as a stable and backward compatible
> interface, removing this should be okay.
>
>
>
> On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
>
> > Hi all,
> >
> > In HDFS, the code that relates to webhdfs and UIs has diverged quite a
> bit
> > between branch-2 and trunk. This complicates the processes of maintaining
> > webhdfs in several jiras (e.g., HDFS-5946)
> >
> > Most of the divergences come from removing the JSP UI and the related
> clean
> > ups that are committed to trunk.
> >
> > Though I myself have to take the blame of creating these divergences -- I
> > wonder, is it acceptable to resolve these divergences, by committing
> these
> > patches that remove JSP UI to branch-2? This would save us the pains of
> > providing two patches when fixing issues in webhdfs.
> >
> > Note that the new HTML5-based UI is fairly mature, as it has been around
> > since 2.2, and it has been the default UI since 2.3. Compatibility,
> > however, might be a concern -- according to the compatibility guide,
> Hadoop
> > claims no compatibility for web UIs, thus we might be able to get away
> with
> > that.
> >
> > Thoughts?
> >
> > Regards,
> > Haohui
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> http://hortonworks.com/download/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between
branch-2 and trunk
Posted by Suresh Srinivas <su...@hortonworks.com>.
Sorry for responding very late. I am +1 on removing the JSP UI in branch-2
for the following reasons:
1. The new web UI added has all the functionality of JSP UI.
2. For folks who might have used web UI to scrape the HDFS status, we have
build a http JMX interface a long time ago.
Given we do not guarantee the web UI as a stable and backward compatible
interface, removing this should be okay.
On Thu, Sep 11, 2014 at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
> Hi all,
>
> In HDFS, the code that relates to webhdfs and UIs has diverged quite a bit
> between branch-2 and trunk. This complicates the processes of maintaining
> webhdfs in several jiras (e.g., HDFS-5946)
>
> Most of the divergences come from removing the JSP UI and the related clean
> ups that are committed to trunk.
>
> Though I myself have to take the blame of creating these divergences -- I
> wonder, is it acceptable to resolve these divergences, by committing these
> patches that remove JSP UI to branch-2? This would save us the pains of
> providing two patches when fixing issues in webhdfs.
>
> Note that the new HTML5-based UI is fairly mature, as it has been around
> since 2.2, and it has been the default UI since 2.3. Compatibility,
> however, might be a concern -- according to the compatibility guide, Hadoop
> claims no compatibility for web UIs, thus we might be able to get away with
> that.
>
> Thoughts?
>
> Regards,
> Haohui
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [DISCUSS] Resolving the divergences for web-related code between branch-2 and trunk
Posted by Allen Wittenauer <aw...@altiscale.com>.
On Sep 11, 2014, at 3:29 PM, Haohui Mai <hm...@hortonworks.com> wrote:
> Though I myself have to take the blame of creating these divergences -- I
> wonder, is it acceptable to resolve these divergences, by committing these
> patches that remove JSP UI to branch-2?
...
>
> Thoughts?
a) We have three years of patches sitting in trunk.
b) What makes this code harder to deal with than other incompatible changes?
c) We have three years of patches sitting in trunk.
d) Why is the 2.6 change list so fricking huge?
e) We have three years of patches sitting in trunk.
f) We as a community clearly learned nothing about the fiasco that was 1.x -> 2.x.
g) We have three years of patches sitting in trunk.
To summarize:
Let's stop shipping branch-2 (so no 2.6 release) and ship trunk instead. Some ops teams I've talked to have basically said that 2.6 is looking to be a release to skip because it is bound to broken with as much churn has happened in it. Let's bite the bullet, start getting serious about 3.x and work on that instead. There is more than enough "new stuff" in it, ignoring 2.6, to make it a viable major release.