You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jeff Hammerbacher (JIRA)" <ji...@apache.org> on 2010/09/24 10:56:34 UTC
[jira] Created: (HIVE-1668) Move HWI out to Github
Move HWI out to Github
----------------------
Key: HIVE-1668
URL: https://issues.apache.org/jira/browse/HIVE-1668
Project: Hadoop Hive
Issue Type: Improvement
Components: Web UI
Reporter: Jeff Hammerbacher
I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914584#action_12914584 ]
Edward Capriolo commented on HIVE-1668:
---------------------------------------
Jeff,
I disagree. The build and test errors are not insurmountable. In fact some if not most of the "ERRORS" were cascading changes that were not tested properly. For example:
https://issues.apache.org/jira/browse/HIVE-1183 was a fix I had to do because someone broke it here. https://issues.apache.org/jira/browse/HIVE-978 because someone wanted all jars to be named whatever.${version} and did not bother to look across all the shell script files that startup hive.
https://issues.apache.org/jira/browse/HIVE-1294 again someone changed some shell scripts and only tested the cli.
https://issues.apache.org/jira/browse/HIVE-752 again someone broke hwi without testing it.
https://issues.apache.org/jira/browse/HIVE-1615, not really anyone's fault but no API stability across hive. I do not see why one method went away and another similar method took its place.
I have been of course talking about moving HWI to wikit for a while moving from JSP to Servlet/ Java code will fix errors, but the little time I do have I usually have to spend detecting and cleaning up other breakages.
HUE and Beeswax I honestly do not know, but sounds like you need extra magical stuff to make this work, and HWI works with hive on its own (onless people break it)
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914741#action_12914741 ]
Edward Capriolo commented on HIVE-1668:
---------------------------------------
{quote}It should also help mature the product for eventual inclusion in trunk.{quote}
Why would we move something from hive out to github, just to move it back to hive?
{quote}Empirically, they don't. The value of the web interface to users is not nearly as high as the pain it causes the developers for maintenance.{quote}
Who are these developers who maintain it? Has anyone every added a feature beside me? I'm not complaining.
http://blog.milford.io/2010/06/getting-the-hive-web-interface-hwi-to-work-on-centos/
{quote}The Hive Web Interface is a pretty sweet deal.{quote}
Sounds like people like it.
Why are we debating the past state of hwi? It works now. If someone reports a bug I typically investigate and patch that same day.
I challenge anyone to open a ticket on core user, called "remove name node web interface to github" and tried to say "XXXX now offers a better name node interface using python." The ticket would instantly get a "RESOLVED: WILL NOT FIX". Why is this any different?
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914669#action_12914669 ]
Edward Capriolo commented on HIVE-1668:
---------------------------------------
{quote}That's not a great argument for keeping code that's onerous to maintain in trunk.{quote}
Its not onerous to maintain. As you can see from the tickets I pointed out it broke because it was not tested.
For example,
https://issues.apache.org/jira/browse/HIVE-752 when designing SHIM classes that specify a classname in a string, one has to make sure they get the class name correct. I know it was an over site, but I am sure someone fired up the CLI and made sure the class name was correct.
As for https://issues.apache.org/jira/browse/HIVE-978, I specifically mentioned how to test this any why it should be tested in the patch and it still turned out not to work right.
pragmatic is the perfect word. HWI was never made to be fancy. Anyone who has hive can build and run the web interface. With no extra dependencies. It looks like to use Beeswax you need Hue, which means you need to go somewhere else and get it and install it. It seems like you need to patch or load extra plugins to your namenode and datanode like org.apache.hadoop.thriftfs.NamenodePlugin, It looks like (http://archive.cloudera.com/cdh/3/hue/manual.html#_install_hue) you need:
gcc gcc
libxml2-devel libxml2-dev
libxslt-devel libxslt-dev
mysql-devel librarysqlclient-dev
python-devel python-dev
python-setuptools python-setuptools
sqlite-devel libsqlite3-dev
The pragmatic approach, is to use the web interface provided by hive. You do not need anything external like python, or have to make any changes to their environment. That is why I think we should stay part of the hive distribution.
I'm -1 on taking it out.
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Jay Booth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914577#action_12914577 ]
Jay Booth commented on HIVE-1668:
---------------------------------
Are you guys contributing Hue to ASF? It seems hasty to remove functionality in favor of a replacement if that replacement isn't going to be shipped with mainline Hive.
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914653#action_12914653 ]
Jeff Hammerbacher commented on HIVE-1668:
-----------------------------------------
bq. Are you guys contributing Hue to ASF?
No.
bq. It seems hasty to remove functionality in favor of a replacement if that replacement isn't going to be shipped with mainline Hive.
There's certainly precedent in other projects. For example, indexed HBase was moved out to Github for very similar reasons: while it provided a useful feature, it did so in a somewhat flaky way.
bq. I have learned to roll with the punches.
That's not a great argument for keeping code that's onerous to maintain in trunk.
Just trying to be pragmatic here.
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914605#action_12914605 ]
Edward Capriolo commented on HIVE-1668:
---------------------------------------
Plus, not to get too far off topic, but there is a huge portion of the hadoop community that thinks "Security? So what? Who cares?" I am not going to run Active Directory or Kerberos just so I can say "My hadoop is is secure" . It adds latency to many processes, complexity to the overall design of hadoop, and does not even encrypt data in transit. Many people are going to elect not to use "hadoop security" for those reasons. Is "extra work" a reason not to do something? Are we going to move the Hive Thrift server out to github too because of the burden of "extra work"? It is a lot of extra work for me when hadoop renames all its jmx counters or tells me "all my code is deprecated because of our new slick mapreduce.* api". I have learned to roll with the punches.
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914673#action_12914673 ]
Jeff Hammerbacher commented on HIVE-1668:
-----------------------------------------
bq. Anyone who has hive can build and run the web interface.
Empirically, they don't. The value of the web interface to users is not nearly as high as the pain it causes the developers for maintenance.
Moving it to Github will help determine if there's demand for the interface. It should also help mature the product for eventual inclusion in trunk.
> Move HWI out to Github
> ----------------------
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
> Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going to cost us some extra work for integration with security. We've worked on hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.