You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by st...@apache.org on 2013/04/06 08:08:58 UTC
svn commit: r1465200 [9/21] - in /hbase/hbase.apache.org/trunk: ./ book/
case_studies/ community/ configuration/ css/ developer/ getting_started/
images/ ops_mgt/ performance/ rpc/ schema_design/ security/ shell/
troubleshooting/ upgrading/
Modified: hbase/hbase.apache.org/trunk/case_studies/casestudies.perftroub.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/case_studies/casestudies.perftroub.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/case_studies/casestudies.perftroub.html (original)
+++ hbase/hbase.apache.org/trunk/case_studies/casestudies.perftroub.html Sat Apr 6 06:08:56 2013
@@ -1,33 +1,33 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>1.3. Performance/Troubleshooting</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="up" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="prev" href="casestudies.schema.html" title="1.2. Schema Design"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">1.3. Performance/Troubleshooting</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="casestudies.schema.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> </td></tr></table><hr></div><div class="section" title="1.3. Performance/Troubleshooti
ng"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="casestudies.perftroub"></a>1.3. Performance/Troubleshooting</h2></div></div></div><div class="section" title="1.3.1. Case Study #1 (Performance Issue On A Single Node)"><div class="titlepage"><div><div><h3 class="title"><a name="casestudies.slownode"></a>1.3.1. Case Study #1 (Performance Issue On A Single Node)</h3></div></div></div><div class="section" title="1.3.1.1. Scenario"><div class="titlepage"><div><div><h4 class="title"><a name="d2403e82"></a>1.3.1.1. Scenario</h4></div></div></div><p>Following a scheduled reboot, one data node began exhibiting unusual behavior. Routine MapReduce
+ <title>1.3. Performance/Troubleshooting</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="up" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="prev" href="casestudies.schema.html" title="1.2. Schema Design"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">1.3. Performance/Troubleshooting</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="casestudies.schema.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> </td></tr></table><hr></div><div class="section" title="1.3. Performance/Troubleshooti
ng"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="casestudies.perftroub"></a>1.3. Performance/Troubleshooting</h2></div></div></div><div class="section" title="1.3.1. Case Study #1 (Performance Issue On A Single Node)"><div class="titlepage"><div><div><h3 class="title"><a name="casestudies.slownode"></a>1.3.1. Case Study #1 (Performance Issue On A Single Node)</h3></div></div></div><div class="section" title="1.3.1.1. Scenario"><div class="titlepage"><div><div><h4 class="title"><a name="d2465e30"></a>1.3.1.1. Scenario</h4></div></div></div><p>Following a scheduled reboot, one data node began exhibiting unusual behavior. Routine MapReduce
jobs run against HBase tables which regularly completed in five or six minutes began taking 30 or 40 minutes
to finish. These jobs were consistently found to be waiting on map and reduce tasks assigned to the troubled data node
(e.g., the slow map tasks all had the same Input Split).
The situation came to a head during a distributed copy, when the copy was severely prolonged by the lagging node.
- </p></div><div class="section" title="1.3.1.2. Hardware"><div class="titlepage"><div><div><h4 class="title"><a name="d2403e87"></a>1.3.1.2. Hardware</h4></div></div></div><p>Datanodes:
+ </p></div><div class="section" title="1.3.1.2. Hardware"><div class="titlepage"><div><div><h4 class="title"><a name="d2465e35"></a>1.3.1.2. Hardware</h4></div></div></div><p>Datanodes:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem">Two 12-core processors</li><li class="listitem">Six Enerprise SATA disks</li><li class="listitem">24GB of RAM</li><li class="listitem">Two bonded gigabit NICs</li></ul></div><p>
</p><p>Network:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem">10 Gigabit top-of-rack switches</li><li class="listitem">20 Gigabit bonded interconnects between racks.</li></ul></div><p>
- </p></div><div class="section" title="1.3.1.3. Hypotheses"><div class="titlepage"><div><div><h4 class="title"><a name="d2403e110"></a>1.3.1.3. Hypotheses</h4></div></div></div><div class="section" title="1.3.1.3.1. HBase "Hot Spot" Region"><div class="titlepage"><div><div><h5 class="title"><a name="d2403e113"></a>1.3.1.3.1. HBase "Hot Spot" Region</h5></div></div></div><p>We hypothesized that we were experiencing a familiar point of pain: a "hot spot" region in an HBase table,
+ </p></div><div class="section" title="1.3.1.3. Hypotheses"><div class="titlepage"><div><div><h4 class="title"><a name="d2465e58"></a>1.3.1.3. Hypotheses</h4></div></div></div><div class="section" title="1.3.1.3.1. HBase "Hot Spot" Region"><div class="titlepage"><div><div><h5 class="title"><a name="d2465e61"></a>1.3.1.3.1. HBase "Hot Spot" Region</h5></div></div></div><p>We hypothesized that we were experiencing a familiar point of pain: a "hot spot" region in an HBase table,
where uneven key-space distribution can funnel a huge number of requests to a single HBase region, bombarding the RegionServer
process and cause slow response time. Examination of the HBase Master status page showed that the number of HBase requests to the
troubled node was almost zero. Further, examination of the HBase logs showed that there were no region splits, compactions, or other region transitions
in progress. This effectively ruled out a "hot spot" as the root cause of the observed slowness.
- </p></div><div class="section" title="1.3.1.3.2. HBase Region With Non-Local Data"><div class="titlepage"><div><div><h5 class="title"><a name="d2403e118"></a>1.3.1.3.2. HBase Region With Non-Local Data</h5></div></div></div><p>Our next hypothesis was that one of the MapReduce tasks was requesting data from HBase that was not local to the datanode, thus
+ </p></div><div class="section" title="1.3.1.3.2. HBase Region With Non-Local Data"><div class="titlepage"><div><div><h5 class="title"><a name="d2465e66"></a>1.3.1.3.2. HBase Region With Non-Local Data</h5></div></div></div><p>Our next hypothesis was that one of the MapReduce tasks was requesting data from HBase that was not local to the datanode, thus
forcing HDFS to request data blocks from other servers over the network. Examination of the datanode logs showed that there were very
few blocks being requested over the network, indicating that the HBase region was correctly assigned, and that the majority of the necessary
data was located on the node. This ruled out the possibility of non-local data causing a slowdown.
- </p></div><div class="section" title="1.3.1.3.3. Excessive I/O Wait Due To Swapping Or An Over-Worked Or Failing Hard Disk"><div class="titlepage"><div><div><h5 class="title"><a name="d2403e123"></a>1.3.1.3.3. Excessive I/O Wait Due To Swapping Or An Over-Worked Or Failing Hard Disk</h5></div></div></div><p>After concluding that the Hadoop and HBase were not likely to be the culprits, we moved on to troubleshooting the datanode's hardware.
+ </p></div><div class="section" title="1.3.1.3.3. Excessive I/O Wait Due To Swapping Or An Over-Worked Or Failing Hard Disk"><div class="titlepage"><div><div><h5 class="title"><a name="d2465e71"></a>1.3.1.3.3. Excessive I/O Wait Due To Swapping Or An Over-Worked Or Failing Hard Disk</h5></div></div></div><p>After concluding that the Hadoop and HBase were not likely to be the culprits, we moved on to troubleshooting the datanode's hardware.
Java, by design, will periodically scan its entire memory space to do garbage collection. If system memory is heavily overcommitted, the Linux
kernel may enter a vicious cycle, using up all of its resources swapping Java heap back and forth from disk to RAM as Java tries to run garbage
collection. Further, a failing hard disk will often retry reads and/or writes many times before giving up and returning an error. This can manifest
as high iowait, as running processes wait for reads and writes to complete. Finally, a disk nearing the upper edge of its performance envelope will
begin to cause iowait as it informs the kernel that it cannot accept any more data, and the kernel queues incoming data into the dirty write pool in memory.
However, using <code class="code">vmstat(1)</code> and <code class="code">free(1)</code>, we could see that no swap was being used, and the amount of disk IO was only a few kilobytes per second.
- </p></div><div class="section" title="1.3.1.3.4. Slowness Due To High Processor Usage"><div class="titlepage"><div><div><h5 class="title"><a name="d2403e134"></a>1.3.1.3.4. Slowness Due To High Processor Usage</h5></div></div></div><p>Next, we checked to see whether the system was performing slowly simply due to very high computational load. <code class="code">top(1)</code> showed that the system load
+ </p></div><div class="section" title="1.3.1.3.4. Slowness Due To High Processor Usage"><div class="titlepage"><div><div><h5 class="title"><a name="d2465e82"></a>1.3.1.3.4. Slowness Due To High Processor Usage</h5></div></div></div><p>Next, we checked to see whether the system was performing slowly simply due to very high computational load. <code class="code">top(1)</code> showed that the system load
was higher than normal, but <code class="code">vmstat(1)</code> and <code class="code">mpstat(1)</code> showed that the amount of processor being used for actual computation was low.
- </p></div><div class="section" title="1.3.1.3.5. Network Saturation (The Winner)"><div class="titlepage"><div><div><h5 class="title"><a name="d2403e148"></a>1.3.1.3.5. Network Saturation (The Winner)</h5></div></div></div><p>Since neither the disks nor the processors were being utilized heavily, we moved on to the performance of the network interfaces. The datanode had two
+ </p></div><div class="section" title="1.3.1.3.5. Network Saturation (The Winner)"><div class="titlepage"><div><div><h5 class="title"><a name="d2465e96"></a>1.3.1.3.5. Network Saturation (The Winner)</h5></div></div></div><p>Since neither the disks nor the processors were being utilized heavily, we moved on to the performance of the network interfaces. The datanode had two
gigabit ethernet adapters, bonded to form an active-standby interface. <code class="code">ifconfig(8)</code> showed some unusual anomalies, namely interface errors, overruns, framing errors.
While not unheard of, these kinds of errors are exceedingly rare on modern hardware which is operating as it should:
</p><pre class="programlisting">
@@ -72,7 +72,7 @@ Current message level: 0x00000003 (3)
Link detected: yes
</pre><p>
</p><p>In normal operation, the ICMP ping round trip time should be around 20ms, and the interface speed and duplex should read, "1000MB/s", and, "Full", respectively.
- </p></div></div><div class="section" title="1.3.1.4. Resolution"><div class="titlepage"><div><div><h4 class="title"><a name="d2403e169"></a>1.3.1.4. Resolution</h4></div></div></div><p>After determining that the active ethernet adapter was at the incorrect speed, we used the <code class="code">ifenslave(8)</code> command to make the standby interface
+ </p></div></div><div class="section" title="1.3.1.4. Resolution"><div class="titlepage"><div><div><h4 class="title"><a name="d2465e117"></a>1.3.1.4. Resolution</h4></div></div></div><p>After determining that the active ethernet adapter was at the incorrect speed, we used the <code class="code">ifenslave(8)</code> command to make the standby interface
the active interface, which yielded an immediate improvement in MapReduce performance, and a 10 times improvement in network throughput:
</p><p>On the next trip to the datacenter, we determined that the line speed issue was ultimately caused by a bad network cable, which was replaced.
</p></div></div><div class="section" title="1.3.2. Case Study #2 (Performance Research 2012)"><div class="titlepage"><div><div><h3 class="title"><a name="casestudies.perf.1"></a>1.3.2. Case Study #2 (Performance Research 2012)</h3></div></div></div><p>Investigation results of a self-described "we're not sure what's wrong, but it seems slow" problem.
Modified: hbase/hbase.apache.org/trunk/case_studies/casestudies.schema.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/case_studies/casestudies.schema.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/case_studies/casestudies.schema.html (original)
+++ hbase/hbase.apache.org/trunk/case_studies/casestudies.schema.html Sat Apr 6 06:08:56 2013
@@ -1,114 +1,7 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>1.2. Schema Design</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="up" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="prev" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="next" href="casestudies.perftroub.html" title="1.3. Performance/Troubleshooting"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">1.2. Schema Design</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="case_studies.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a acc
esskey="n" href="casestudies.perftroub.html">Next</a></td></tr></table><hr></div><div class="section" title="1.2. Schema Design"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="casestudies.schema"></a>1.2. Schema Design</h2></div></div></div><div class="section" title="1.2.1. List Data"><div class="titlepage"><div><div><h3 class="title"><a name="casestudies.schema.listdata"></a>1.2.1. List Data</h3></div></div></div><p>The following is an exchange from the user dist-list regarding a fairly common question:
- how to handle per-user list data in Apache HBase.
- </p><p>*** QUESTION ***</p><p>
- We're looking at how to store a large amount of (per-user) list data in
-HBase, and we were trying to figure out what kind of access pattern made
-the most sense. One option is store the majority of the data in a key, so
-we could have something like:
- </p><pre class="programlisting">
-<FixedWidthUserName><FixedWidthValueId1>:"" (no value)
-<FixedWidthUserName><FixedWidthValueId2>:"" (no value)
-<FixedWidthUserName><FixedWidthValueId3>:"" (no value)
- </pre>
-
-The other option we had was to do this entirely using:
- <pre class="programlisting">
-<FixedWidthUserName><FixedWidthPageNum0>:<FixedWidthLength><FixedIdNextPageNum><ValueId1><ValueId2><ValueId3>...
-<FixedWidthUserName><FixedWidthPageNum1>:<FixedWidthLength><FixedIdNextPageNum><ValueId1><ValueId2><ValueId3>...
- </pre><p>
-where each row would contain multiple values.
-So in one case reading the first thirty values would be:
- </p><pre class="programlisting">
-scan { STARTROW => 'FixedWidthUsername' LIMIT => 30}
- </pre>
-And in the second case it would be
- <pre class="programlisting">
-get 'FixedWidthUserName\x00\x00\x00\x00'
- </pre><p>
-The general usage pattern would be to read only the first 30 values of
-these lists, with infrequent access reading deeper into the lists. Some
-users would have <= 30 total values in these lists, and some users would
-have millions (i.e. power-law distribution)
- </p><p>
- The single-value format seems like it would take up more space on HBase,
-but would offer some improved retrieval / pagination flexibility. Would
-there be any significant performance advantages to be able to paginate via
-gets vs paginating with scans?
- </p><p>
- My initial understanding was that doing a scan should be faster if our
-paging size is unknown (and caching is set appropriately), but that gets
-should be faster if we'll always need the same page size. I've ended up
-hearing different people tell me opposite things about performance. I
-assume the page sizes would be relatively consistent, so for most use cases
-we could guarantee that we only wanted one page of data in the
-fixed-page-length case. I would also assume that we would have infrequent
-updates, but may have inserts into the middle of these lists (meaning we'd
-need to update all subsequent rows).
- </p><p>
-Thanks for help / suggestions / follow-up questions.
- </p><p>*** ANSWER ***</p><p>
-If I understand you correctly, you're ultimately trying to store
-triples in the form "user, valueid, value", right? E.g., something
-like:
- </p><pre class="programlisting">
-"user123, firstname, Paul",
-"user234, lastname, Smith"
- </pre><p>
-(But the usernames are fixed width, and the valueids are fixed width).
- </p><p>
-And, your access pattern is along the lines of: "for user X, list the
-next 30 values, starting with valueid Y". Is that right? And these
-values should be returned sorted by valueid?
- </p><p>
-The tl;dr version is that you should probably go with one row per
-user+value, and not build a complicated intra-row pagination scheme on
-your own unless you're really sure it is needed.
- </p><p>
-Your two options mirror a common question people have when designing
-HBase schemas: should I go "tall" or "wide"? Your first schema is
-"tall": each row represents one value for one user, and so there are
-many rows in the table for each user; the row key is user + valueid,
-and there would be (presumably) a single column qualifier that means
-"the value". This is great if you want to scan over rows in sorted
-order by row key (thus my question above, about whether these ids are
-sorted correctly). You can start a scan at any user+valueid, read the
-next 30, and be done. What you're giving up is the ability to have
-transactional guarantees around all the rows for one user, but it
-doesn't sound like you need that. Doing it this way is generally
-recommended (see
-here <a class="link" href="http://hbase.apache.org/book.html#schema.smackdown" target="_top">http://hbase.apache.org/book.html#schema.smackdown</a>).
- </p><p>
-Your second option is "wide": you store a bunch of values in one row,
-using different qualifiers (where the qualifier is the valueid). The
-simple way to do that would be to just store ALL values for one user
-in a single row. I'm guessing you jumped to the "paginated" version
-because you're assuming that storing millions of columns in a single
-row would be bad for performance, which may or may not be true; as
-long as you're not trying to do too much in a single request, or do
-things like scanning over and returning all of the cells in the row,
-it shouldn't be fundamentally worse. The client has methods that allow
-you to get specific slices of columns.
- </p><p>
-Note that neither case fundamentally uses more disk space than the
-other; you're just "shifting" part of the identifying information for
-a value either to the left (into the row key, in option one) or to the
-right (into the column qualifiers in option 2). Under the covers,
-every key/value still stores the whole row key, and column family
-name. (If this is a bit confusing, take an hour and watch Lars
-George's excellent video about understanding HBase schema design:
-<a class="link" href="http://www.youtube.com/watch?v=_HLoH_PgrLk)" target="_top">http://www.youtube.com/watch?v=_HLoH_PgrLk)</a>.
- </p><p>
-A manually paginated version has lots more complexities, as you note,
-like having to keep track of how many things are in each page,
-re-shuffling if new values are inserted, etc. That seems significantly
-more complex. It might have some slight speed advantages (or
-disadvantages!) at extremely high throughput, and the only way to
-really know that would be to try it out. If you don't have time to
-build it both ways and compare, my advice would be to start with the
-simplest option (one row per user+value). Start simple and iterate! :)
- </p></div></div><div id="disqus_thread"></div><script type="text/javascript">
+ <title>1.2. Schema Design</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="up" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="prev" href="case_studies.html" title="Chapter 1. Apache HBase (TM) Case Studies"><link rel="next" href="casestudies.perftroub.html" title="1.3. Performance/Troubleshooting"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">1.2. Schema Design</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="case_studies.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a acc
esskey="n" href="casestudies.perftroub.html">Next</a></td></tr></table><hr></div><div class="section" title="1.2. Schema Design"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="casestudies.schema"></a>1.2. Schema Design</h2></div></div></div><p>See the schema design case studies here: <a class="xref" href="">???</a>
+ </p></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
var disqus_identifier = 'casestudies.schema';
Modified: hbase/hbase.apache.org/trunk/community.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/community.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/community.html (original)
+++ hbase/hbase.apache.org/trunk/community.html Sat Apr 6 06:08:56 2013
@@ -1,9 +1,9 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 1. Community</title><link rel="stylesheet" type="text/css" href="css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="chapter" title="Chapter 1. Community"><div class="titlepage"><div><div><h2 class="title"><a name="community"></a>Chapter 1. Community</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#decisions">1.1. Decisions</a></span></dt><dd><dl><dt><span class="section"><a href="#feature_branches">1.1.1. Feature Branches</a></span></dt><dt><span class="section"><a href="#patchplusonepolicy">1.1.2. Patch +1 Policy</a></span></dt></dl></dd><dt><span class="section"><a href="#community.roles">1.2. Community Roles</a></span></dt><dd><dl><dt><span class="section"><a href="#OWNER">1.2.1. Component Owner</a></span></dt></dl></dd></dl
></div><div class="section" title="1.1. Decisions"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="decisions"></a>1.1. Decisions</h2></div></div></div><div class="section" title="1.1.1. Feature Branches"><div class="titlepage"><div><div><h3 class="title"><a name="feature_branches"></a>1.1.1. Feature Branches</h3></div></div></div><p>Feature Branches are easy to make. You do not have to be a committer to make one. Just request the name of your branch be added to JIRA up on the
+ <title>Chapter 1. Community</title><link rel="stylesheet" type="text/css" href="css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="chapter" title="Chapter 1. Community"><div class="titlepage"><div><div><h2 class="title"><a name="community"></a>Chapter 1. Community</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#decisions">1.1. Decisions</a></span></dt><dd><dl><dt><span class="section"><a href="#feature_branches">1.1.1. Feature Branches</a></span></dt><dt><span class="section"><a href="#patchplusonepolicy">1.1.2. Patch +1 Policy</a></span></dt><dt><span class="section"><a href="#hbase.fix.version.in.JIRA">1.1.3. How to set fix version in JIRA on issue resolve</a></span></dt></dl></dd><dt><span class="section"><a href="#community.roles">1.2. C
ommunity Roles</a></span></dt><dd><dl><dt><span class="section"><a href="#OWNER">1.2.1. Component Owner</a></span></dt></dl></dd></dl></div><div class="section" title="1.1. Decisions"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="decisions"></a>1.1. Decisions</h2></div></div></div><div class="section" title="1.1.1. Feature Branches"><div class="titlepage"><div><div><h3 class="title"><a name="feature_branches"></a>1.1.1. Feature Branches</h3></div></div></div><p>Feature Branches are easy to make. You do not have to be a committer to make one. Just request the name of your branch be added to JIRA up on the
developer's mailing list and a committer will add it for you. Thereafter you can file issues against your feature branch in Apache HBase (TM) JIRA. Your code you
keep elsewhere -- it should be public so it can be observed -- and you can update dev mailing list on progress. When the feature is ready for commit,
- 3 +1s from committers will get your feature merged<sup>[<a name="d2377e13" href="#ftn.d2377e13" class="footnote">1</a>]</sup>
+ 3 +1s from committers will get your feature merged<sup>[<a name="d2431e13" href="#ftn.d2431e13" class="footnote">1</a>]</sup>
</p></div><div class="section" title="1.1.2. Patch +1 Policy"><div class="titlepage"><div><div><h3 class="title"><a name="patchplusonepolicy"></a>1.1.2. Patch +1 Policy</h3></div></div></div><p>
The below policy is something we put in place 09/2012. It is a
suggested policy rather than a hard requirement. We want to try it
@@ -11,7 +11,7 @@ first to see if it works before we cast
</p><p>
Apache HBase is made of
<a class="link" href="https://issues.apache.org/jira/browse/HBASE#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel" target="_top">components</a>.
-Components have one or more <a class="xref" href="#OWNER" title="1.2.1. Component Owner">Section 1.2.1, “Component Owner”</a>s. See the 'Description' field on the
+Components have one or more <a class="xref" href="#OWNER" title="1.2.1. Component Owner">Section 1.2.1, “Component Owner”</a>s. See the 'Description' field on the
<a class="link" href="https://issues.apache.org/jira/browse/HBASE#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel" target="_top">components</a>
JIRA page for who the current owners are by component.
</p><p>
@@ -27,7 +27,20 @@ first pass).
</p><p>
Any -1 on a patch by anyone vetos a patch; it cannot be committed
until the justification for the -1 is addressed.
-</p></div></div><div class="section" title="1.2. Community Roles"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="community.roles"></a>1.2. Community Roles</h2></div></div></div><div class="section" title="1.2.1. Component Owner"><div class="titlepage"><div><div><h3 class="title"><a name="OWNER"></a>1.2.1. Component Owner</h3></div></div></div><p>
+</p></div><div class="section" title="1.1.3. How to set fix version in JIRA on issue resolve"><div class="titlepage"><div><div><h3 class="title"><a name="hbase.fix.version.in.JIRA"></a>1.1.3. How to set fix version in JIRA on issue resolve</h3></div></div></div><p>Here is how <a class="link" href="http://search-hadoop.com/m/azemIi5RCJ1" target="_top">we agreed</a> to set versions in JIRA when we
+ resolve an issue. If trunk is going to be 0.98.0 then:
+ </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
+ Commit only to trunk: Mark with 0.98
+ </p></li><li class="listitem"><p>
+ Commit to 0.95 and trunk : Mark with 0.98, and 0.95.x
+ </p></li><li class="listitem"><p>
+ Commit to 0.94.x and 0.95, and trunk: Mark with 0.98, 0.95.x, and 0.94.x
+ </p></li><li class="listitem"><p>
+ Commit to 89-fb: Mark with 89-fb.
+ </p></li><li class="listitem"><p>
+ Commit site fixes: no version
+ </p></li></ul></div><p>
+ </p></div></div><div class="section" title="1.2. Community Roles"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="community.roles"></a>1.2. Community Roles</h2></div></div></div><div class="section" title="1.2.1. Component Owner"><div class="titlepage"><div><div><h3 class="title"><a name="OWNER"></a>1.2.1. Component Owner</h3></div></div></div><p>
Component owners are listed in the description field on this Apache HBase JIRA <a class="link" href="https://issues.apache.org/jira/browse/HBASE#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel" target="_top">components</a>
page. The owners are listed in the 'Description' field rather than in the 'Component
Lead' field because the latter only allows us list one individual
@@ -47,7 +60,7 @@ goals or the design toward which they ar
</p><p>
If you would like to be volunteer as a component owner, just write the
dev list and we'll sign you up. Owners do not need to be committers.
-</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2377e13" href="#d2377e13" class="para">1</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/asM982C5FkS1" target="_top">HBase, mail # dev - Thoughts about large feature dev branches</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
+</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2431e13" href="#d2431e13" class="para">1</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/asM982C5FkS1" target="_top">HBase, mail # dev - Thoughts about large feature dev branches</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
var disqus_identifier = 'community';
Modified: hbase/hbase.apache.org/trunk/community/community.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/community/community.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/community/community.html (original)
+++ hbase/hbase.apache.org/trunk/community/community.html Sat Apr 6 06:08:56 2013
@@ -1,9 +1,9 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 1. Community</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="community.html" title="Chapter 1. Community"><link rel="next" href="community.roles.html" title="1.2. Community Roles"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 1. Community</th></tr><tr><td width="20%" align="left"> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="community.roles.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 1. Community"><div class="titlepage"><div><div><h2 class="title"><a name="community"></a>Chapter 1. Community</h2></div></div></div><div class="toc">
<p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="community.html#decisions">1.1. Decisions</a></span></dt><dd><dl><dt><span class="section"><a href="community.html#feature_branches">1.1.1. Feature Branches</a></span></dt><dt><span class="section"><a href="community.html#patchplusonepolicy">1.1.2. Patch +1 Policy</a></span></dt></dl></dd><dt><span class="section"><a href="community.roles.html">1.2. Community Roles</a></span></dt><dd><dl><dt><span class="section"><a href="community.roles.html#OWNER">1.2.1. Component Owner</a></span></dt></dl></dd></dl></div><div class="section" title="1.1. Decisions"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="decisions"></a>1.1. Decisions</h2></div></div></div><div class="section" title="1.1.1. Feature Branches"><div class="titlepage"><div><div><h3 class="title"><a name="feature_branches"></a>1.1.1. Feature Branches</h3></div></div></div><p>Feature Branches are easy t
o make. You do not have to be a committer to make one. Just request the name of your branch be added to JIRA up on the
+ <title>Chapter 1. Community</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="community.html" title="Chapter 1. Community"><link rel="next" href="community.roles.html" title="1.2. Community Roles"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 1. Community</th></tr><tr><td width="20%" align="left"> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="community.roles.html">Next</a></td></tr></table><hr></div><div class="chapter" title="Chapter 1. Community"><div class="titlepage"><div><div><h2 class="title"><a name="community"></a>Chapter 1. Community</h2></div></div></div><div class="toc">
<p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="community.html#decisions">1.1. Decisions</a></span></dt><dd><dl><dt><span class="section"><a href="community.html#feature_branches">1.1.1. Feature Branches</a></span></dt><dt><span class="section"><a href="community.html#patchplusonepolicy">1.1.2. Patch +1 Policy</a></span></dt><dt><span class="section"><a href="community.html#hbase.fix.version.in.JIRA">1.1.3. How to set fix version in JIRA on issue resolve</a></span></dt></dl></dd><dt><span class="section"><a href="community.roles.html">1.2. Community Roles</a></span></dt><dd><dl><dt><span class="section"><a href="community.roles.html#OWNER">1.2.1. Component Owner</a></span></dt></dl></dd></dl></div><div class="section" title="1.1. Decisions"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="decisions"></a>1.1. Decisions</h2></div></div></div><div class="section" title="1.1.1. Feature Branches"><div class="titl
epage"><div><div><h3 class="title"><a name="feature_branches"></a>1.1.1. Feature Branches</h3></div></div></div><p>Feature Branches are easy to make. You do not have to be a committer to make one. Just request the name of your branch be added to JIRA up on the
developer's mailing list and a committer will add it for you. Thereafter you can file issues against your feature branch in Apache HBase (TM) JIRA. Your code you
keep elsewhere -- it should be public so it can be observed -- and you can update dev mailing list on progress. When the feature is ready for commit,
- 3 +1s from committers will get your feature merged<sup>[<a name="d2377e13" href="#ftn.d2377e13" class="footnote">1</a>]</sup>
+ 3 +1s from committers will get your feature merged<sup>[<a name="d2431e13" href="#ftn.d2431e13" class="footnote">1</a>]</sup>
</p></div><div class="section" title="1.1.2. Patch +1 Policy"><div class="titlepage"><div><div><h3 class="title"><a name="patchplusonepolicy"></a>1.1.2. Patch +1 Policy</h3></div></div></div><p>
The below policy is something we put in place 09/2012. It is a
suggested policy rather than a hard requirement. We want to try it
@@ -11,7 +11,7 @@ first to see if it works before we cast
</p><p>
Apache HBase is made of
<a class="link" href="https://issues.apache.org/jira/browse/HBASE#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel" target="_top">components</a>.
-Components have one or more <a class="xref" href="community.roles.html#OWNER" title="1.2.1. Component Owner">Section 1.2.1, “Component Owner”</a>s. See the 'Description' field on the
+Components have one or more <a class="xref" href="community.roles.html#OWNER" title="1.2.1. Component Owner">Section 1.2.1, “Component Owner”</a>s. See the 'Description' field on the
<a class="link" href="https://issues.apache.org/jira/browse/HBASE#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel" target="_top">components</a>
JIRA page for who the current owners are by component.
</p><p>
@@ -27,7 +27,20 @@ first pass).
</p><p>
Any -1 on a patch by anyone vetos a patch; it cannot be committed
until the justification for the -1 is addressed.
-</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2377e13" href="#d2377e13" class="para">1</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/asM982C5FkS1" target="_top">HBase, mail # dev - Thoughts about large feature dev branches</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
+</p></div><div class="section" title="1.1.3. How to set fix version in JIRA on issue resolve"><div class="titlepage"><div><div><h3 class="title"><a name="hbase.fix.version.in.JIRA"></a>1.1.3. How to set fix version in JIRA on issue resolve</h3></div></div></div><p>Here is how <a class="link" href="http://search-hadoop.com/m/azemIi5RCJ1" target="_top">we agreed</a> to set versions in JIRA when we
+ resolve an issue. If trunk is going to be 0.98.0 then:
+ </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
+ Commit only to trunk: Mark with 0.98
+ </p></li><li class="listitem"><p>
+ Commit to 0.95 and trunk : Mark with 0.98, and 0.95.x
+ </p></li><li class="listitem"><p>
+ Commit to 0.94.x and 0.95, and trunk: Mark with 0.98, 0.95.x, and 0.94.x
+ </p></li><li class="listitem"><p>
+ Commit to 89-fb: Mark with 89-fb.
+ </p></li><li class="listitem"><p>
+ Commit site fixes: no version
+ </p></li></ul></div><p>
+ </p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2431e13" href="#d2431e13" class="para">1</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/asM982C5FkS1" target="_top">HBase, mail # dev - Thoughts about large feature dev branches</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
var disqus_url = 'http://hbase.apache.org/book';
var disqus_identifier = 'community';
Modified: hbase/hbase.apache.org/trunk/configuration.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/configuration.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/configuration.html (original)
+++ hbase/hbase.apache.org/trunk/configuration.html Sat Apr 6 06:08:56 2013
@@ -1,6 +1,6 @@
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
- <title>Chapter 1. Apache HBase (TM) Configuration</title><link rel="stylesheet" type="text/css" href="css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="chapter" title="Chapter 1. Apache HBase (TM) Configuration"><div class="titlepage"><div><div><h2 class="title"><a name="configuration"></a>Chapter 1. Apache HBase (TM) Configuration</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#basic.prerequisites">1.1. Basic Prerequisites</a></span></dt><dd><dl><dt><span class="section"><a href="#java">1.1.1. Java</a></span></dt><dt><span class="section"><a href="#os">1.1.2. Operating System</a></span></dt><dt><span class="section"><a href="#hadoop">1.1.3. Hadoop</a></span></dt></dl></dd><dt><span class="section"><a href="#standalone_dist">1.2. HBase run
modes: Standalone and Distributed</a></span></dt><dd><dl><dt><span class="section"><a href="#standalone">1.2.1. Standalone HBase</a></span></dt><dt><span class="section"><a href="#distributed">1.2.2. Distributed</a></span></dt><dt><span class="section"><a href="#confirm">1.2.3. Running and Confirming Your Installation</a></span></dt></dl></dd><dt><span class="section"><a href="#config.files">1.3. Configuration Files</a></span></dt><dd><dl><dt><span class="section"><a href="#hbase.site">1.3.1. <code class="filename">hbase-site.xml</code> and <code class="filename">hbase-default.xml</code></a></span></dt><dt><span class="section"><a href="#hbase.env.sh">1.3.2. <code class="filename">hbase-env.sh</code></a></span></dt><dt><span class="section"><a href="#log4j">1.3.3. <code class="filename">log4j.properties</code></a></span></dt><dt><span class="section"><a href="#client_dependencies">1.3.4. Client configuration and dependencies connecting to an HBase cluster</a></span></dt></dl
></dd><dt><span class="section"><a href="#example_config">1.4. Example Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="#d1897e2050">1.4.1. Basic Distributed HBase Install</a></span></dt></dl></dd><dt><span class="section"><a href="#important_configurations">1.5. The Important Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="#required_configuration">1.5.1. Required Configurations</a></span></dt><dt><span class="section"><a href="#recommended_configurations">1.5.2. Recommended Configurations</a></span></dt><dt><span class="section"><a href="#other_configuration">1.5.3. Other Configurations</a></span></dt></dl></dd></dl></div><p>This chapter is the Not-So-Quick start guide to Apache HBase (TM) configuration. It goes
+ <title>Chapter 1. Apache HBase (TM) Configuration</title><link rel="stylesheet" type="text/css" href="css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="chapter" title="Chapter 1. Apache HBase (TM) Configuration"><div class="titlepage"><div><div><h2 class="title"><a name="configuration"></a>Chapter 1. Apache HBase (TM) Configuration</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#basic.prerequisites">1.1. Basic Prerequisites</a></span></dt><dd><dl><dt><span class="section"><a href="#java">1.1.1. Java</a></span></dt><dt><span class="section"><a href="#os">1.1.2. Operating System</a></span></dt><dt><span class="section"><a href="#hadoop">1.1.3. Hadoop</a></span></dt></dl></dd><dt><span class="section"><a href="#standalone_dist">1.2. HBase run
modes: Standalone and Distributed</a></span></dt><dd><dl><dt><span class="section"><a href="#standalone">1.2.1. Standalone HBase</a></span></dt><dt><span class="section"><a href="#distributed">1.2.2. Distributed</a></span></dt><dt><span class="section"><a href="#confirm">1.2.3. Running and Confirming Your Installation</a></span></dt></dl></dd><dt><span class="section"><a href="#config.files">1.3. Configuration Files</a></span></dt><dd><dl><dt><span class="section"><a href="#hbase.site">1.3.1. <code class="filename">hbase-site.xml</code> and <code class="filename">hbase-default.xml</code></a></span></dt><dt><span class="section"><a href="#hbase.env.sh">1.3.2. <code class="filename">hbase-env.sh</code></a></span></dt><dt><span class="section"><a href="#log4j">1.3.3. <code class="filename">log4j.properties</code></a></span></dt><dt><span class="section"><a href="#client_dependencies">1.3.4. Client configuration and dependencies connecting to an HBase cluster</a></span></dt></dl
></dd><dt><span class="section"><a href="#example_config">1.4. Example Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="#d1951e2050">1.4.1. Basic Distributed HBase Install</a></span></dt></dl></dd><dt><span class="section"><a href="#important_configurations">1.5. The Important Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="#required_configuration">1.5.1. Required Configurations</a></span></dt><dt><span class="section"><a href="#recommended_configurations">1.5.2. Recommended Configurations</a></span></dt><dt><span class="section"><a href="#other_configuration">1.5.3. Other Configurations</a></span></dt></dl></dd></dl></div><p>This chapter is the Not-So-Quick start guide to Apache HBase (TM) configuration. It goes
over system requirements, Hadoop setup, the different Apache HBase run modes, and the
various configurations in HBase. Please read this chapter carefully. At a mimimum
ensure that all <a class="xref" href="#basic.prerequisites" title="1.1. Basic Prerequisites">Section 1.1, “Basic Prerequisites”</a> have
@@ -13,7 +13,7 @@
off the ground -- and then add configuration to an XML file to
do things like override HBase defaults, tell HBase what Filesystem to
use, and the location of the ZooKeeper ensemble
- <sup>[<a name="d1897e14" href="#ftn.d1897e14" class="footnote">1</a>]</sup>
+ <sup>[<a name="d1951e14" href="#ftn.d1951e14" class="footnote">1</a>]</sup>
.
</p><p>When running in distributed mode, after you make
an edit to an HBase configuration, make sure you copy the
@@ -30,7 +30,7 @@
on the hadoop wiki.</p></div><div class="section" title="1.1.2.2. DNS"><div class="titlepage"><div><div><h4 class="title"><a name="dns"></a>1.1.2.2. DNS</h4></div></div></div><p>HBase uses the local hostname to self-report its IP address.
Both forward and reverse DNS resolving must work in versions of
HBase previous to 0.92.0
- <sup>[<a name="d1897e63" href="#ftn.d1897e63" class="footnote">2</a>]</sup>.</p><p>If your machine has multiple interfaces, HBase will use the
+ <sup>[<a name="d1951e63" href="#ftn.d1951e63" class="footnote">2</a>]</sup>.</p><p>If your machine has multiple interfaces, HBase will use the
interface that the primary hostname resolves to.</p><p>If this is insufficient, you can set
<code class="varname">hbase.regionserver.dns.interface</code> to indicate the
primary interface. This only works if your cluster configuration is
@@ -42,9 +42,9 @@
<a class="link" href="http://en.wikipedia.org/wiki/Network_Time_Protocol" target="_top">NTP</a>
on your cluster, or an equivalent.</p><p>If you are having problems querying data, or "weird" cluster
operations, check system time!</p></div><div class="section" title="1.1.2.5. ulimit and nproc"><div class="titlepage"><div><div><h4 class="title"><a name="ulimit"></a>1.1.2.5.
- <code class="varname">ulimit</code><a class="indexterm" name="d1897e103"></a>
+ <code class="varname">ulimit</code><a class="indexterm" name="d1951e103"></a>
and
- <code class="varname">nproc</code><a class="indexterm" name="d1897e109"></a>
+ <code class="varname">nproc</code><a class="indexterm" name="d1951e109"></a>
</h4></div></div></div><p>Apache HBase is a database. It uses a lot of files all at the same time.
The default ulimit -n -- i.e. user file limit -- of 1024 on most *nix systems
is insufficient (On mac os x its 256). Any significant amount of loading will
@@ -62,15 +62,15 @@
</p><p>You should also up the hbase users'
<code class="varname">nproc</code> setting; under load, a low-nproc
setting could manifest as <code class="classname">OutOfMemoryError</code>
- <sup>[<a name="d1897e128" href="#ftn.d1897e128" class="footnote">3</a>]</sup>
- <sup>[<a name="d1897e135" href="#ftn.d1897e135" class="footnote">4</a>]</sup>.
+ <sup>[<a name="d1951e128" href="#ftn.d1951e128" class="footnote">3</a>]</sup>
+ <sup>[<a name="d1951e135" href="#ftn.d1951e135" class="footnote">4</a>]</sup>.
</p><p>To be clear, upping the file descriptors and nproc for the user who is
running the HBase process is an operating system configuration, not an
HBase configuration. Also, a common mistake is that administrators
will up the file descriptors for a particular user but for whatever
reason, HBase will be running as some one else. HBase prints in its
logs as the first line the ulimit its seeing. Ensure its correct.
- <sup>[<a name="d1897e147" href="#ftn.d1897e147" class="footnote">5</a>]</sup></p><div class="section" title="1.1.2.5.1. ulimit on Ubuntu"><div class="titlepage"><div><div><h5 class="title"><a name="ulimit_ubuntu"></a>1.1.2.5.1. <code class="varname">ulimit</code> on Ubuntu</h5></div></div></div><p>If you are on Ubuntu you will need to make the following
+ <sup>[<a name="d1951e147" href="#ftn.d1951e147" class="footnote">5</a>]</sup></p><div class="section" title="1.1.2.5.1. ulimit on Ubuntu"><div class="titlepage"><div><div><h5 class="title"><a name="ulimit_ubuntu"></a>1.1.2.5.1. <code class="varname">ulimit</code> on Ubuntu</h5></div></div></div><p>If you are on Ubuntu you will need to make the following
changes:</p><p>In the file <code class="filename">/etc/security/limits.conf</code> add
a line like: </p><pre class="programlisting">hadoop - nofile 32768</pre><p>
Replace <code class="varname">hadoop</code> with whatever user is running
@@ -87,18 +87,18 @@
the <a class="link" href="http://hbase.apache.org/cygwin.html" target="_top">Windows
Installation</a> guide. Also
<a class="link" href="http://search-hadoop.com/?q=hbase+windows&fc_project=HBase&fc_type=mail+_hash_+dev" target="_top">search our user mailing list</a> to pick
- up latest fixes figured by Windows users.</p></div></div><div class="section" title="1.1.3. Hadoop"><div class="titlepage"><div><div><h3 class="title"><a name="hadoop"></a>1.1.3. <a class="link" href="http://hadoop.apache.org" target="_top">Hadoop</a><a class="indexterm" name="d1897e207"></a></h3></div></div></div><p>Selecting a Hadoop version is critical for your HBase deployment. Below table shows some information about what versions of Hadoop are supported by various HBase versions. Based on the version of HBase, you should select the most appropriate version of Hadoop. We are not in the Hadoop distro selection business. You can use Hadoop distributions from Apache, or learn about vendor distributions of Hadoop at <a class="link" href="http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support" target="_top">http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support</a></p><p>
- </p><div class="table"><a name="d1897e215"></a><p class="title"><b>Table 1.1. Hadoop version support matrix</b></p><div class="table-contents"><table summary="Hadoop version support matrix" border="1"><colgroup><col align="left" class="c1"><col align="center" class="c2"><col align="center" class="c3"><col align="center" class="c4"></colgroup><thead><tr><th align="left"> </th><th align="center">HBase-0.92.x</th><th align="center">HBase-0.94.x</th><th align="center">HBase-0.96</th></tr></thead><tbody><tr><td align="left">Hadoop-0.20.205</td><td align="center">S</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">Hadoop-0.22.x </td><td align="center">S</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">Hadoop-1.0.x </td><td align="center">S</td><td align="center">S</td><td align="center">S</td></tr><tr><td align="left">Hadoop-1.1.x </td><td align="center">NT</td><td align="center">S</td>
<td align="center">S</td></tr><tr><td align="left">Hadoop-0.23.x </td><td align="center">X</td><td align="center">S</td><td align="center">NT</td></tr><tr><td align="left">Hadoop-2.x </td><td align="center">X</td><td align="center">S</td><td align="center">S</td></tr></tbody></table></div></div><p><br class="table-break">
+ up latest fixes figured by Windows users.</p></div></div><div class="section" title="1.1.3. Hadoop"><div class="titlepage"><div><div><h3 class="title"><a name="hadoop"></a>1.1.3. <a class="link" href="http://hadoop.apache.org" target="_top">Hadoop</a><a class="indexterm" name="d1951e207"></a></h3></div></div></div><p>Selecting a Hadoop version is critical for your HBase deployment. Below table shows some information about what versions of Hadoop are supported by various HBase versions. Based on the version of HBase, you should select the most appropriate version of Hadoop. We are not in the Hadoop distro selection business. You can use Hadoop distributions from Apache, or learn about vendor distributions of Hadoop at <a class="link" href="http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support" target="_top">http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support</a></p><p>
+ </p><div class="table"><a name="d1951e215"></a><p class="title"><b>Table 1.1. Hadoop version support matrix</b></p><div class="table-contents"><table summary="Hadoop version support matrix" border="1"><colgroup><col align="left" class="c1"><col align="center" class="c2"><col align="center" class="c3"><col align="center" class="c4"></colgroup><thead><tr><th align="left"> </th><th align="center">HBase-0.92.x</th><th align="center">HBase-0.94.x</th><th align="center">HBase-0.96</th></tr></thead><tbody><tr><td align="left">Hadoop-0.20.205</td><td align="center">S</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">Hadoop-0.22.x </td><td align="center">S</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">Hadoop-1.0.x </td><td align="center">S</td><td align="center">S</td><td align="center">S</td></tr><tr><td align="left">Hadoop-1.1.x </td><td align="center">NT</td><td align="center">S</td>
<td align="center">S</td></tr><tr><td align="left">Hadoop-0.23.x </td><td align="center">X</td><td align="center">S</td><td align="center">NT</td></tr><tr><td align="left">Hadoop-2.x </td><td align="center">X</td><td align="center">S</td><td align="center">S</td></tr></tbody></table></div></div><p><br class="table-break">
Where
</p><table border="0" summary="Simple list" class="simplelist"><tr><td>S = supported and tested,</td></tr><tr><td>X = not supported,</td></tr><tr><td>NT = it should run, but not tested enough.</td></tr></table><p>
</p><p>
Because HBase depends on Hadoop, it bundles an instance of the Hadoop jar under its <code class="filename">lib</code> directory. The bundled jar is ONLY for use in standalone mode. In distributed mode, it is <span class="emphasis"><em>critical</em></span> that the version of Hadoop that is out on your cluster match what is under HBase. Replace the hadoop jar found in the HBase lib directory with the hadoop jar you are running on your cluster to avoid version mismatch issues. Make sure you replace the jar in HBase everywhere on your cluster. Hadoop version mismatch issues have various manifestations but often all looks like its hung up.
</p><div class="section" title="1.1.3.1. Apache HBase 0.92 and 0.94"><div class="titlepage"><div><div><h4 class="title"><a name="hadoop.hbase-0.94"></a>1.1.3.1. Apache HBase 0.92 and 0.94</h4></div></div></div><p>HBase 0.92 and 0.94 versions can work with Hadoop versions, 0.20.205, 0.22.x, 1.0.x, and 1.1.x. HBase-0.94 can additionally work with Hadoop-0.23.x and 2.x, but you may have to recompile the code using the specific maven profile (see top level pom.xml)</p></div><div class="section" title="1.1.3.2. Apache HBase 0.96"><div class="titlepage"><div><div><h4 class="title"><a name="hadoop.hbase-0.96"></a>1.1.3.2. Apache HBase 0.96</h4></div></div></div><p>Apache HBase 0.96.0 requires Apache Hadoop 1.x at a minimum, and it can run equally well on hadoop-2.0.
- As of Apache HBase 0.96.x, Apache Hadoop 1.0.x at least is required. We will no longer run properly on older Hadoops such as 0.20.205 or branch-0.20-append. Do not move to Apache HBase 0.96.x if you cannot upgrade your Hadoop<sup>[<a name="d1897e315" href="#ftn.d1897e315" class="footnote">6</a>]</sup>.</p></div><div class="section" title="1.1.3.3. Hadoop versions 0.20.x - 1.x"><div class="titlepage"><div><div><h4 class="title"><a name="hadoop.older.versions"></a>1.1.3.3. Hadoop versions 0.20.x - 1.x</h4></div></div></div><p>
+ As of Apache HBase 0.96.x, Apache Hadoop 1.0.x at least is required. We will no longer run properly on older Hadoops such as 0.20.205 or branch-0.20-append. Do not move to Apache HBase 0.96.x if you cannot upgrade your Hadoop<sup>[<a name="d1951e315" href="#ftn.d1951e315" class="footnote">6</a>]</sup>.</p></div><div class="section" title="1.1.3.3. Hadoop versions 0.20.x - 1.x"><div class="titlepage"><div><div><h4 class="title"><a name="hadoop.older.versions"></a>1.1.3.3. Hadoop versions 0.20.x - 1.x</h4></div></div></div><p>
HBase will lose data unless it is running on an HDFS that has a durable
<code class="code">sync</code> implementation. DO NOT use Hadoop 0.20.2, Hadoop 0.20.203.0, and Hadoop 0.20.204.0 which DO NOT have this attribute. Currently only Hadoop versions 0.20.205.x or any release in excess of this version -- this includes hadoop-1.0.0 -- have a working, durable sync
- <sup>[<a name="d1897e329" href="#ftn.d1897e329" class="footnote">7</a>]</sup>. Sync has to be explicitly enabled by setting
+ <sup>[<a name="d1951e329" href="#ftn.d1951e329" class="footnote">7</a>]</sup>. Sync has to be explicitly enabled by setting
<code class="varname">dfs.support.append</code> equal
to true on both the client side -- in <code class="filename">hbase-site.xml</code>
-- and on the serverside in <code class="filename">hdfs-site.xml</code> (The sync
@@ -116,7 +116,7 @@
security features as long as you do as
suggested above and replace the Hadoop jar that ships with HBase
with the secure version. If you want to read more about how to setup
- Secure HBase, see <a class="xref" href="#">???</a>.</p></div><div class="section" title="1.1.3.5. dfs.datanode.max.xcievers"><div class="titlepage"><div><div><h4 class="title"><a name="dfs.datanode.max.xcievers"></a>1.1.3.5. <code class="varname">dfs.datanode.max.xcievers</code><a class="indexterm" name="d1897e365"></a></h4></div></div></div><p>An Hadoop HDFS datanode has an upper bound on the number of
+ Secure HBase, see <a class="xref" href="#">???</a>.</p></div><div class="section" title="1.1.3.5. dfs.datanode.max.xcievers"><div class="titlepage"><div><div><h4 class="title"><a name="dfs.datanode.max.xcievers"></a>1.1.3.5. <code class="varname">dfs.datanode.max.xcievers</code><a class="indexterm" name="d1951e365"></a></h4></div></div></div><p>An Hadoop HDFS datanode has an upper bound on the number of
files that it will serve at any one time. The upper bound parameter is
called <code class="varname">xcievers</code> (yes, this is misspelled). Again,
before doing any loading, make sure you have configured Hadoop's
@@ -136,7 +136,7 @@
blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node:
java.io.IOException: No live nodes contain current block. Will get new
block locations from namenode and retry...</code>
- <sup>[<a name="d1897e388" href="#ftn.d1897e388" class="footnote">8</a>]</sup></p><p>See also <a class="xref" href="#">???</a>
+ <sup>[<a name="d1951e388" href="#ftn.d1951e388" class="footnote">8</a>]</sup></p><p>See also <a class="xref" href="#">???</a>
</p></div></div></div><div class="section" title="1.2. HBase run modes: Standalone and Distributed"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="standalone_dist"></a>1.2. HBase run modes: Standalone and Distributed</h2></div></div></div><p>HBase has two run modes: <a class="xref" href="#standalone" title="1.2.1. Standalone HBase">Section 1.2.1, “Standalone HBase”</a> and <a class="xref" href="#distributed" title="1.2.2. Distributed">Section 1.2.2, “Distributed”</a>. Out of the box, HBase runs in
standalone mode. To set up a distributed deploy, you will need to
configure HBase by editing files in the HBase <code class="filename">conf</code>
@@ -155,7 +155,7 @@
daemons run on a single node -- a.k.a
<span class="emphasis"><em>pseudo-distributed</em></span>-- and
<span class="emphasis"><em>fully-distributed</em></span> where the daemons are spread
- across all nodes in the cluster <sup>[<a name="d1897e446" href="#ftn.d1897e446" class="footnote">9</a>]</sup>.</p><p>Distributed modes require an instance of the <span class="emphasis"><em>Hadoop
+ across all nodes in the cluster <sup>[<a name="d1951e446" href="#ftn.d1951e446" class="footnote">9</a>]</sup>.</p><p>Distributed modes require an instance of the <span class="emphasis"><em>Hadoop
Distributed File System</em></span> (HDFS). See the Hadoop <a class="link" href="http://hadoop.apache.org/common/docs/r1.1.1/api/overview-summary.html#overview_description" target="_top">
requirements and instructions</a> for how to set up a HDFS. Before
proceeding, ensure you have an appropriate, working HDFS.</p><p>Below we describe the different distributed setups. Starting,
@@ -174,7 +174,7 @@
Note that the <code class="varname">hbase.rootdir</code> property points to the
local HDFS instance.
</p><p>Now skip to <a class="xref" href="#confirm" title="1.2.3. Running and Confirming Your Installation">Section 1.2.3, “Running and Confirming Your Installation”</a> for how to start and verify your
- pseudo-distributed install. <sup>[<a name="d1897e494" href="#ftn.d1897e494" class="footnote">10</a>]</sup></p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Let HBase create the <code class="varname">hbase.rootdir</code>
+ pseudo-distributed install. <sup>[<a name="d1951e494" href="#ftn.d1951e494" class="footnote">10</a>]</sup></p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Let HBase create the <code class="varname">hbase.rootdir</code>
directory. If you don't, you'll get warning saying HBase needs a
migration run because the directory is missing files expected by
HBase (it'll create them if you let it).</p></div><div class="section" title="1.2.2.1.1. Pseudo-distributed Configuration File"><div class="titlepage"><div><div><h5 class="title"><a name="pseudo.config"></a>1.2.2.1.1. Pseudo-distributed Configuration File</h5></div></div></div><p>Below is a sample pseudo-distributed file for the node <code class="varname">h-24-30.example.com</code>.
@@ -761,7 +761,7 @@ config.set("hbase.zookeeper.quorum", "lo
This populated <code class="classname">Configuration</code> instance can then be passed to an
<a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html" target="_top">HTable</a>,
and so on.
- </p></div></div></div><div class="section" title="1.4. Example Configurations"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="example_config"></a>1.4. Example Configurations</h2></div></div></div><div class="section" title="1.4.1. Basic Distributed HBase Install"><div class="titlepage"><div><div><h3 class="title"><a name="d1897e2050"></a>1.4.1. Basic Distributed HBase Install</h3></div></div></div><p>Here is an example basic configuration for a distributed ten
+ </p></div></div></div><div class="section" title="1.4. Example Configurations"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="example_config"></a>1.4. Example Configurations</h2></div></div></div><div class="section" title="1.4.1. Basic Distributed HBase Install"><div class="titlepage"><div><div><h3 class="title"><a name="d1951e2050"></a>1.4.1. Basic Distributed HBase Install</h3></div></div></div><p>Here is an example basic configuration for a distributed ten
node cluster. The nodes are named <code class="varname">example0</code>,
<code class="varname">example1</code>, etc., through node
<code class="varname">example9</code> in this example. The HBase Master and the
@@ -963,7 +963,7 @@ index e70ebc6..96f8c27 100644
Keeping 5 regions per RS would be too low for a job, whereas 1000 will generate too many maps.
</p></div></div><div class="section" title="1.5.2.7. Managed Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="disable.splitting"></a>1.5.2.7. Managed Splitting</h4></div></div></div><p>
Rather than let HBase auto-split your Regions, manage the splitting manually
- <sup>[<a name="d1897e2276" href="#ftn.d1897e2276" class="footnote">11</a>]</sup>.
+ <sup>[<a name="d1951e2276" href="#ftn.d1951e2276" class="footnote">11</a>]</sup>.
With growing amounts of data, splits will continually be needed. Since
you always know exactly what regions you have, long-term debugging and
profiling is much easier with manual splits. It is hard to trace the logs to
@@ -1023,26 +1023,26 @@ of all regions.
<a class="link" href="http://search-hadoop.com/m/pduLg2fydtE/Inconsistent+scan+performance+with+caching+set+&subj=Re+Inconsistent+scan+performance+with+caching+set+to+1" target="_top">Inconsistent scan performance with caching set to 1</a>
and the issue cited therein where setting notcpdelay improved scan speeds. You might also
see the graphs on the tail of <a class="link" href="https://issues.apache.org/jira/browse/HBASE-7008" target="_top">HBASE-7008 Set scanner caching to a better default</a>
- where our Lars Hofhansl tries various data sizes w/ Nagle's on and off measuring the effect.</p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d1897e14" href="#d1897e14" class="para">1</a>] </sup>
+ where our Lars Hofhansl tries various data sizes w/ Nagle's on and off measuring the effect.</p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d1951e14" href="#d1951e14" class="para">1</a>] </sup>
Be careful editing XML. Make sure you close all elements.
Run your file through <span class="command"><strong>xmllint</strong></span> or similar
to ensure well-formedness of your document after an edit session.
-</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e63" href="#d1897e63" class="para">2</a>] </sup>The <a class="link" href="https://github.com/sujee/hadoop-dns-checker" target="_top">hadoop-dns-checker</a> tool can be used to verify
+</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e63" href="#d1951e63" class="para">2</a>] </sup>The <a class="link" href="https://github.com/sujee/hadoop-dns-checker" target="_top">hadoop-dns-checker</a> tool can be used to verify
DNS is working correctly on the cluster. The project README file provides detailed instructions on usage.
-</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e128" href="#d1897e128" class="para">3</a>] </sup>See Jack Levin's <a class="link" href="" target="_top">major hdfs issues</a>
- note up on the user list.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e135" href="#d1897e135" class="para">4</a>] </sup>The requirement that a database requires upping of system limits
+</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e128" href="#d1951e128" class="para">3</a>] </sup>See Jack Levin's <a class="link" href="" target="_top">major hdfs issues</a>
+ note up on the user list.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e135" href="#d1951e135" class="para">4</a>] </sup>The requirement that a database requires upping of system limits
is not peculiar to Apache HBase. See for example the section
<span class="emphasis"><em>Setting Shell Limits for the Oracle User</em></span> in
<a class="link" href="http://www.akadia.com/services/ora_linux_install_10g.html" target="_top">
- Short Guide to install Oracle 10 on Linux</a>.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e147" href="#d1897e147" class="para">5</a>] </sup>A useful read setting config on you hadoop cluster is Aaron
+ Short Guide to install Oracle 10 on Linux</a>.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e147" href="#d1951e147" class="para">5</a>] </sup>A useful read setting config on you hadoop cluster is Aaron
Kimballs' Configuration
- Parameters: What can you just ignore?</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e315" href="#d1897e315" class="para">6</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/7vFVx4EsUb2" target="_top">HBase, mail # dev - DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?</a></p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e329" href="#d1897e329" class="para">7</a>] </sup>The Cloudera blog post <a class="link" href="http://www.cloudera.com/blog/2012/01/an-update-on-apache-hadoop-1-0/" target="_top">An update on Apache Hadoop 1.0</a>
+ Parameters: What can you just ignore?</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e315" href="#d1951e315" class="para">6</a>] </sup>See <a class="link" href="http://search-hadoop.com/m/7vFVx4EsUb2" target="_top">HBase, mail # dev - DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?</a></p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e329" href="#d1951e329" class="para">7</a>] </sup>The Cloudera blog post <a class="link" href="http://www.cloudera.com/blog/2012/01/an-update-on-apache-hadoop-1-0/" target="_top">An update on Apache Hadoop 1.0</a>
by Charles Zedlweski has a nice exposition on how all the Hadoop versions relate.
Its worth checking out if you are having trouble making sense of the
Hadoop version morass.
- </p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e388" href="#d1897e388" class="para">8</a>] </sup>See <a class="link" href="http://ccgtech.blogspot.com/2010/02/hadoop-hdfs-deceived-by-xciever.html" target="_top">Hadoop HDFS: Deceived by Xciever</a> for an informative rant on xceivering.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e446" href="#d1897e446" class="para">9</a>] </sup>The pseudo-distributed vs fully-distributed nomenclature
- comes from Hadoop.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e494" href="#d1897e494" class="para">10</a>] </sup>See <a class="xref" href="#pseudo.extras" title="1.2.2.1.2. Pseudo-distributed Extras">Section 1.2.2.1.2, “Pseudo-distributed Extras”</a> for notes on how to start extra Masters and
- RegionServers when running pseudo-distributed.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1897e2276" href="#d1897e2276" class="para">11</a>] </sup>What follows is taken from the javadoc at the head of
+ </p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e388" href="#d1951e388" class="para">8</a>] </sup>See <a class="link" href="http://ccgtech.blogspot.com/2010/02/hadoop-hdfs-deceived-by-xciever.html" target="_top">Hadoop HDFS: Deceived by Xciever</a> for an informative rant on xceivering.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e446" href="#d1951e446" class="para">9</a>] </sup>The pseudo-distributed vs fully-distributed nomenclature
+ comes from Hadoop.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e494" href="#d1951e494" class="para">10</a>] </sup>See <a class="xref" href="#pseudo.extras" title="1.2.2.1.2. Pseudo-distributed Extras">Section 1.2.2.1.2, “Pseudo-distributed Extras”</a> for notes on how to start extra Masters and
+ RegionServers when running pseudo-distributed.</p></div><div class="footnote"><p><sup>[<a id="ftn.d1951e2276" href="#d1951e2276" class="para">11</a>] </sup>What follows is taken from the javadoc at the head of
the <code class="classname">org.apache.hadoop.hbase.util.RegionSplitter</code> tool
added to HBase post-0.90.0 release.
</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">