You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2010/03/29 23:01:10 UTC
connection.isMasterRunning()
Hi,
While going over TableServers.isMasterRunning() in HConnectionManager, I see
this:
public boolean isMasterRunning() {
if (this.master == null) {
try {
getMaster();
} catch (MasterNotRunningException e) {
return false;
}
}
return true;
}
When isMasterRunning() is called the first time, if master is obtained
successfully, master field would contain reference to HMasterInterface.
Subsequent calls to isMasterRunning() wouldn't throw
MasterNotRunningException even if master server stops responding to clients.
I think master.isMasterRunning() should be called if master isn't null.
Regards
Re: connection.isMasterRunning()
Posted by Ted Yu <yu...@gmail.com>.
More background on why I brought up the issue.
Since we cannot cache hbaseadmin, we call the following code in a dedicated
thread:
try {
HBaseConfiguration _conf = new HBaseConfiguration();
HBaseAdmin _admin = new HBaseAdmin(_conf);
isConnected = true;
return _admin;
} catch (ConcurrentModificationException _e) {
_ex = _e;
}
After some time, I saw:
Exception in thread "HBase-Connector" java.lang.OutOfMemoryError: Java heap
space
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createChunk(DeferredDocumentImpl.java:1921)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.ensureCapacity(DeferredDocumentImpl.java:1826)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createNode(DeferredDocumentImpl.java:1840)
at
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.createDeferredTextNode(DeferredDocumentImpl.java:534)
at
com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1189)
at
com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1082)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:463)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1078)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1029)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:979)
at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1015)
at
org.apache.hadoop.hbase.HBaseConfiguration.hashCode(HBaseConfiguration.java:63)
at java.util.WeakHashMap.put(WeakHashMap.java:401)
at
org.apache.hadoop.conf.Configuration.<init>(Configuration.java:213)
at
org.apache.hadoop.conf.Configuration.<init>(Configuration.java:197)
at
org.apache.hadoop.hbase.HBaseConfiguration.<init>(HBaseConfiguration.java:33)
at
net.ks.datastore.HBaseDataStore.getHBaseAdmin(HBaseDataStore.java:159)
at
net.ks.datastore.HBaseDataStore.isConnected(HBaseDataStore.java:476)
at
net.ks.datastore.HBaseDataStore.access$100(HBaseDataStore.java:39)
at net.ks.datastore.HBaseDataStore$1.run(HBaseDataStore.java:71)
On Mon, Mar 29, 2010 at 2:35 PM, Ted Yu <yu...@gmail.com> wrote:
> https://issues.apache.org/jira/browse/HBASE-2391 has been filed.
>
> We call HBaseAdmin.isMasterRunning() to see if client has connection with
> HBase. If there is a working alternative, we'd love to use it.
>
> Thanks J-D.
>
>
> On Mon, Mar 29, 2010 at 2:25 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> I think this method wasn't updated when we moved to Zookeeper (since
>> in pre-0.20, dead master = dead cluster), also looking at when this is
>> called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
>> which in turn isn't called anywhere in the java code (I think we call
>> it in the shell tho).
>>
>> So we should first consider if this method is useful at all, and if so
>> then what would be the best fix. For example, if you run a HMerge
>> while the master is down but the region servers are up, you're going
>> to end up with something wrong since it expects hbase to be completely
>> down.
>>
>> Can you open a jira to continue discussions there?
>>
>> Thx!
>>
>> J-D
>>
>> On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yu...@gmail.com> wrote:
>> > Hi,
>> > While going over TableServers.isMasterRunning() in HConnectionManager, I
>> see
>> > this:
>> > public boolean isMasterRunning() {
>> > if (this.master == null) {
>> > try {
>> > getMaster();
>> >
>> > } catch (MasterNotRunningException e) {
>> > return false;
>> > }
>> > }
>> > return true;
>> > }
>> > When isMasterRunning() is called the first time, if master is obtained
>> > successfully, master field would contain reference to HMasterInterface.
>> > Subsequent calls to isMasterRunning() wouldn't throw
>> > MasterNotRunningException even if master server stops responding to
>> clients.
>> >
>> > I think master.isMasterRunning() should be called if master isn't null.
>> >
>> > Regards
>> >
>>
>
>
Re: connection.isMasterRunning()
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Check if in Zookeeper the znode /hbase/master is there. You can easily
do this by using
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readMasterAddressOrThrow().
This method isn't static, you have to create your own instance.
J-D
On Tue, May 11, 2010 at 5:38 AM, Al Lias <al...@gmx.de> wrote:
> Am 29.03.2010 23:35, schrieb Ted Yu:
>> https://issues.apache.org/jira/browse/HBASE-2391 has been filed.
>>
>> We call HBaseAdmin.isMasterRunning() to see if client has connection with
>> HBase. If there is a working alternative, we'd love to use it.
>>
>
> Just to catch up on this....what is the recommended way to
> programmatically check if Hbase is up (say on 0.20.4)?
>
> Thx,
>
> Al
>
>> ...
>> On Mon, Mar 29, 2010 at 2:25 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>>
>>> I think this method wasn't updated when we moved to Zookeeper (since
>>> in pre-0.20, dead master = dead cluster), also looking at when this is
>>> called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
>>> which in turn isn't called anywhere in the java code (I think we call
>>> it in the shell tho).
>>>
>>> So we should first consider if this method is useful at all, and if so
>>> then what would be the best fix. For example, if you run a HMerge
>>> while the master is down but the region servers are up, you're going
>>> to end up with something wrong since it expects hbase to be completely
>>> down.
>>>
>>> Can you open a jira to continue discussions there?
>>>
>>> Thx!
>>>
>>> J-D
>>>
>>> On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yu...@gmail.com> wrote:
>>>> Hi,
>>>> While going over TableServers.isMasterRunning() in HConnectionManager, I
>>> see
>>>> this:
>>>> public boolean isMasterRunning() {
>>>> if (this.master == null) {
>>>> try {
>>>> getMaster();
>>>>
>>>> } catch (MasterNotRunningException e) {
>>>> return false;
>>>> }
>>>> }
>>>> return true;
>>>> }
>>>> When isMasterRunning() is called the first time, if master is obtained
>>>> successfully, master field would contain reference to HMasterInterface.
>>>> Subsequent calls to isMasterRunning() wouldn't throw
>>>> MasterNotRunningException even if master server stops responding to
>>> clients.
>>>>
>>>> I think master.isMasterRunning() should be called if master isn't null.
>>>>
>>>> Regards
>>>>
>>>
>>
>
>
Re: connection.isMasterRunning()
Posted by Al Lias <al...@gmx.de>.
Am 29.03.2010 23:35, schrieb Ted Yu:
> https://issues.apache.org/jira/browse/HBASE-2391 has been filed.
>
> We call HBaseAdmin.isMasterRunning() to see if client has connection with
> HBase. If there is a working alternative, we'd love to use it.
>
Just to catch up on this....what is the recommended way to
programmatically check if Hbase is up (say on 0.20.4)?
Thx,
Al
> ...
> On Mon, Mar 29, 2010 at 2:25 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> I think this method wasn't updated when we moved to Zookeeper (since
>> in pre-0.20, dead master = dead cluster), also looking at when this is
>> called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
>> which in turn isn't called anywhere in the java code (I think we call
>> it in the shell tho).
>>
>> So we should first consider if this method is useful at all, and if so
>> then what would be the best fix. For example, if you run a HMerge
>> while the master is down but the region servers are up, you're going
>> to end up with something wrong since it expects hbase to be completely
>> down.
>>
>> Can you open a jira to continue discussions there?
>>
>> Thx!
>>
>> J-D
>>
>> On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yu...@gmail.com> wrote:
>>> Hi,
>>> While going over TableServers.isMasterRunning() in HConnectionManager, I
>> see
>>> this:
>>> public boolean isMasterRunning() {
>>> if (this.master == null) {
>>> try {
>>> getMaster();
>>>
>>> } catch (MasterNotRunningException e) {
>>> return false;
>>> }
>>> }
>>> return true;
>>> }
>>> When isMasterRunning() is called the first time, if master is obtained
>>> successfully, master field would contain reference to HMasterInterface.
>>> Subsequent calls to isMasterRunning() wouldn't throw
>>> MasterNotRunningException even if master server stops responding to
>> clients.
>>>
>>> I think master.isMasterRunning() should be called if master isn't null.
>>>
>>> Regards
>>>
>>
>
Re: connection.isMasterRunning()
Posted by Ted Yu <yu...@gmail.com>.
https://issues.apache.org/jira/browse/HBASE-2391 has been filed.
We call HBaseAdmin.isMasterRunning() to see if client has connection with
HBase. If there is a working alternative, we'd love to use it.
Thanks J-D.
On Mon, Mar 29, 2010 at 2:25 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> I think this method wasn't updated when we moved to Zookeeper (since
> in pre-0.20, dead master = dead cluster), also looking at when this is
> called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
> which in turn isn't called anywhere in the java code (I think we call
> it in the shell tho).
>
> So we should first consider if this method is useful at all, and if so
> then what would be the best fix. For example, if you run a HMerge
> while the master is down but the region servers are up, you're going
> to end up with something wrong since it expects hbase to be completely
> down.
>
> Can you open a jira to continue discussions there?
>
> Thx!
>
> J-D
>
> On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yu...@gmail.com> wrote:
> > Hi,
> > While going over TableServers.isMasterRunning() in HConnectionManager, I
> see
> > this:
> > public boolean isMasterRunning() {
> > if (this.master == null) {
> > try {
> > getMaster();
> >
> > } catch (MasterNotRunningException e) {
> > return false;
> > }
> > }
> > return true;
> > }
> > When isMasterRunning() is called the first time, if master is obtained
> > successfully, master field would contain reference to HMasterInterface.
> > Subsequent calls to isMasterRunning() wouldn't throw
> > MasterNotRunningException even if master server stops responding to
> clients.
> >
> > I think master.isMasterRunning() should be called if master isn't null.
> >
> > Regards
> >
>
Re: connection.isMasterRunning()
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I think this method wasn't updated when we moved to Zookeeper (since
in pre-0.20, dead master = dead cluster), also looking at when this is
called, I only see it from HMerge and HBaseAdmin.isMasterRunning()...
which in turn isn't called anywhere in the java code (I think we call
it in the shell tho).
So we should first consider if this method is useful at all, and if so
then what would be the best fix. For example, if you run a HMerge
while the master is down but the region servers are up, you're going
to end up with something wrong since it expects hbase to be completely
down.
Can you open a jira to continue discussions there?
Thx!
J-D
On Mon, Mar 29, 2010 at 2:01 PM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> While going over TableServers.isMasterRunning() in HConnectionManager, I see
> this:
> public boolean isMasterRunning() {
> if (this.master == null) {
> try {
> getMaster();
>
> } catch (MasterNotRunningException e) {
> return false;
> }
> }
> return true;
> }
> When isMasterRunning() is called the first time, if master is obtained
> successfully, master field would contain reference to HMasterInterface.
> Subsequent calls to isMasterRunning() wouldn't throw
> MasterNotRunningException even if master server stops responding to clients.
>
> I think master.isMasterRunning() should be called if master isn't null.
>
> Regards
>