You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Raja Kulasekaran <cu...@gmail.com> on 2013/02/21 06:14:20 UTC
Nutch 2.1 / Hbase / Gora / Solr
Hi,
Is the Nutch 2.1 is the stable version ?
I had an issue with compiling gora with maven in my machine .Is there any
compiled version available in online ?
Thanks,
Raja
Re: gora zookeeper error
Posted by Lewis John Mcgibbney <le...@gmail.com>.
I take it you read the mailing list thread I pointed you to?
Unfortunately, although linked problems of this nature as not solved
directly with Nutch.
On Thu, Feb 21, 2013 at 3:47 PM, kaveh minooie <ka...@plutoz.com> wrote:
> thanks man, this worked:
> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
> conf="*->default" >
> <exclude org="org.apache.hbase" name="hbase" />
> </dependency>
>
> <dependency org="org.apache.hbase" name="hbase" rev="0.90.5"
> conf="*->default" />
>
>
> turned out maven repo1 doesn't even have the 90.6 in it, at least yet, :)
> so I tried with 0.90.5, and I am still getting this:
>
> 13/02/21 15:39:37 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/02/21 15:39:37 INFO mapred.JobClient: Running job: job_201302191325_0006
> 13/02/21 15:39:38 INFO mapred.JobClient: map 0% reduce 0%
> 13/02/21 15:40:01 INFO mapred.JobClient: Task Id :
> attempt_201302191325_0006_m_**000000_0, Status : FAILED
> org.apache.gora.util.**GoraException: org.apache.hadoop.hbase.**ZooKeeperConnectionException:
> HBase is able to connect to ZooKeeper but the connection closes
> immediately. This could be a sign that the server has too many connections
> (30 is the default). Consider inspecting your ZK server logs for that error
> and then make sure you are reusing HBaseConfiguration as often as you can.
> See HTable's javadoc for more information.
> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
> DataStoreFactory.java:167)
> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
> DataStoreFactory.java:118)
>
>
> this is output of a nutch inject commoand.
>
> BTW, what is snappy ?
>
>
>
> On 02/21/2013 12:17 PM, Lewis John Mcgibbney wrote:
>
>> Try this
>>
>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>> conf="*->default" >
>> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4" />
>> </dependency>
>> ...
>> <dependency org="org.apache.hbase" name="hbase" rev="0.90.6"
>> conf="*->default" >
>>
>> hth
>>
>>
>> On Thu, Feb 21, 2013 at 11:41 AM, kaveh minooie <ka...@plutoz.com> wrote:
>>
>> this is what i ended up in my ivy.xml:
>>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>>> conf="*->default" >
>>> <!-- exclude org="org.apache.hbase" name="hbase" rev="0.90.4"
>>> /-->
>>> <include org="org.apache.hbase" name="hbase" rev="0.90.6" />
>>> </dependency>
>>>
>>> which caused this:
>>>
>>> BUILD FAILED
>>> /source/nutch/nutch/build.xml:****440: syntax errors in ivy file:
>>> java.text.ParseException: [xml parsing: ivy.xml:115:63:
>>> cvc-complex-type.3.2.2: Attribute 'org' is not allowed to appear in
>>> element
>>> 'include'. in file:/source/nutch/nutch/ivy/****ivy.xml
>>> , xml parsing: ivy.xml:115:63: cvc-complex-type.3.2.2: Attribute 'rev' is
>>> not allowed to appear in element 'include'. in
>>> file:/source/nutch/nutch/ivy/
>>> **ivy.xml
>>>
>>>
>>>
>>> it is the same with exclude tag included as well.
>>>
>>>
>>> On 02/21/2013 11:19 AM, Lewis John Mcgibbney wrote:
>>>
>>> replace
>>>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>>>> conf="*->default" />
>>>> with
>>>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>>>> conf="*->default"
>>>> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4">
>>>> <include org="org.apache.hbase" name="hbase" rev="0.90.6">
>>>> </dependency>
>>>>
>>>> hopefully something like this will work.
>>>>
>>>> hth
>>>>
>>>> On Thu, Feb 21, 2013 at 11:00 AM, kaveh minooie <ka...@plutoz.com>
>>>> wrote:
>>>>
>>>> thanks Lewis.
>>>>
>>>>> I am sorry but i have to bother you with this. could tell me how I can
>>>>> have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to
>>>>> ant
>>>>> report, it is being fetch indirectly by gora 0.2.1. I am actually using
>>>>> 0.90.6 for my hbase, but I don't how modify ivy.xml file to accomplish
>>>>> that.
>>>>>
>>>>> thanks,
>>>>>
>>>>> On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
>>>>>
>>>>> http://s.apache.org/WbGsorry for ridiculous size of font
>>>>>
>>>>>> hth
>>>>>>
>>>>>> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Has anyone encounter this error before:
>>>>>>
>>>>>>>
>>>>>>> org.apache.gora.util.********GoraException:
>>>>>>> org.apache.hadoop.hbase.****
>>>>>>> ZooKeeperConnectionException:
>>>>>>> HBase is able to connect to ZooKeeper but the connection closes
>>>>>>> immediately. This could be a sign that the server has too many
>>>>>>> connections
>>>>>>> (30 is the default). Consider inspecting your ZK server logs for that
>>>>>>> error
>>>>>>> and then make sure you are reusing HBaseConfiguration as often as you
>>>>>>> can.
>>>>>>> See HTable's javadoc for more information.
>>>>>>> at org.apache.gora.store.********DataStoreFactory.****
>>>>>>> createDataStore(**
>>>>>>> DataStoreFactory.java:167)
>>>>>>> at org.apache.gora.store.********DataStoreFactory.****
>>>>>>> createDataStore(**
>>>>>>> DataStoreFactory.java:118)
>>>>>>> at org.apache.gora.mapreduce.******
>>>>>>> **GoraOutputFormat.****
>>>>>>> getRecordWriter(
>>>>>>> **GoraOutputFormat.java:88)
>>>>>>> at org.apache.hadoop.mapred.********MapTask$****
>>>>>>> NewDirectOutputCollector.<
>>>>>>> **init>(MapTask.java:628)
>>>>>>> at org.apache.hadoop.mapred.********
>>>>>>> MapTask.runNewMapper(MapTask.******
>>>>>>> **
>>>>>>> java:753)
>>>>>>> at org.apache.hadoop.mapred.********
>>>>>>> MapTask.run(MapTask.java:370)
>>>>>>> at org.apache.hadoop.mapred.********
>>>>>>> Child$4.run(Child.java:255)
>>>>>>> at java.security.********AccessController.doPrivileged(**
>>>>>>> **
>>>>>>> ****Native
>>>>>>> Method)
>>>>>>> at javax.security.auth.Subject.********doAs(Unknown
>>>>>>> Source)
>>>>>>> at org.apache.hadoop.security.*******
>>>>>>> *UserGroupInformation.doAs(**
>>>>>>> UserGroupInformation.java:********1136)
>>>>>>> at org.apache.hadoop.mapred.*******
>>>>>>> *Child.main(Child.java:249)
>>>>>>> Caused by: org.apache.hadoop.hbase.********
>>>>>>> ZooKeeperConnectionException:
>>>>>>> HBase
>>>>>>> is able to connect to ZooKeeper but the connection closes
>>>>>>> immediately.
>>>>>>> This
>>>>>>> could be a sign that the server has too many connections (30 is the
>>>>>>> default). Consider inspecting your ZK server logs for that error and
>>>>>>> then
>>>>>>> make sure you are reusing HBaseConfiguration as often as you can. See
>>>>>>> HTable's javadoc for more information.
>>>>>>> at org.apache.hadoop.hbase.******
>>>>>>> zookeeper.ZooKeeperWatcher.<**
>>>>>>> init>(ZooKeeperWatcher.java:********155)
>>>>>>> at org.apache.hadoop.hbase.******
>>>>>>> client.HConnectionManager$**
>>>>>>> HConnectionImplementation.********getZooKeeperWatcher(**
>>>>>>> HConnectionManager.java:1002)
>>>>>>> at org.apache.hadoop.hbase.******
>>>>>>> client.HConnectionManager$**
>>>>>>> HConnectionImplementation.********setupZookeeperTrackers(**
>>>>>>> HConnectionManager.java:304)
>>>>>>> at org.apache.hadoop.hbase.******
>>>>>>> client.HConnectionManager$**
>>>>>>> HConnectionImplementation.<********init>(HConnectionManager.**
>>>>>>> java:****295)
>>>>>>> at org.apache.hadoop.hbase.******
>>>>>>> client.HConnectionManager.**
>>>>>>> getConnection(********HConnectionManager.java:157)
>>>>>>> at org.apache.hadoop.hbase.********
>>>>>>> client.HBaseAdmin.<init>(**
>>>>>>> HBaseAdmin.java:90)
>>>>>>> at org.apache.gora.hbase.store.******
>>>>>>> **HBaseStore.initialize(**
>>>>>>> HBaseStore.java:108)
>>>>>>> at org.apache.gora.store.********DataStoreFactory.****
>>>>>>> initializeDataStore(
>>>>>>> **DataStoreFactory.java:102)
>>>>>>> at org.apache.gora.store.********DataStoreFactory.****
>>>>>>> createDataStore(**
>>>>>>> DataStoreFactory.java:161)
>>>>>>>
>>>>>>>
>>>>>>> what configuration in zookeeper is it talking about?
>>>>>>> for the record, I think, this is the error in the zookeeper log file
>>>>>>> that
>>>>>>> it is talking about ( it is the only error in my log file):
>>>>>>>
>>>>>>>
>>>>>>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:
>>>>>>> 0.0.0.0/*** <http://0.0.0.0/*>
>>>>>>> *** <http://0.0.0.0/**>
>>>>>>> ** <http://0.0.0.0/**>
>>>>>>> 0.0.0.0:2181:NIOServerCnxn@349******<http://0.0.0.0/0.0.0.0:****
>>>>>>> 2181:** <http://0.0.0.0/0.0.0.0:**2181:**><http://0.0.0.0/0.0.0.**
>>>>>>> 0:2181:** <http://0.0.0.0/0.0.0.0:2181:**>>
>>>>>>> NIOServerCnxn@349 <ht...@349>
>>>>>>> <ht...@349>
>>>>>>> >
>>>>>>>
>>>>>>>>
>>>>>>>>> **] - caught end of stream exception
>>>>>>> EndOfStreamException: Unable to read additional data from client
>>>>>>> sessionid
>>>>>>> 0x13cfc5527be0017, likely client has closed socket
>>>>>>> at org.apache.zookeeper.server.****
>>>>>>> ****NIOServerCnxn.doIO(**
>>>>>>> NIOServerCnxn.java:220)
>>>>>>> at org.apache.zookeeper.server.******
>>>>>>> **NIOServerCnxnFactory.run(**
>>>>>>> NIOServerCnxnFactory.java:208)
>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:
>>>>>>> 0.0.0.0/*** <http://0.0.0.0/*>
>>>>>>> *** <http://0.0.0.0/**>
>>>>>>> ** <http://0.0.0.0/**>
>>>>>>> 0.0.0.0:2181:NIOServerCnxn@********1001<http://0.0.0.0/0.0.0.**
>>>>>>> 0:**** <http://0.0.0.0/0.0.0.0:****><http://0.0.0.0/0.0.0.0:****<http://0.0.0.0/0.0.0.0:**>
>>>>>>> >
>>>>>>> 2181:NIOServerCnxn@1001 <http://0.0.0.0/0.0.0.0:2181:****<http://0.0.0.0/0.0.0.0:2181:**>
>>>>>>> NIOServerCnxn@1001 <ht...@1001>
>>>>>>> >>
>>>>>>>
>>>>>>> ]
>>>>>>>>
>>>>>>>> - Closed socket connection for client /127.0.0.1:42306 which had
>>>>>>> sessionid 0x13cfc5527be0017
>>>>>>>
>>>>>>> I am also concern about this happening on the loop back interface.
>>>>>>> doesn't
>>>>>>> look right to me.
>>>>>>>
>>>>>>> oh and btw this is on hadoop cluster with 10 nodes
>>>>>>> --
>>>>>>> Kaveh Minooie
>>>>>>>
>>>>>>> www.plutoz.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>> Kaveh Minooie
>>>>>
>>>>> www.plutoz.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>> Kaveh Minooie
>>>
>>> www.plutoz.com
>>>
>>>
>>
>>
>>
> --
> Kaveh Minooie
>
> www.plutoz.com
>
--
*Lewis*
Re: gora zookeeper error
Posted by kaveh minooie <ka...@plutoz.com>.
thanks man, this worked:
<dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
conf="*->default" >
<exclude org="org.apache.hbase" name="hbase" />
</dependency>
<dependency org="org.apache.hbase" name="hbase" rev="0.90.5"
conf="*->default" />
turned out maven repo1 doesn't even have the 90.6 in it, at least yet,
:) so I tried with 0.90.5, and I am still getting this:
13/02/21 15:39:37 WARN snappy.LoadSnappy: Snappy native library not loaded
13/02/21 15:39:37 INFO mapred.JobClient: Running job: job_201302191325_0006
13/02/21 15:39:38 INFO mapred.JobClient: map 0% reduce 0%
13/02/21 15:40:01 INFO mapred.JobClient: Task Id :
attempt_201302191325_0006_m_000000_0, Status : FAILED
org.apache.gora.util.GoraException:
org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to
connect to ZooKeeper but the connection closes immediately. This could
be a sign that the server has too many connections (30 is the default).
Consider inspecting your ZK server logs for that error and then make
sure you are reusing HBaseConfiguration as often as you can. See
HTable's javadoc for more information.
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:167)
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
this is output of a nutch inject commoand.
BTW, what is snappy ?
On 02/21/2013 12:17 PM, Lewis John Mcgibbney wrote:
> Try this
>
> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
> conf="*->default" >
> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4" />
> </dependency>
> ...
> <dependency org="org.apache.hbase" name="hbase" rev="0.90.6"
> conf="*->default" >
>
> hth
>
>
> On Thu, Feb 21, 2013 at 11:41 AM, kaveh minooie <ka...@plutoz.com> wrote:
>
>> this is what i ended up in my ivy.xml:
>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>> conf="*->default" >
>> <!-- exclude org="org.apache.hbase" name="hbase" rev="0.90.4" /-->
>> <include org="org.apache.hbase" name="hbase" rev="0.90.6" />
>> </dependency>
>>
>> which caused this:
>>
>> BUILD FAILED
>> /source/nutch/nutch/build.xml:**440: syntax errors in ivy file:
>> java.text.ParseException: [xml parsing: ivy.xml:115:63:
>> cvc-complex-type.3.2.2: Attribute 'org' is not allowed to appear in element
>> 'include'. in file:/source/nutch/nutch/ivy/**ivy.xml
>> , xml parsing: ivy.xml:115:63: cvc-complex-type.3.2.2: Attribute 'rev' is
>> not allowed to appear in element 'include'. in file:/source/nutch/nutch/ivy/
>> **ivy.xml
>>
>>
>>
>> it is the same with exclude tag included as well.
>>
>>
>> On 02/21/2013 11:19 AM, Lewis John Mcgibbney wrote:
>>
>>> replace
>>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>>> conf="*->default" />
>>> with
>>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>>> conf="*->default"
>>> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4">
>>> <include org="org.apache.hbase" name="hbase" rev="0.90.6">
>>> </dependency>
>>>
>>> hopefully something like this will work.
>>>
>>> hth
>>>
>>> On Thu, Feb 21, 2013 at 11:00 AM, kaveh minooie <ka...@plutoz.com> wrote:
>>>
>>> thanks Lewis.
>>>> I am sorry but i have to bother you with this. could tell me how I can
>>>> have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to
>>>> ant
>>>> report, it is being fetch indirectly by gora 0.2.1. I am actually using
>>>> 0.90.6 for my hbase, but I don't how modify ivy.xml file to accomplish
>>>> that.
>>>>
>>>> thanks,
>>>>
>>>> On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
>>>>
>>>> http://s.apache.org/WbGsorry for ridiculous size of font
>>>>> hth
>>>>>
>>>>> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> Has anyone encounter this error before:
>>>>>>
>>>>>> org.apache.gora.util.******GoraException: org.apache.hadoop.hbase.****
>>>>>> ZooKeeperConnectionException:
>>>>>> HBase is able to connect to ZooKeeper but the connection closes
>>>>>> immediately. This could be a sign that the server has too many
>>>>>> connections
>>>>>> (30 is the default). Consider inspecting your ZK server logs for that
>>>>>> error
>>>>>> and then make sure you are reusing HBaseConfiguration as often as you
>>>>>> can.
>>>>>> See HTable's javadoc for more information.
>>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>>> createDataStore(**
>>>>>> DataStoreFactory.java:167)
>>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>>> createDataStore(**
>>>>>> DataStoreFactory.java:118)
>>>>>> at org.apache.gora.mapreduce.******GoraOutputFormat.****
>>>>>> getRecordWriter(
>>>>>> **GoraOutputFormat.java:88)
>>>>>> at org.apache.hadoop.mapred.******MapTask$****
>>>>>> NewDirectOutputCollector.<
>>>>>> **init>(MapTask.java:628)
>>>>>> at org.apache.hadoop.mapred.******
>>>>>> MapTask.runNewMapper(MapTask.****
>>>>>> **
>>>>>> java:753)
>>>>>> at org.apache.hadoop.mapred.******
>>>>>> MapTask.run(MapTask.java:370)
>>>>>> at org.apache.hadoop.mapred.******
>>>>>> Child$4.run(Child.java:255)
>>>>>> at java.security.******AccessController.doPrivileged(**
>>>>>> ****Native
>>>>>> Method)
>>>>>> at javax.security.auth.Subject.******doAs(Unknown Source)
>>>>>> at org.apache.hadoop.security.*****
>>>>>> *UserGroupInformation.doAs(**
>>>>>> UserGroupInformation.java:******1136)
>>>>>> at org.apache.hadoop.mapred.******Child.main(Child.java:249)
>>>>>> Caused by: org.apache.hadoop.hbase.******ZooKeeperConnectionException:
>>>>>> HBase
>>>>>> is able to connect to ZooKeeper but the connection closes immediately.
>>>>>> This
>>>>>> could be a sign that the server has too many connections (30 is the
>>>>>> default). Consider inspecting your ZK server logs for that error and
>>>>>> then
>>>>>> make sure you are reusing HBaseConfiguration as often as you can. See
>>>>>> HTable's javadoc for more information.
>>>>>> at org.apache.hadoop.hbase.******
>>>>>> zookeeper.ZooKeeperWatcher.<**
>>>>>> init>(ZooKeeperWatcher.java:******155)
>>>>>> at org.apache.hadoop.hbase.******
>>>>>> client.HConnectionManager$**
>>>>>> HConnectionImplementation.******getZooKeeperWatcher(**
>>>>>> HConnectionManager.java:1002)
>>>>>> at org.apache.hadoop.hbase.******
>>>>>> client.HConnectionManager$**
>>>>>> HConnectionImplementation.******setupZookeeperTrackers(**
>>>>>> HConnectionManager.java:304)
>>>>>> at org.apache.hadoop.hbase.******
>>>>>> client.HConnectionManager$**
>>>>>> HConnectionImplementation.<******init>(HConnectionManager.**
>>>>>> java:****295)
>>>>>> at org.apache.hadoop.hbase.******
>>>>>> client.HConnectionManager.**
>>>>>> getConnection(******HConnectionManager.java:157)
>>>>>> at org.apache.hadoop.hbase.******client.HBaseAdmin.<init>(**
>>>>>> HBaseAdmin.java:90)
>>>>>> at org.apache.gora.hbase.store.****
>>>>>> **HBaseStore.initialize(**
>>>>>> HBaseStore.java:108)
>>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>>> initializeDataStore(
>>>>>> **DataStoreFactory.java:102)
>>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>>> createDataStore(**
>>>>>> DataStoreFactory.java:161)
>>>>>>
>>>>>>
>>>>>> what configuration in zookeeper is it talking about?
>>>>>> for the record, I think, this is the error in the zookeeper log file
>>>>>> that
>>>>>> it is talking about ( it is the only error in my log file):
>>>>>>
>>>>>>
>>>>>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/*
>>>>>> *** <http://0.0.0.0/**>
>>>>>> ** <http://0.0.0.0/**>
>>>>>> 0.0.0.0:2181:NIOServerCnxn@349****<http://0.0.0.0/0.0.0.0:**2181:**<http://0.0.0.0/0.0.0.0:2181:**>
>>>>>> NIOServerCnxn@349 <ht...@349>
>>>>>>>>
>>>>>> **] - caught end of stream exception
>>>>>> EndOfStreamException: Unable to read additional data from client
>>>>>> sessionid
>>>>>> 0x13cfc5527be0017, likely client has closed socket
>>>>>> at org.apache.zookeeper.server.******NIOServerCnxn.doIO(**
>>>>>> NIOServerCnxn.java:220)
>>>>>> at org.apache.zookeeper.server.****
>>>>>> **NIOServerCnxnFactory.run(**
>>>>>> NIOServerCnxnFactory.java:208)
>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/*
>>>>>> *** <http://0.0.0.0/**>
>>>>>> ** <http://0.0.0.0/**>
>>>>>> 0.0.0.0:2181:NIOServerCnxn@******1001<http://0.0.0.0/0.0.0.0:****<http://0.0.0.0/0.0.0.0:**>
>>>>>> 2181:NIOServerCnxn@1001 <http://0.0.0.0/0.0.0.0:2181:**
>>>>>> NIOServerCnxn@1001 <ht...@1001>>
>>>>>>
>>>>>>> ]
>>>>>>>
>>>>>> - Closed socket connection for client /127.0.0.1:42306 which had
>>>>>> sessionid 0x13cfc5527be0017
>>>>>>
>>>>>> I am also concern about this happening on the loop back interface.
>>>>>> doesn't
>>>>>> look right to me.
>>>>>>
>>>>>> oh and btw this is on hadoop cluster with 10 nodes
>>>>>> --
>>>>>> Kaveh Minooie
>>>>>>
>>>>>> www.plutoz.com
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>> Kaveh Minooie
>>>>
>>>> www.plutoz.com
>>>>
>>>>
>>>
>>>
>>>
>> --
>> Kaveh Minooie
>>
>> www.plutoz.com
>>
>
>
>
--
Kaveh Minooie
www.plutoz.com
Re: gora zookeeper error
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Try this
<dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
conf="*->default" >
<exclude org="org.apache.hbase" name="hbase" rev="0.90.4" />
</dependency>
...
<dependency org="org.apache.hbase" name="hbase" rev="0.90.6"
conf="*->default" >
hth
On Thu, Feb 21, 2013 at 11:41 AM, kaveh minooie <ka...@plutoz.com> wrote:
> this is what i ended up in my ivy.xml:
> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
> conf="*->default" >
> <!-- exclude org="org.apache.hbase" name="hbase" rev="0.90.4" /-->
> <include org="org.apache.hbase" name="hbase" rev="0.90.6" />
> </dependency>
>
> which caused this:
>
> BUILD FAILED
> /source/nutch/nutch/build.xml:**440: syntax errors in ivy file:
> java.text.ParseException: [xml parsing: ivy.xml:115:63:
> cvc-complex-type.3.2.2: Attribute 'org' is not allowed to appear in element
> 'include'. in file:/source/nutch/nutch/ivy/**ivy.xml
> , xml parsing: ivy.xml:115:63: cvc-complex-type.3.2.2: Attribute 'rev' is
> not allowed to appear in element 'include'. in file:/source/nutch/nutch/ivy/
> **ivy.xml
>
>
>
> it is the same with exclude tag included as well.
>
>
> On 02/21/2013 11:19 AM, Lewis John Mcgibbney wrote:
>
>> replace
>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>> conf="*->default" />
>> with
>> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
>> conf="*->default"
>> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4">
>> <include org="org.apache.hbase" name="hbase" rev="0.90.6">
>> </dependency>
>>
>> hopefully something like this will work.
>>
>> hth
>>
>> On Thu, Feb 21, 2013 at 11:00 AM, kaveh minooie <ka...@plutoz.com> wrote:
>>
>> thanks Lewis.
>>> I am sorry but i have to bother you with this. could tell me how I can
>>> have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to
>>> ant
>>> report, it is being fetch indirectly by gora 0.2.1. I am actually using
>>> 0.90.6 for my hbase, but I don't how modify ivy.xml file to accomplish
>>> that.
>>>
>>> thanks,
>>>
>>> On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
>>>
>>> http://s.apache.org/WbGsorry for ridiculous size of font
>>>> hth
>>>>
>>>> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com>
>>>> wrote:
>>>>
>>>>
>>>> Has anyone encounter this error before:
>>>>>
>>>>> org.apache.gora.util.******GoraException: org.apache.hadoop.hbase.****
>>>>> ZooKeeperConnectionException:
>>>>> HBase is able to connect to ZooKeeper but the connection closes
>>>>> immediately. This could be a sign that the server has too many
>>>>> connections
>>>>> (30 is the default). Consider inspecting your ZK server logs for that
>>>>> error
>>>>> and then make sure you are reusing HBaseConfiguration as often as you
>>>>> can.
>>>>> See HTable's javadoc for more information.
>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>> createDataStore(**
>>>>> DataStoreFactory.java:167)
>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>> createDataStore(**
>>>>> DataStoreFactory.java:118)
>>>>> at org.apache.gora.mapreduce.******GoraOutputFormat.****
>>>>> getRecordWriter(
>>>>> **GoraOutputFormat.java:88)
>>>>> at org.apache.hadoop.mapred.******MapTask$****
>>>>> NewDirectOutputCollector.<
>>>>> **init>(MapTask.java:628)
>>>>> at org.apache.hadoop.mapred.******
>>>>> MapTask.runNewMapper(MapTask.****
>>>>> **
>>>>> java:753)
>>>>> at org.apache.hadoop.mapred.******
>>>>> MapTask.run(MapTask.java:370)
>>>>> at org.apache.hadoop.mapred.******
>>>>> Child$4.run(Child.java:255)
>>>>> at java.security.******AccessController.doPrivileged(**
>>>>> ****Native
>>>>> Method)
>>>>> at javax.security.auth.Subject.******doAs(Unknown Source)
>>>>> at org.apache.hadoop.security.*****
>>>>> *UserGroupInformation.doAs(**
>>>>> UserGroupInformation.java:******1136)
>>>>> at org.apache.hadoop.mapred.******Child.main(Child.java:249)
>>>>> Caused by: org.apache.hadoop.hbase.******ZooKeeperConnectionException:
>>>>> HBase
>>>>> is able to connect to ZooKeeper but the connection closes immediately.
>>>>> This
>>>>> could be a sign that the server has too many connections (30 is the
>>>>> default). Consider inspecting your ZK server logs for that error and
>>>>> then
>>>>> make sure you are reusing HBaseConfiguration as often as you can. See
>>>>> HTable's javadoc for more information.
>>>>> at org.apache.hadoop.hbase.******
>>>>> zookeeper.ZooKeeperWatcher.<**
>>>>> init>(ZooKeeperWatcher.java:******155)
>>>>> at org.apache.hadoop.hbase.******
>>>>> client.HConnectionManager$**
>>>>> HConnectionImplementation.******getZooKeeperWatcher(**
>>>>> HConnectionManager.java:1002)
>>>>> at org.apache.hadoop.hbase.******
>>>>> client.HConnectionManager$**
>>>>> HConnectionImplementation.******setupZookeeperTrackers(**
>>>>> HConnectionManager.java:304)
>>>>> at org.apache.hadoop.hbase.******
>>>>> client.HConnectionManager$**
>>>>> HConnectionImplementation.<******init>(HConnectionManager.**
>>>>> java:****295)
>>>>> at org.apache.hadoop.hbase.******
>>>>> client.HConnectionManager.**
>>>>> getConnection(******HConnectionManager.java:157)
>>>>> at org.apache.hadoop.hbase.******client.HBaseAdmin.<init>(**
>>>>> HBaseAdmin.java:90)
>>>>> at org.apache.gora.hbase.store.****
>>>>> **HBaseStore.initialize(**
>>>>> HBaseStore.java:108)
>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>> initializeDataStore(
>>>>> **DataStoreFactory.java:102)
>>>>> at org.apache.gora.store.******DataStoreFactory.****
>>>>> createDataStore(**
>>>>> DataStoreFactory.java:161)
>>>>>
>>>>>
>>>>> what configuration in zookeeper is it talking about?
>>>>> for the record, I think, this is the error in the zookeeper log file
>>>>> that
>>>>> it is talking about ( it is the only error in my log file):
>>>>>
>>>>>
>>>>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/*
>>>>> *** <http://0.0.0.0/**>
>>>>> ** <http://0.0.0.0/**>
>>>>> 0.0.0.0:2181:NIOServerCnxn@349****<http://0.0.0.0/0.0.0.0:**2181:**<http://0.0.0.0/0.0.0.0:2181:**>
>>>>> NIOServerCnxn@349 <ht...@349>
>>>>> >>
>>>>> **] - caught end of stream exception
>>>>> EndOfStreamException: Unable to read additional data from client
>>>>> sessionid
>>>>> 0x13cfc5527be0017, likely client has closed socket
>>>>> at org.apache.zookeeper.server.******NIOServerCnxn.doIO(**
>>>>> NIOServerCnxn.java:220)
>>>>> at org.apache.zookeeper.server.****
>>>>> **NIOServerCnxnFactory.run(**
>>>>> NIOServerCnxnFactory.java:208)
>>>>> at java.lang.Thread.run(Unknown Source)
>>>>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/*
>>>>> *** <http://0.0.0.0/**>
>>>>> ** <http://0.0.0.0/**>
>>>>> 0.0.0.0:2181:NIOServerCnxn@******1001<http://0.0.0.0/0.0.0.0:****<http://0.0.0.0/0.0.0.0:**>
>>>>> 2181:NIOServerCnxn@1001 <http://0.0.0.0/0.0.0.0:2181:**
>>>>> NIOServerCnxn@1001 <ht...@1001>>
>>>>>
>>>>>> ]
>>>>>>
>>>>> - Closed socket connection for client /127.0.0.1:42306 which had
>>>>> sessionid 0x13cfc5527be0017
>>>>>
>>>>> I am also concern about this happening on the loop back interface.
>>>>> doesn't
>>>>> look right to me.
>>>>>
>>>>> oh and btw this is on hadoop cluster with 10 nodes
>>>>> --
>>>>> Kaveh Minooie
>>>>>
>>>>> www.plutoz.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>> Kaveh Minooie
>>>
>>> www.plutoz.com
>>>
>>>
>>
>>
>>
> --
> Kaveh Minooie
>
> www.plutoz.com
>
--
*Lewis*
Re: gora zookeeper error
Posted by kaveh minooie <ka...@plutoz.com>.
this is what i ended up in my ivy.xml:
<dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
conf="*->default" >
<!-- exclude org="org.apache.hbase" name="hbase" rev="0.90.4" /-->
<include org="org.apache.hbase" name="hbase" rev="0.90.6" />
</dependency>
which caused this:
BUILD FAILED
/source/nutch/nutch/build.xml:440: syntax errors in ivy file:
java.text.ParseException: [xml parsing: ivy.xml:115:63:
cvc-complex-type.3.2.2: Attribute 'org' is not allowed to appear in
element 'include'. in file:/source/nutch/nutch/ivy/ivy.xml
, xml parsing: ivy.xml:115:63: cvc-complex-type.3.2.2: Attribute 'rev'
is not allowed to appear in element 'include'. in
file:/source/nutch/nutch/ivy/ivy.xml
it is the same with exclude tag included as well.
On 02/21/2013 11:19 AM, Lewis John Mcgibbney wrote:
> replace
> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
> conf="*->default" />
> with
> <dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
> conf="*->default"
> <exclude org="org.apache.hbase" name="hbase" rev="0.90.4">
> <include org="org.apache.hbase" name="hbase" rev="0.90.6">
> </dependency>
>
> hopefully something like this will work.
>
> hth
>
> On Thu, Feb 21, 2013 at 11:00 AM, kaveh minooie <ka...@plutoz.com> wrote:
>
>> thanks Lewis.
>> I am sorry but i have to bother you with this. could tell me how I can
>> have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to ant
>> report, it is being fetch indirectly by gora 0.2.1. I am actually using
>> 0.90.6 for my hbase, but I don't how modify ivy.xml file to accomplish that.
>>
>> thanks,
>>
>> On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
>>
>>> http://s.apache.org/WbGsorry for ridiculous size of font
>>> hth
>>>
>>> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com> wrote:
>>>
>>>
>>>> Has anyone encounter this error before:
>>>>
>>>> org.apache.gora.util.****GoraException: org.apache.hadoop.hbase.****
>>>> ZooKeeperConnectionException:
>>>> HBase is able to connect to ZooKeeper but the connection closes
>>>> immediately. This could be a sign that the server has too many
>>>> connections
>>>> (30 is the default). Consider inspecting your ZK server logs for that
>>>> error
>>>> and then make sure you are reusing HBaseConfiguration as often as you
>>>> can.
>>>> See HTable's javadoc for more information.
>>>> at org.apache.gora.store.****DataStoreFactory.****
>>>> createDataStore(**
>>>> DataStoreFactory.java:167)
>>>> at org.apache.gora.store.****DataStoreFactory.****
>>>> createDataStore(**
>>>> DataStoreFactory.java:118)
>>>> at org.apache.gora.mapreduce.****GoraOutputFormat.****
>>>> getRecordWriter(
>>>> **GoraOutputFormat.java:88)
>>>> at org.apache.hadoop.mapred.****MapTask$****
>>>> NewDirectOutputCollector.<
>>>> **init>(MapTask.java:628)
>>>> at org.apache.hadoop.mapred.****MapTask.runNewMapper(MapTask.**
>>>> **
>>>> java:753)
>>>> at org.apache.hadoop.mapred.****MapTask.run(MapTask.java:370)
>>>> at org.apache.hadoop.mapred.****Child$4.run(Child.java:255)
>>>> at java.security.****AccessController.doPrivileged(****Native
>>>> Method)
>>>> at javax.security.auth.Subject.****doAs(Unknown Source)
>>>> at org.apache.hadoop.security.****UserGroupInformation.doAs(**
>>>> UserGroupInformation.java:****1136)
>>>> at org.apache.hadoop.mapred.****Child.main(Child.java:249)
>>>> Caused by: org.apache.hadoop.hbase.****ZooKeeperConnectionException:
>>>> HBase
>>>> is able to connect to ZooKeeper but the connection closes immediately.
>>>> This
>>>> could be a sign that the server has too many connections (30 is the
>>>> default). Consider inspecting your ZK server logs for that error and then
>>>> make sure you are reusing HBaseConfiguration as often as you can. See
>>>> HTable's javadoc for more information.
>>>> at org.apache.hadoop.hbase.****zookeeper.ZooKeeperWatcher.<**
>>>> init>(ZooKeeperWatcher.java:****155)
>>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>>> HConnectionImplementation.****getZooKeeperWatcher(**
>>>> HConnectionManager.java:1002)
>>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>>> HConnectionImplementation.****setupZookeeperTrackers(**
>>>> HConnectionManager.java:304)
>>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>>> HConnectionImplementation.<****init>(HConnectionManager.java:****295)
>>>> at org.apache.hadoop.hbase.****client.HConnectionManager.**
>>>> getConnection(****HConnectionManager.java:157)
>>>> at org.apache.hadoop.hbase.****client.HBaseAdmin.<init>(**
>>>> HBaseAdmin.java:90)
>>>> at org.apache.gora.hbase.store.****HBaseStore.initialize(**
>>>> HBaseStore.java:108)
>>>> at org.apache.gora.store.****DataStoreFactory.****
>>>> initializeDataStore(
>>>> **DataStoreFactory.java:102)
>>>> at org.apache.gora.store.****DataStoreFactory.****
>>>> createDataStore(**
>>>> DataStoreFactory.java:161)
>>>>
>>>>
>>>> what configuration in zookeeper is it talking about?
>>>> for the record, I think, this is the error in the zookeeper log file that
>>>> it is talking about ( it is the only error in my log file):
>>>>
>>>>
>>>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/**
>>>> ** <http://0.0.0.0/**>
>>>> 0.0.0.0:2181:NIOServerCnxn@349**<http://0.0.0.0/0.0.0.0:2181:**
>>>> NIOServerCnxn@349 <ht...@349>>
>>>> **] - caught end of stream exception
>>>> EndOfStreamException: Unable to read additional data from client
>>>> sessionid
>>>> 0x13cfc5527be0017, likely client has closed socket
>>>> at org.apache.zookeeper.server.****NIOServerCnxn.doIO(**
>>>> NIOServerCnxn.java:220)
>>>> at org.apache.zookeeper.server.****NIOServerCnxnFactory.run(**
>>>> NIOServerCnxnFactory.java:208)
>>>> at java.lang.Thread.run(Unknown Source)
>>>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/**
>>>> ** <http://0.0.0.0/**>
>>>> 0.0.0.0:2181:NIOServerCnxn@****1001<http://0.0.0.0/0.0.0.0:**
>>>> 2181:NIOServerCnxn@1001 <ht...@1001>
>>>>> ]
>>>> - Closed socket connection for client /127.0.0.1:42306 which had
>>>> sessionid 0x13cfc5527be0017
>>>>
>>>> I am also concern about this happening on the loop back interface.
>>>> doesn't
>>>> look right to me.
>>>>
>>>> oh and btw this is on hadoop cluster with 10 nodes
>>>> --
>>>> Kaveh Minooie
>>>>
>>>> www.plutoz.com
>>>>
>>>>
>>>
>>>
>>>
>> --
>> Kaveh Minooie
>>
>> www.plutoz.com
>>
>
>
>
--
Kaveh Minooie
www.plutoz.com
Re: gora zookeeper error
Posted by Lewis John Mcgibbney <le...@gmail.com>.
replace
<dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
conf="*->default" />
with
<dependency org="org.apache.gora" name="gora-hbase" rev="0.2.1"
conf="*->default"
<exclude org="org.apache.hbase" name="hbase" rev="0.90.4">
<include org="org.apache.hbase" name="hbase" rev="0.90.6">
</dependency>
hopefully something like this will work.
hth
On Thu, Feb 21, 2013 at 11:00 AM, kaveh minooie <ka...@plutoz.com> wrote:
> thanks Lewis.
> I am sorry but i have to bother you with this. could tell me how I can
> have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to ant
> report, it is being fetch indirectly by gora 0.2.1. I am actually using
> 0.90.6 for my hbase, but I don't how modify ivy.xml file to accomplish that.
>
> thanks,
>
> On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
>
>> http://s.apache.org/WbGsorry for ridiculous size of font
>> hth
>>
>> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com> wrote:
>>
>>
>>> Has anyone encounter this error before:
>>>
>>> org.apache.gora.util.****GoraException: org.apache.hadoop.hbase.****
>>> ZooKeeperConnectionException:
>>> HBase is able to connect to ZooKeeper but the connection closes
>>> immediately. This could be a sign that the server has too many
>>> connections
>>> (30 is the default). Consider inspecting your ZK server logs for that
>>> error
>>> and then make sure you are reusing HBaseConfiguration as often as you
>>> can.
>>> See HTable's javadoc for more information.
>>> at org.apache.gora.store.****DataStoreFactory.****
>>> createDataStore(**
>>> DataStoreFactory.java:167)
>>> at org.apache.gora.store.****DataStoreFactory.****
>>> createDataStore(**
>>> DataStoreFactory.java:118)
>>> at org.apache.gora.mapreduce.****GoraOutputFormat.****
>>> getRecordWriter(
>>> **GoraOutputFormat.java:88)
>>> at org.apache.hadoop.mapred.****MapTask$****
>>> NewDirectOutputCollector.<
>>> **init>(MapTask.java:628)
>>> at org.apache.hadoop.mapred.****MapTask.runNewMapper(MapTask.**
>>> **
>>> java:753)
>>> at org.apache.hadoop.mapred.****MapTask.run(MapTask.java:370)
>>> at org.apache.hadoop.mapred.****Child$4.run(Child.java:255)
>>> at java.security.****AccessController.doPrivileged(****Native
>>> Method)
>>> at javax.security.auth.Subject.****doAs(Unknown Source)
>>> at org.apache.hadoop.security.****UserGroupInformation.doAs(**
>>> UserGroupInformation.java:****1136)
>>> at org.apache.hadoop.mapred.****Child.main(Child.java:249)
>>> Caused by: org.apache.hadoop.hbase.****ZooKeeperConnectionException:
>>> HBase
>>> is able to connect to ZooKeeper but the connection closes immediately.
>>> This
>>> could be a sign that the server has too many connections (30 is the
>>> default). Consider inspecting your ZK server logs for that error and then
>>> make sure you are reusing HBaseConfiguration as often as you can. See
>>> HTable's javadoc for more information.
>>> at org.apache.hadoop.hbase.****zookeeper.ZooKeeperWatcher.<**
>>> init>(ZooKeeperWatcher.java:****155)
>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>> HConnectionImplementation.****getZooKeeperWatcher(**
>>> HConnectionManager.java:1002)
>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>> HConnectionImplementation.****setupZookeeperTrackers(**
>>> HConnectionManager.java:304)
>>> at org.apache.hadoop.hbase.****client.HConnectionManager$**
>>> HConnectionImplementation.<****init>(HConnectionManager.java:****295)
>>> at org.apache.hadoop.hbase.****client.HConnectionManager.**
>>> getConnection(****HConnectionManager.java:157)
>>> at org.apache.hadoop.hbase.****client.HBaseAdmin.<init>(**
>>> HBaseAdmin.java:90)
>>> at org.apache.gora.hbase.store.****HBaseStore.initialize(**
>>> HBaseStore.java:108)
>>> at org.apache.gora.store.****DataStoreFactory.****
>>> initializeDataStore(
>>> **DataStoreFactory.java:102)
>>> at org.apache.gora.store.****DataStoreFactory.****
>>> createDataStore(**
>>> DataStoreFactory.java:161)
>>>
>>>
>>> what configuration in zookeeper is it talking about?
>>> for the record, I think, this is the error in the zookeeper log file that
>>> it is talking about ( it is the only error in my log file):
>>>
>>>
>>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/**
>>> ** <http://0.0.0.0/**>
>>> 0.0.0.0:2181:NIOServerCnxn@349**<http://0.0.0.0/0.0.0.0:2181:**
>>> NIOServerCnxn@349 <ht...@349>>
>>> **] - caught end of stream exception
>>> EndOfStreamException: Unable to read additional data from client
>>> sessionid
>>> 0x13cfc5527be0017, likely client has closed socket
>>> at org.apache.zookeeper.server.****NIOServerCnxn.doIO(**
>>> NIOServerCnxn.java:220)
>>> at org.apache.zookeeper.server.****NIOServerCnxnFactory.run(**
>>> NIOServerCnxnFactory.java:208)
>>> at java.lang.Thread.run(Unknown Source)
>>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/**
>>> ** <http://0.0.0.0/**>
>>> 0.0.0.0:2181:NIOServerCnxn@****1001<http://0.0.0.0/0.0.0.0:**
>>> 2181:NIOServerCnxn@1001 <ht...@1001>
>>> >]
>>> - Closed socket connection for client /127.0.0.1:42306 which had
>>> sessionid 0x13cfc5527be0017
>>>
>>> I am also concern about this happening on the loop back interface.
>>> doesn't
>>> look right to me.
>>>
>>> oh and btw this is on hadoop cluster with 10 nodes
>>> --
>>> Kaveh Minooie
>>>
>>> www.plutoz.com
>>>
>>>
>>
>>
>>
> --
> Kaveh Minooie
>
> www.plutoz.com
>
--
*Lewis*
Re: gora zookeeper error
Posted by kaveh minooie <ka...@plutoz.com>.
thanks Lewis.
I am sorry but i have to bother you with this. could tell me how I can
have nutch ant fetch the 0.90.6 instead of 0.90.4 because, according to
ant report, it is being fetch indirectly by gora 0.2.1. I am actually
using 0.90.6 for my hbase, but I don't how modify ivy.xml file to
accomplish that.
thanks,
On 02/21/2013 10:39 AM, Lewis John Mcgibbney wrote:
> http://s.apache.org/WbGsorry for ridiculous size of font
> hth
>
> On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com> wrote:
>
>>
>> Has anyone encounter this error before:
>>
>> org.apache.gora.util.**GoraException: org.apache.hadoop.hbase.**ZooKeeperConnectionException:
>> HBase is able to connect to ZooKeeper but the connection closes
>> immediately. This could be a sign that the server has too many connections
>> (30 is the default). Consider inspecting your ZK server logs for that error
>> and then make sure you are reusing HBaseConfiguration as often as you can.
>> See HTable's javadoc for more information.
>> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
>> DataStoreFactory.java:167)
>> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
>> DataStoreFactory.java:118)
>> at org.apache.gora.mapreduce.**GoraOutputFormat.**getRecordWriter(
>> **GoraOutputFormat.java:88)
>> at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.<
>> **init>(MapTask.java:628)
>> at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**
>> java:753)
>> at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:370)
>> at org.apache.hadoop.mapred.**Child$4.run(Child.java:255)
>> at java.security.**AccessController.doPrivileged(**Native Method)
>> at javax.security.auth.Subject.**doAs(Unknown Source)
>> at org.apache.hadoop.security.**UserGroupInformation.doAs(**
>> UserGroupInformation.java:**1136)
>> at org.apache.hadoop.mapred.**Child.main(Child.java:249)
>> Caused by: org.apache.hadoop.hbase.**ZooKeeperConnectionException: HBase
>> is able to connect to ZooKeeper but the connection closes immediately. This
>> could be a sign that the server has too many connections (30 is the
>> default). Consider inspecting your ZK server logs for that error and then
>> make sure you are reusing HBaseConfiguration as often as you can. See
>> HTable's javadoc for more information.
>> at org.apache.hadoop.hbase.**zookeeper.ZooKeeperWatcher.<**
>> init>(ZooKeeperWatcher.java:**155)
>> at org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**getZooKeeperWatcher(**
>> HConnectionManager.java:1002)
>> at org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.**setupZookeeperTrackers(**
>> HConnectionManager.java:304)
>> at org.apache.hadoop.hbase.**client.HConnectionManager$**
>> HConnectionImplementation.<**init>(HConnectionManager.java:**295)
>> at org.apache.hadoop.hbase.**client.HConnectionManager.**
>> getConnection(**HConnectionManager.java:157)
>> at org.apache.hadoop.hbase.**client.HBaseAdmin.<init>(**
>> HBaseAdmin.java:90)
>> at org.apache.gora.hbase.store.**HBaseStore.initialize(**
>> HBaseStore.java:108)
>> at org.apache.gora.store.**DataStoreFactory.**initializeDataStore(
>> **DataStoreFactory.java:102)
>> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
>> DataStoreFactory.java:161)
>>
>>
>> what configuration in zookeeper is it talking about?
>> for the record, I think, this is the error in the zookeeper log file that
>> it is talking about ( it is the only error in my log file):
>>
>>
>> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/**
>> 0.0.0.0:2181:NIOServerCnxn@349<ht...@349>
>> **] - caught end of stream exception
>> EndOfStreamException: Unable to read additional data from client sessionid
>> 0x13cfc5527be0017, likely client has closed socket
>> at org.apache.zookeeper.server.**NIOServerCnxn.doIO(**
>> NIOServerCnxn.java:220)
>> at org.apache.zookeeper.server.**NIOServerCnxnFactory.run(**
>> NIOServerCnxnFactory.java:208)
>> at java.lang.Thread.run(Unknown Source)
>> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/**
>> 0.0.0.0:2181:NIOServerCnxn@**1001<ht...@1001>]
>> - Closed socket connection for client /127.0.0.1:42306 which had
>> sessionid 0x13cfc5527be0017
>>
>> I am also concern about this happening on the loop back interface. doesn't
>> look right to me.
>>
>> oh and btw this is on hadoop cluster with 10 nodes
>> --
>> Kaveh Minooie
>>
>> www.plutoz.com
>>
>
>
>
--
Kaveh Minooie
www.plutoz.com
Re: gora zookeeper error
Posted by Lewis John Mcgibbney <le...@gmail.com>.
http://s.apache.org/WbGsorry for ridiculous size of font
hth
On Thu, Feb 21, 2013 at 10:31 AM, kaveh minooie <ka...@plutoz.com> wrote:
>
> Has anyone encounter this error before:
>
> org.apache.gora.util.**GoraException: org.apache.hadoop.hbase.**ZooKeeperConnectionException:
> HBase is able to connect to ZooKeeper but the connection closes
> immediately. This could be a sign that the server has too many connections
> (30 is the default). Consider inspecting your ZK server logs for that error
> and then make sure you are reusing HBaseConfiguration as often as you can.
> See HTable's javadoc for more information.
> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
> DataStoreFactory.java:167)
> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
> DataStoreFactory.java:118)
> at org.apache.gora.mapreduce.**GoraOutputFormat.**getRecordWriter(
> **GoraOutputFormat.java:88)
> at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.<
> **init>(MapTask.java:628)
> at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**
> java:753)
> at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.**Child$4.run(Child.java:255)
> at java.security.**AccessController.doPrivileged(**Native Method)
> at javax.security.auth.Subject.**doAs(Unknown Source)
> at org.apache.hadoop.security.**UserGroupInformation.doAs(**
> UserGroupInformation.java:**1136)
> at org.apache.hadoop.mapred.**Child.main(Child.java:249)
> Caused by: org.apache.hadoop.hbase.**ZooKeeperConnectionException: HBase
> is able to connect to ZooKeeper but the connection closes immediately. This
> could be a sign that the server has too many connections (30 is the
> default). Consider inspecting your ZK server logs for that error and then
> make sure you are reusing HBaseConfiguration as often as you can. See
> HTable's javadoc for more information.
> at org.apache.hadoop.hbase.**zookeeper.ZooKeeperWatcher.<**
> init>(ZooKeeperWatcher.java:**155)
> at org.apache.hadoop.hbase.**client.HConnectionManager$**
> HConnectionImplementation.**getZooKeeperWatcher(**
> HConnectionManager.java:1002)
> at org.apache.hadoop.hbase.**client.HConnectionManager$**
> HConnectionImplementation.**setupZookeeperTrackers(**
> HConnectionManager.java:304)
> at org.apache.hadoop.hbase.**client.HConnectionManager$**
> HConnectionImplementation.<**init>(HConnectionManager.java:**295)
> at org.apache.hadoop.hbase.**client.HConnectionManager.**
> getConnection(**HConnectionManager.java:157)
> at org.apache.hadoop.hbase.**client.HBaseAdmin.<init>(**
> HBaseAdmin.java:90)
> at org.apache.gora.hbase.store.**HBaseStore.initialize(**
> HBaseStore.java:108)
> at org.apache.gora.store.**DataStoreFactory.**initializeDataStore(
> **DataStoreFactory.java:102)
> at org.apache.gora.store.**DataStoreFactory.**createDataStore(**
> DataStoreFactory.java:161)
>
>
> what configuration in zookeeper is it talking about?
> for the record, I think, this is the error in the zookeeper log file that
> it is talking about ( it is the only error in my log file):
>
>
> 2013-02-21 02:57:43,099 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/**
> 0.0.0.0:2181:NIOServerCnxn@349<ht...@349>
> **] - caught end of stream exception
> EndOfStreamException: Unable to read additional data from client sessionid
> 0x13cfc5527be0017, likely client has closed socket
> at org.apache.zookeeper.server.**NIOServerCnxn.doIO(**
> NIOServerCnxn.java:220)
> at org.apache.zookeeper.server.**NIOServerCnxnFactory.run(**
> NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Unknown Source)
> 2013-02-21 02:57:43,100 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/**
> 0.0.0.0:2181:NIOServerCnxn@**1001<ht...@1001>]
> - Closed socket connection for client /127.0.0.1:42306 which had
> sessionid 0x13cfc5527be0017
>
> I am also concern about this happening on the loop back interface. doesn't
> look right to me.
>
> oh and btw this is on hadoop cluster with 10 nodes
> --
> Kaveh Minooie
>
> www.plutoz.com
>
--
*Lewis*
gora zookeeper error
Posted by kaveh minooie <ka...@plutoz.com>.
Has anyone encounter this error before:
org.apache.gora.util.GoraException:
org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to
connect to ZooKeeper but the connection closes immediately. This could
be a sign that the server has too many connections (30 is the default).
Consider inspecting your ZK server logs for that error and then make
sure you are reusing HBaseConfiguration as often as you can. See
HTable's javadoc for more information.
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:167)
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
at
org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:88)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:628)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:753)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase
is able to connect to ZooKeeper but the connection closes immediately.
This could be a sign that the server has too many connections (30 is the
default). Consider inspecting your ZK server logs for that error and
then make sure you are reusing HBaseConfiguration as often as you can.
See HTable's javadoc for more information.
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1002)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:304)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:295)
at
org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:157)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:108)
at
org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102)
at
org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161)
what configuration in zookeeper is it talking about?
for the record, I think, this is the error in the zookeeper log file
that it is talking about ( it is the only error in my log file):
2013-02-21 02:57:43,099 [myid:] - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@349] - caught
end of stream exception
EndOfStreamException: Unable to read additional data from client
sessionid 0x13cfc5527be0017, likely client has closed socket
at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Unknown Source)
2013-02-21 02:57:43,100 [myid:] - INFO
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed
socket connection for client /127.0.0.1:42306 which had sessionid
0x13cfc5527be0017
I am also concern about this happening on the loop back interface.
doesn't look right to me.
oh and btw this is on hadoop cluster with 10 nodes
--
Kaveh Minooie
www.plutoz.com
Re: Nutch 2.1 / Hbase / Gora / Solr
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Unfortunately you need to downgrade your hbase distribution.
We currently support 0.90.X
On Tuesday, June 18, 2013, dima <jj...@gmail.com> wrote:
> Hello,
> did anyone have this exception when running Nutch 2.2 with Gora and Hbase
> 0.94.8
> Hbase is up and running(tested the shell)
> but when running nutch inject url
> I get this error:
> InjectorJob: java.io.IOException: No FileSystem for scheme: http
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1408)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1429)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
> at
>
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:372)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:215)
> at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:251)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:273)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:282)
>
>
>
> --
> View this message in context:
http://lucene.472066.n3.nabble.com/Nutch-2-1-Hbase-Gora-Solr-tp4041776p4071335.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
--
*Lewis*
Re: Nutch 2.1 / Hbase / Gora / Solr
Posted by dima <jj...@gmail.com>.
Hello,
did anyone have this exception when running Nutch 2.2 with Gora and Hbase
0.94.8
Hbase is up and running(tested the shell)
but when running nutch inject url
I get this error:
InjectorJob: java.io.IOException: No FileSystem for scheme: http
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1408)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1429)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:372)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:215)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:251)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:273)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:282)
--
View this message in context: http://lucene.472066.n3.nabble.com/Nutch-2-1-Hbase-Gora-Solr-tp4041776p4071335.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Nutch 2.1 / Hbase / Gora / Solr
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Raja,
There are certainly issues with the 2.x branch (of which 2.1 is the most
recent release).
Dependencies are managed via Ivy, so to build 2.1, just use the ant runtime
target.
You can see the Gora artifacts here
http://search.maven.org/#search|ga|1|gora
On Wed, Feb 20, 2013 at 9:14 PM, Raja Kulasekaran <cu...@gmail.com> wrote:
> Hi,
>
> Is the Nutch 2.1 is the stable version ?
>
> I had an issue with compiling gora with maven in my machine .Is there any
> compiled version available in online ?
>
> Thanks,
> Raja
>
--
*Lewis*