You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Wim Jongman <wi...@gmail.com> on 2010/08/19 10:15:46 UTC

Zookeeper stops

Hi,

I have a zookeeper server running that can sometimes run for days and then
quits:

Is there somebody with a clue to the problem?

I am running 64 bit Ubuntu with

java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Zookeeper 3.3.0

The log below has some context before it shows the fatal error. Our
component.id=40676 indicates that it is the 40676th time that I ask ZK to
publish this information. It has been seen to go up to half a million before
stopping.

Regards,

Wim

ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:28 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
component.id=40676,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
}]]
ZooDiscovery> Service Published: Aug 18, 2010 11:17:29 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
component.id=40677,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
}]]
[log;+0200 2010.08.18
23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
async remote service interface found with
name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
Severe unrecoverable error, exiting
java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e
(No such file or directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
        at
org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
        at
org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
        at
org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:37 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
component.id=40677,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
}]]

Re: Zookeeper stops

Posted by Wim Jongman <wi...@gmail.com>.
Ah, thanks guys! I did not realize that this was a user setting.

Will try.

Best regards,

Wim

On Thu, Aug 19, 2010 at 4:43 PM, Ted Dunning <te...@gmail.com> wrote:

> Also, /tmp is not a great place to keep things that are intended for
> persistence.
>
> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar <mahadev@yahoo-inc.com
> >wrote:
>
> > Hi Wim,
> >  It mostly looks like that zookeeper is not able to create files on the
> > /tmp filesystem. Is there is a space shortage or is it possible the file
> is
> > being deleted as its being written to?
> >
> > Sometimes admins have a crontab on /tmp that cleans up the /tmp
> filesystem.
> >
> > Thanks
> > mahadev
> >
> >
> > On 8/19/10 1:15 AM, "Wim Jongman" <wi...@gmail.com> wrote:
> >
> > Hi,
> >
> > I have a zookeeper server running that can sometimes run for days and
> then
> > quits:
> >
> > Is there somebody with a clue to the problem?
> >
> > I am running 64 bit Ubuntu with
> >
> > java version "1.6.0_18"
> > OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
> > OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
> >
> > Zookeeper 3.3.0
> >
> > The log below has some context before it shows the fatal error. Our
> > component.id=40676 indicates that it is the 40676th time that I ask ZK
> to
> > publish this information. It has been seen to go up to half a million
> > before
> > stopping.
> >
> > Regards,
> >
> > Wim
> >
> > ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:28 PM.
> > ServiceInfo[uri=osgiservices://
> >
> >
> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> > ,
> >
> >
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> > ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> > =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
> > component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
> > component.id=40676,
> >
> >
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
> > }]]
> > ZooDiscovery> Service Published: Aug 18, 2010 11:17:29 PM.
> > ServiceInfo[uri=osgiservices://
> >
> >
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> > ,
> >
> >
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> > ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> > =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
> > component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> > component.id=40677,
> >
> >
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> > }]]
> > [log;+0200 2010.08.18
> >
> >
> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
> > async remote service interface found with
> > name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
> >
> >
> class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
> > 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
> > Severe unrecoverable error, exiting
> > java.io.FileNotFoundException:
> /tmp/zookeeperData/version-2/snapshot.13e2e
> > (No such file or directory)
> >        at java.io.FileOutputStream.open(Native Method)
> >        at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
> >        at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
> >        at
> >
> >
> org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
> >        at
> >
> >
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
> >        at
> >
> >
> org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
> >        at
> >
> >
> org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
> > ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:37 PM.
> > ServiceInfo[uri=osgiservices://
> >
> >
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> > ,
> >
> >
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> > ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> > =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
> > component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> > component.id=40677,
> >
> >
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> > }]]
> >
> >
>

Re: Zookeeper stops

Posted by Patrick Hunt <ph...@apache.org>.
Btw, a zk server should never just "stop", ZK is "fail fast" so you really
should have a supervisor to restart it if it does exit, more here:

http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_supervision

Patrick

On Thu, Aug 26, 2010 at 1:03 PM, Mahadev Konar <ma...@yahoo-inc.com>wrote:

> HI Ted,
>  You can take a look at
> http://hadoop.apache.org/zookeeper/docs/r3.3.1/zookeeperAdmin.html
>
> To see how to set up directory outside of /tmp.
> I am not sure if you zookeeper instance is part of hbase installation or
> not. In that case you would be better off posting this question on hbase
> list.
>
> Thanks
> mahadev
>
>
>
>
> On 8/26/10 9:05 AM, "Ted Yu" <yu...@gmail.com> wrote:
>
> I saw the same error in hbase-hadoop-zookeeper-X.log
> zookeeper-3.2.2 is used and managed by HBase.
>
> How do I use a directory outside of /tmp for zookeeper persistence ?
>
> Thanks
>
> On Thu, Aug 19, 2010 at 1:42 PM, Patrick Hunt <ph...@apache.org> wrote:
>
> > No. You configure it in the server configuration file.
> >
> > Patrick
> >
> >
> > On 08/19/2010 01:19 PM, Wim Jongman wrote:
> >
> >> Hi,
> >>
> >> But zk does default to /tmp?
> >>
> >> Regards,
> >>
> >> Wim
> >>
> >>
> >>
> >>
> >>
> >> On Thursday, August 19, 2010, Patrick Hunt<ph...@apache.org>  wrote:
> >>
> >>> +1 on that Ted. I frequently see this issue crop up as "I just rebooted
> >>> my server and lost all my data ..." -- many os's will cleanup tmp on
> reboot.
> >>> :-)
> >>>
> >>> Patrick
> >>>
> >>> On 08/19/2010 07:43 AM, Ted Dunning wrote:
> >>>
> >>> Also, /tmp is not a great place to keep things that are intended for
> >>> persistence.
> >>>
> >>> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<mahadev@yahoo-inc.com
> >>> >wrote:
> >>>
> >>>
> >>> Hi Wim,
> >>>   It mostly looks like that zookeeper is not able to create files on
> the
> >>> /tmp filesystem. Is there is a space shortage or is it possible the
> file
> >>> is
> >>> being deleted as its being written to?
> >>>
> >>> Sometimes admins have a crontab on /tmp that cleans up the /tmp
> >>> filesystem.
> >>>
> >>> Thanks
> >>> mahadev
> >>>
> >>>
> >>> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>    wrote:
> >>>
> >>> Hi,
> >>>
> >>> I have a zookeeper server running that can sometimes run for days and
> >>> then
> >>> quits:
> >>>
> >>> Is there somebody with a clue to the problem?
> >>>
> >>> I am running 64 bit Ubuntu with
> >>>
> >>> java version "1.6.0_18"
> >>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
> >>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
> >>>
> >>> Zookeeper 3.3.0
> >>>
> >>> The log below has some context before it shows the fatal error. Our
> >>> component.id=40676 indicates that it is the 40676th time that I ask ZK
> >>> to
> >>> publish this information. It has been seen to go up to half a million
> >>> before
> >>> stopping.
> >>>
> >>> Regards,
> >>>
> >>> Wim
> >>>
> >>> ZooDiscovery>    Service Unpublished: Aug 18, 2010 11:17:28 PM.
> >>> ServiceInfo[uri=osgiservices://
> >>>
> >>>
> >>>
> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> <
> http://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice
> >
> >>> ,
> >>>
> >>>
> >>>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> >>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> >>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081
> ,
> >>> component.name=Star Wars Quotes Service,
> ecf.sp.ect=ecf.generic.server,
> >>> component.id=40676,
> >>>
> >>>
> >>>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
> >>> }]]
> >>> ZooDiscovery>    Service Published: Aug 18, 2010 11:17:29 PM.
> >>> ServiceInfo[uri=osgiservices://
> >>>
> >>>
> >>>
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> <
> http://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice
> >
> >>> ,
> >>>
> >>>
> >>>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> >>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> >>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4
> ,
> >>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> >>> component.id=40677,
> >>>
> >>>
> >>>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> >>> }]]
> >>> [log;+0200 2010.08.18
> >>>
> >>>
> >>>
> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo
> >>>
> >>
>
>

Re: Zookeeper stops

Posted by Mahadev Konar <ma...@yahoo-inc.com>.
HI Ted,
 You can take a look at
http://hadoop.apache.org/zookeeper/docs/r3.3.1/zookeeperAdmin.html

To see how to set up directory outside of /tmp.
I am not sure if you zookeeper instance is part of hbase installation or not. In that case you would be better off posting this question on hbase list.

Thanks
mahadev




On 8/26/10 9:05 AM, "Ted Yu" <yu...@gmail.com> wrote:

I saw the same error in hbase-hadoop-zookeeper-X.log
zookeeper-3.2.2 is used and managed by HBase.

How do I use a directory outside of /tmp for zookeeper persistence ?

Thanks

On Thu, Aug 19, 2010 at 1:42 PM, Patrick Hunt <ph...@apache.org> wrote:

> No. You configure it in the server configuration file.
>
> Patrick
>
>
> On 08/19/2010 01:19 PM, Wim Jongman wrote:
>
>> Hi,
>>
>> But zk does default to /tmp?
>>
>> Regards,
>>
>> Wim
>>
>>
>>
>>
>>
>> On Thursday, August 19, 2010, Patrick Hunt<ph...@apache.org>  wrote:
>>
>>> +1 on that Ted. I frequently see this issue crop up as "I just rebooted
>>> my server and lost all my data ..." -- many os's will cleanup tmp on reboot.
>>> :-)
>>>
>>> Patrick
>>>
>>> On 08/19/2010 07:43 AM, Ted Dunning wrote:
>>>
>>> Also, /tmp is not a great place to keep things that are intended for
>>> persistence.
>>>
>>> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<mahadev@yahoo-inc.com
>>> >wrote:
>>>
>>>
>>> Hi Wim,
>>>   It mostly looks like that zookeeper is not able to create files on the
>>> /tmp filesystem. Is there is a space shortage or is it possible the file
>>> is
>>> being deleted as its being written to?
>>>
>>> Sometimes admins have a crontab on /tmp that cleans up the /tmp
>>> filesystem.
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>    wrote:
>>>
>>> Hi,
>>>
>>> I have a zookeeper server running that can sometimes run for days and
>>> then
>>> quits:
>>>
>>> Is there somebody with a clue to the problem?
>>>
>>> I am running 64 bit Ubuntu with
>>>
>>> java version "1.6.0_18"
>>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
>>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>>>
>>> Zookeeper 3.3.0
>>>
>>> The log below has some context before it shows the fatal error. Our
>>> component.id=40676 indicates that it is the 40676th time that I ask ZK
>>> to
>>> publish this information. It has been seen to go up to half a million
>>> before
>>> stopping.
>>>
>>> Regards,
>>>
>>> Wim
>>>
>>> ZooDiscovery>    Service Unpublished: Aug 18, 2010 11:17:28 PM.
>>> ServiceInfo[uri=osgiservices://
>>>
>>>
>>> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice<http://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice>
>>> ,
>>>
>>>
>>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
>>> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
>>> component.id=40676,
>>>
>>>
>>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
>>> }]]
>>> ZooDiscovery>    Service Published: Aug 18, 2010 11:17:29 PM.
>>> ServiceInfo[uri=osgiservices://
>>>
>>>
>>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice<http://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice>
>>> ,
>>>
>>>
>>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>>> component.id=40677,
>>>
>>>
>>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>>> }]]
>>> [log;+0200 2010.08.18
>>>
>>>
>>> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo
>>>
>>

Re: Zookeeper stops

Posted by Ted Yu <yu...@gmail.com>.
I saw the same error in hbase-hadoop-zookeeper-X.log
zookeeper-3.2.2 is used and managed by HBase.

How do I use a directory outside of /tmp for zookeeper persistence ?

Thanks

On Thu, Aug 19, 2010 at 1:42 PM, Patrick Hunt <ph...@apache.org> wrote:

> No. You configure it in the server configuration file.
>
> Patrick
>
>
> On 08/19/2010 01:19 PM, Wim Jongman wrote:
>
>> Hi,
>>
>> But zk does default to /tmp?
>>
>> Regards,
>>
>> Wim
>>
>>
>>
>>
>>
>> On Thursday, August 19, 2010, Patrick Hunt<ph...@apache.org>  wrote:
>>
>>> +1 on that Ted. I frequently see this issue crop up as "I just rebooted
>>> my server and lost all my data ..." -- many os's will cleanup tmp on reboot.
>>> :-)
>>>
>>> Patrick
>>>
>>> On 08/19/2010 07:43 AM, Ted Dunning wrote:
>>>
>>> Also, /tmp is not a great place to keep things that are intended for
>>> persistence.
>>>
>>> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<mahadev@yahoo-inc.com
>>> >wrote:
>>>
>>>
>>> Hi Wim,
>>>   It mostly looks like that zookeeper is not able to create files on the
>>> /tmp filesystem. Is there is a space shortage or is it possible the file
>>> is
>>> being deleted as its being written to?
>>>
>>> Sometimes admins have a crontab on /tmp that cleans up the /tmp
>>> filesystem.
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>    wrote:
>>>
>>> Hi,
>>>
>>> I have a zookeeper server running that can sometimes run for days and
>>> then
>>> quits:
>>>
>>> Is there somebody with a clue to the problem?
>>>
>>> I am running 64 bit Ubuntu with
>>>
>>> java version "1.6.0_18"
>>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
>>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>>>
>>> Zookeeper 3.3.0
>>>
>>> The log below has some context before it shows the fatal error. Our
>>> component.id=40676 indicates that it is the 40676th time that I ask ZK
>>> to
>>> publish this information. It has been seen to go up to half a million
>>> before
>>> stopping.
>>>
>>> Regards,
>>>
>>> Wim
>>>
>>> ZooDiscovery>    Service Unpublished: Aug 18, 2010 11:17:28 PM.
>>> ServiceInfo[uri=osgiservices://
>>>
>>>
>>> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice<http://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice>
>>> ,
>>>
>>>
>>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
>>> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
>>> component.id=40676,
>>>
>>>
>>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
>>> }]]
>>> ZooDiscovery>    Service Published: Aug 18, 2010 11:17:29 PM.
>>> ServiceInfo[uri=osgiservices://
>>>
>>>
>>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice<http://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID%5Btype%3DServiceTypeID%5BtypeName%3D_osgiservices._tcp.default._iana%5D%3Blocation%3Dosgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=%5D;priority=0;weight=0;props=ServiceProperties%5B%7Becf.rsvc.ns=ecf.namespace.generic.remoteservice>
>>> ,
>>>
>>>
>>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>>> component.id=40677,
>>>
>>>
>>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>>> }]]
>>> [log;+0200 2010.08.18
>>>
>>>
>>> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo
>>>
>>

Re: Zookeeper stops

Posted by Patrick Hunt <ph...@apache.org>.
No. You configure it in the server configuration file.

Patrick

On 08/19/2010 01:19 PM, Wim Jongman wrote:
> Hi,
>
> But zk does default to /tmp?
>
> Regards,
>
> Wim
>
>
>
>
>
> On Thursday, August 19, 2010, Patrick Hunt<ph...@apache.org>  wrote:
>> +1 on that Ted. I frequently see this issue crop up as "I just rebooted my server and lost all my data ..." -- many os's will cleanup tmp on reboot. :-)
>>
>> Patrick
>>
>> On 08/19/2010 07:43 AM, Ted Dunning wrote:
>>
>> Also, /tmp is not a great place to keep things that are intended for
>> persistence.
>>
>> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<ma...@yahoo-inc.com>wrote:
>>
>>
>> Hi Wim,
>>    It mostly looks like that zookeeper is not able to create files on the
>> /tmp filesystem. Is there is a space shortage or is it possible the file is
>> being deleted as its being written to?
>>
>> Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.
>>
>> Thanks
>> mahadev
>>
>>
>> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>    wrote:
>>
>> Hi,
>>
>> I have a zookeeper server running that can sometimes run for days and then
>> quits:
>>
>> Is there somebody with a clue to the problem?
>>
>> I am running 64 bit Ubuntu with
>>
>> java version "1.6.0_18"
>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>>
>> Zookeeper 3.3.0
>>
>> The log below has some context before it shows the fatal error. Our
>> component.id=40676 indicates that it is the 40676th time that I ask ZK to
>> publish this information. It has been seen to go up to half a million
>> before
>> stopping.
>>
>> Regards,
>>
>> Wim
>>
>> ZooDiscovery>    Service Unpublished: Aug 18, 2010 11:17:28 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
>> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
>> component.id=40676,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
>> }]]
>> ZooDiscovery>    Service Published: Aug 18, 2010 11:17:29 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>> component.id=40677,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>> }]]
>> [log;+0200 2010.08.18
>>
>> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo

Re: Zookeeper stops

Posted by Wim Jongman <wi...@gmail.com>.
Hi,

But zk does default to /tmp?

Regards,

Wim





On Thursday, August 19, 2010, Patrick Hunt <ph...@apache.org> wrote:
> +1 on that Ted. I frequently see this issue crop up as "I just rebooted my server and lost all my data ..." -- many os's will cleanup tmp on reboot. :-)
>
> Patrick
>
> On 08/19/2010 07:43 AM, Ted Dunning wrote:
>
> Also, /tmp is not a great place to keep things that are intended for
> persistence.
>
> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<ma...@yahoo-inc.com>wrote:
>
>
> Hi Wim,
>   It mostly looks like that zookeeper is not able to create files on the
> /tmp filesystem. Is there is a space shortage or is it possible the file is
> being deleted as its being written to?
>
> Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.
>
> Thanks
> mahadev
>
>
> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>  wrote:
>
> Hi,
>
> I have a zookeeper server running that can sometimes run for days and then
> quits:
>
> Is there somebody with a clue to the problem?
>
> I am running 64 bit Ubuntu with
>
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>
> Zookeeper 3.3.0
>
> The log below has some context before it shows the fatal error. Our
> component.id=40676 indicates that it is the 40676th time that I ask ZK to
> publish this information. It has been seen to go up to half a million
> before
> stopping.
>
> Regards,
>
> Wim
>
> ZooDiscovery>  Service Unpublished: Aug 18, 2010 11:17:28 PM.
> ServiceInfo[uri=osgiservices://
>
> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> ,
>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
> component.id=40676,
>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
> }]]
> ZooDiscovery>  Service Published: Aug 18, 2010 11:17:29 PM.
> ServiceInfo[uri=osgiservices://
>
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> ,
>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> component.id=40677,
>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> }]]
> [log;+0200 2010.08.18
>
> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo

Re: Zookeeper stops

Posted by Patrick Hunt <ph...@apache.org>.
+1 on that Ted. I frequently see this issue crop up as "I just rebooted 
my server and lost all my data ..." -- many os's will cleanup tmp on 
reboot. :-)

Patrick

On 08/19/2010 07:43 AM, Ted Dunning wrote:
> Also, /tmp is not a great place to keep things that are intended for
> persistence.
>
> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<ma...@yahoo-inc.com>wrote:
>
>> Hi Wim,
>>   It mostly looks like that zookeeper is not able to create files on the
>> /tmp filesystem. Is there is a space shortage or is it possible the file is
>> being deleted as its being written to?
>>
>> Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.
>>
>> Thanks
>> mahadev
>>
>>
>> On 8/19/10 1:15 AM, "Wim Jongman"<wi...@gmail.com>  wrote:
>>
>> Hi,
>>
>> I have a zookeeper server running that can sometimes run for days and then
>> quits:
>>
>> Is there somebody with a clue to the problem?
>>
>> I am running 64 bit Ubuntu with
>>
>> java version "1.6.0_18"
>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>>
>> Zookeeper 3.3.0
>>
>> The log below has some context before it shows the fatal error. Our
>> component.id=40676 indicates that it is the 40676th time that I ask ZK to
>> publish this information. It has been seen to go up to half a million
>> before
>> stopping.
>>
>> Regards,
>>
>> Wim
>>
>> ZooDiscovery>  Service Unpublished: Aug 18, 2010 11:17:28 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
>> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
>> component.id=40676,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
>> }]]
>> ZooDiscovery>  Service Published: Aug 18, 2010 11:17:29 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>> component.id=40677,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>> }]]
>> [log;+0200 2010.08.18
>>
>> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
>> async remote service interface found with
>> name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
>>
>> class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
>> 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
>> Severe unrecoverable error, exiting
>> java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e
>> (No such file or directory)
>>         at java.io.FileOutputStream.open(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
>>         at
>>
>> org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
>>         at
>>
>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
>>         at
>>
>> org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
>>         at
>>
>> org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
>> ZooDiscovery>  Service Unpublished: Aug 18, 2010 11:17:37 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>> component.id=40677,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>> }]]
>>
>>
>

Re: Zookeeper stops

Posted by Ted Dunning <te...@gmail.com>.
Also, /tmp is not a great place to keep things that are intended for
persistence.

On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar <ma...@yahoo-inc.com>wrote:

> Hi Wim,
>  It mostly looks like that zookeeper is not able to create files on the
> /tmp filesystem. Is there is a space shortage or is it possible the file is
> being deleted as its being written to?
>
> Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.
>
> Thanks
> mahadev
>
>
> On 8/19/10 1:15 AM, "Wim Jongman" <wi...@gmail.com> wrote:
>
> Hi,
>
> I have a zookeeper server running that can sometimes run for days and then
> quits:
>
> Is there somebody with a clue to the problem?
>
> I am running 64 bit Ubuntu with
>
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>
> Zookeeper 3.3.0
>
> The log below has some context before it shows the fatal error. Our
> component.id=40676 indicates that it is the 40676th time that I ask ZK to
> publish this information. It has been seen to go up to half a million
> before
> stopping.
>
> Regards,
>
> Wim
>
> ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:28 PM.
> ServiceInfo[uri=osgiservices://
>
> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> ,
>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
> component.id=40676,
>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
> }]]
> ZooDiscovery> Service Published: Aug 18, 2010 11:17:29 PM.
> ServiceInfo[uri=osgiservices://
>
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> ,
>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> component.id=40677,
>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> }]]
> [log;+0200 2010.08.18
>
> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
> async remote service interface found with
> name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
>
> class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
> 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
> Severe unrecoverable error, exiting
> java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e
> (No such file or directory)
>        at java.io.FileOutputStream.open(Native Method)
>        at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
>        at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
>        at
>
> org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
>        at
>
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
>        at
>
> org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
>        at
>
> org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
> ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:37 PM.
> ServiceInfo[uri=osgiservices://
>
> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
> ,
>
> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
> component.id=40677,
>
> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
> }]]
>
>

Re: Zookeeper stops

Posted by Mahadev Konar <ma...@yahoo-inc.com>.
Hi Wim,
  It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to?

Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.

Thanks
mahadev


On 8/19/10 1:15 AM, "Wim Jongman" <wi...@gmail.com> wrote:

Hi,

I have a zookeeper server running that can sometimes run for days and then
quits:

Is there somebody with a clue to the problem?

I am running 64 bit Ubuntu with

java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Zookeeper 3.3.0

The log below has some context before it shows the fatal error. Our
component.id=40676 indicates that it is the 40676th time that I ask ZK to
publish this information. It has been seen to go up to half a million before
stopping.

Regards,

Wim

ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:28 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
component.id=40676,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
}]]
ZooDiscovery> Service Published: Aug 18, 2010 11:17:29 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
component.id=40677,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
}]]
[log;+0200 2010.08.18
23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
async remote service interface found with
name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
Severe unrecoverable error, exiting
java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e
(No such file or directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
        at
org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
        at
org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
        at
org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
ZooDiscovery> Service Unpublished: Aug 18, 2010 11:17:37 PM.
ServiceInfo[uri=osgiservices://
188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice,
osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
component.id=40677,
ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
}]]