You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jonathan Baynes <Jo...@tradeweb.com> on 2017/09/21 08:56:46 UTC
Understanding Messages in the Debug.log
Good Morning Community,
Can anyone help me understand what these messages (and I have a lot of them in the log) mean, they must be a symptom of something but I am unsure how to starting working this out.
I'm using C* 3.0.11, CQL 5.0.1.
I have 6 node ring over 2 DC's (3 in each DC) each of the nodes has this message in the log.
The message in the debug log is
DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 - Ignoring interval time of 2000275419
There are so many of these messages, I'd like to find the root cause of this, if anyone could help that would be great. Or if they can be ignored, how do I supress these?
Thanks
J
Jonathan Baynes
DBA
Tradeweb Europe Limited
Moor Place * 1 Fore Street Avenue * London EC2Y 9DT
P +44 (0)20 77760988 * F +44 (0)20 7776 3201 * M +44 (0)xxxx xxxxxx
Jonathan.Baynes@tradeweb.com
[cid:image001.jpg@01CD26AD.4165F110]<http://www.tradeweb.com/> follow us: [cid:image002.jpg@01CD26AD.4165F110] <https://www.linkedin.com/company/tradeweb?trk=top_nav_home> [cid:image003.jpg@01CD26AD.4165F110] <http://www.twitter.com/Tradeweb>
-----------------------------------------
A leading marketplace<http://www.tradeweb.com/About-Us/Awards/> for electronic fixed income, derivatives and ETF trading
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
Re: Understanding Messages in the Debug.log
Posted by Alex Kotelnikov <al...@diginetica.com>.
I see a lot of that on one of our clusters.
There is pretty much no load on it. Clocks are quite well synchronized.
date +start:%H:%M:%S.%N ; for d in 2 3 4 5 6 8 9; do ssh 10.3.0.17$d date
+%H:%M:%S.%N & done;date +end:%H:%M:%S.%N
start:01:22:45.478726356
end:01:22:45.480993049
01:22:45.570313246
01:22:45.572781468
01:22:45.584917988
01:22:45.595550249
01:22:45.595120011
01:22:45.598196670
01:22:45.863486947
That is 3.0.9. I see the log message ~2000 times an hour
On another cluster, 3.11 I see it happening ~5 times an hour.
On 22 September 2017 at 01:23, Nate McCall <na...@thelastpickle.com> wrote:
>
>> The message in the debug log is
>>
>> DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 -
>> Ignoring interval time of 2000275419
>>
>>
>>
> Did you truncate the log message? There should be and "for [endpoint]" on
> the end which should help you narrow things down to a set of problem nodes.
> I agree with Jeff in that this is most likely NTP sync issue or network
> flap, though.
>
--
Best Regards,
*Alexander Kotelnikov*
*Team Lead*
DIGINETICA
Retail Technology Company
m: +7.921.915.06.28
*www.diginetica.com <http://www.diginetica.com/>*
Re: Understanding Messages in the Debug.log
Posted by Nate McCall <na...@thelastpickle.com>.
>
>
> The message in the debug log is
>
> DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 -
> Ignoring interval time of 2000275419
>
>
>
Did you truncate the log message? There should be and "for [endpoint]" on
the end which should help you narrow things down to a set of problem nodes.
I agree with Jeff in that this is most likely NTP sync issue or network
flap, though.
Re: Understanding Messages in the Debug.log
Posted by Milan Milosevic <mi...@gmail.com>.
The message is caused by the Gossip message exchanged. It doesn't have
anything to do with the load on the cluster. I asked the same question on
the datastax slack. This is the answer from one of the Datastax engineers
(Erick Ramirez):
"`FailureDetector` checks if a node hasn't responded for 5 seconds before
marking it as down, otherwise it doesn't do anything (ignores) if last
response was within 5 seconds (`2002779703` 2 secs in your case)
what i'm trying to say is that you don't have to worry about anything"
On Thu, Sep 21, 2017 at 1:37 PM, Jonathan Baynes <
Jonathan.Baynes@tradeweb.com> wrote:
> Thanks Jacob,
>
>
>
> I’ve already seen this, but it doesn’t answer my question, as this is a
> production Cluster, which at present, has no data and is not being written
> too, its being built to go live shortly, so with no data, a Keyspace with a
> table designed but no data in it, only system databases, what on earth
> could be causing load on the cluster to cause the error message??
>
>
>
>
>
>
>
> *From:* Jacob Shadix [mailto:jacobshadix@gmail.com]
> *Sent:* 21 September 2017 12:34
> *To:* user@cassandra.apache.org
> *Subject:* Re: Understanding Messages in the Debug.log
>
>
>
> check out this thread - https://www.mail-archive.
> com/user@cassandra.apache.org/msg52313.html
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mail-2Darchive.com_user-40cassandra.apache.org_msg52313.html&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=T5vuzfn8ElJgnkkZ4sQTkQUMwN-_GEOacqRbeVayQUQ&e=>
>
>
> -- Jacob Shadix
>
>
>
> On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <
> Jonathan.Baynes@tradeweb.com> wrote:
>
> Good Morning Community,
>
>
>
> Can anyone help me understand what these messages (and I have a lot of
> them in the log) mean, they must be a symptom of something but I am unsure
> how to starting working this out.
>
>
>
> I’m using C* 3.0.11, CQL 5.0.1.
>
>
>
> I have 6 node ring over 2 DC’s (3 in each DC) each of the nodes has this
> message in the log.
>
>
>
> The message in the debug log is
>
> DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 -
> Ignoring interval time of 2000275419
>
>
>
>
>
> There are so many of these messages, I’d like to find the root cause of
> this, if anyone could help that would be great. Or if they can be ignored,
> how do I supress these?
>
>
>
> Thanks
>
> J
>
>
>
> *Jonathan Baynes*
>
> DBA
> Tradeweb Europe Limited
>
> Moor Place • 1 Fore Street Avenue
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=>
> •
> <https://maps.google.com/?q=1%0D+Fore+Street+Avenue%C2%A0%C2%A0%E2%80%A2%C2%A0%C2%A0London%0D+EC2Y+9DT+%3Chttps://urldefense.proofpoint.com/v2/url?u%3Dhttps-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg%26d%3DDwMFaQ%26c%3DsA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk%26r%3DCNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8%26m%3D0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo%26s%3D8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o%26e%3D%3E&entry=gmail&source=g>
> London EC2Y 9DT
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=>
> P +44 (0)20 77760988 <+44%2020%207776%200988> • F +44 (0)20 7776 3201
> <+44%2020%207776%203201> • M +44 (0)xxxx xxxxxx
>
> Jonathan.Baynes@tradeweb.com
>
>
>
> [image: cid:image001.jpg@01CD26AD.4165F110] <http://www.tradeweb.com/>
> follow us: *[image: cid:image002.jpg@01CD26AD.4165F110]*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_company_tradeweb-3Ftrk-3Dtop-5Fnav-5Fhome&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=qEj7m_M2g7N23-PtTGZmpbyO13Zqu-0qypSZg4i3qlA&e=>
> [image: cid:image003.jpg@01CD26AD.4165F110]
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.twitter.com_Tradeweb&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=B5FSJ8OU-4_K4gemgZ7FC02TH3tkIGnZ6oSynGFEyNs&e=>
>
> —————————————————————————————————————————
>
> A leading marketplace <http://www.tradeweb.com/About-Us/Awards/> for
> electronic fixed income, derivatives and ETF trading
>
>
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contactus@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3Dnumber-2B3912826-29-2C-2B1-2BFore-2BStreet-2BAvenue-2BLondon-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=713g1JML8nBUcMKnA8iqjQ1xaCSJ999h4dJQJenCsB4&e=>.
> To see our privacy policy, visit our website @ www.tradeweb.com.
>
>
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contactus@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT
> <https://maps.google.com/?q=number+3912826),+1+Fore+Street+Avenue+London+EC2Y+9DT&entry=gmail&source=g>.
> To see our privacy policy, visit our website @ www.tradeweb.com.
>
RE: Understanding Messages in the Debug.log
Posted by Jonathan Baynes <Jo...@tradeweb.com>.
Thanks Jeff,
They are on all instances of the cluster, with the same timeframe (give or take).
Ill look into your suggestions., thank you for your guidance.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: 21 September 2017 12:48
To: user@cassandra.apache.org
Subject: Re: Understanding Messages in the Debug.log
Could also be a network hiccup, an NTP problem, a disk pause, or anything that causes a long gc pause including misconfigured heap. Are they all on the same machine? Or all instances in the cluster?
If there are so many you need to root cause it, you probably want to look for correlation with other entries in the debug log, or start inspecting the heap/thread pools with some combination of tpstats and dumping the heap and looking at the running threads may give you an indication of what’s going on.
--
Jeff Jirsa
On Sep 21, 2017, at 4:37 AM, Jonathan Baynes <Jo...@tradeweb.com>> wrote:
Thanks Jacob,
I’ve already seen this, but it doesn’t answer my question, as this is a production Cluster, which at present, has no data and is not being written too, its being built to go live shortly, so with no data, a Keyspace with a table designed but no data in it, only system databases, what on earth could be causing load on the cluster to cause the error message??
From: Jacob Shadix [mailto:jacobshadix@gmail.com]
Sent: 21 September 2017 12:34
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Understanding Messages in the Debug.log
check out this thread - https://www.mail-archive.com/user@cassandra.apache.org/msg52313.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mail-2Darchive.com_user-40cassandra.apache.org_msg52313.html&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=T5vuzfn8ElJgnkkZ4sQTkQUMwN-_GEOacqRbeVayQUQ&e=>
-- Jacob Shadix
On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <Jo...@tradeweb.com>> wrote:
Good Morning Community,
Can anyone help me understand what these messages (and I have a lot of them in the log) mean, they must be a symptom of something but I am unsure how to starting working this out.
I’m using C* 3.0.11, CQL 5.0.1.
I have 6 node ring over 2 DC’s (3 in each DC) each of the nodes has this message in the log.
The message in the debug log is
DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 - Ignoring interval time of 2000275419
There are so many of these messages, I’d like to find the root cause of this, if anyone could help that would be great. Or if they can be ignored, how do I supress these?
Thanks
J
Jonathan Baynes
DBA
Tradeweb Europe Limited
Moor Place • 1 Fore Street Avenue<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=> • London EC2Y 9DT<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=>
P +44 (0)20 77760988 • F +44 (0)20 7776 3201 • M +44 (0)xxxx xxxxxx
Jonathan.Baynes@tradeweb.com<ma...@tradeweb.com>
<image001.jpg><http://www.tradeweb.com/> follow us: <image002.jpg><https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_company_tradeweb-3Ftrk-3Dtop-5Fnav-5Fhome&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=qEj7m_M2g7N23-PtTGZmpbyO13Zqu-0qypSZg4i3qlA&e=> <image003.jpg><https://urldefense.proofpoint.com/v2/url?u=http-3A__www.twitter.com_Tradeweb&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=B5FSJ8OU-4_K4gemgZ7FC02TH3tkIGnZ6oSynGFEyNs&e=>
—————————————————————————————————————————
A leading marketplace<http://www.tradeweb.com/About-Us/Awards/> for electronic fixed income, derivatives and ETF trading
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com<ma...@tradeweb.com> or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3Dnumber-2B3912826-29-2C-2B1-2BFore-2BStreet-2BAvenue-2BLondon-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=713g1JML8nBUcMKnA8iqjQ1xaCSJ999h4dJQJenCsB4&e=>. To see our privacy policy, visit our website @ www.tradeweb.com<http://www.tradeweb.com>.
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com<ma...@tradeweb.com> or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com<http://www.tradeweb.com>.
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
Re: Understanding Messages in the Debug.log
Posted by Jeff Jirsa <jj...@gmail.com>.
Could also be a network hiccup, an NTP problem, a disk pause, or anything that causes a long gc pause including misconfigured heap. Are they all on the same machine? Or all instances in the cluster?
If there are so many you need to root cause it, you probably want to look for correlation with other entries in the debug log, or start inspecting the heap/thread pools with some combination of tpstats and dumping the heap and looking at the running threads may give you an indication of what’s going on.
--
Jeff Jirsa
> On Sep 21, 2017, at 4:37 AM, Jonathan Baynes <Jo...@tradeweb.com> wrote:
>
> Thanks Jacob,
>
> I’ve already seen this, but it doesn’t answer my question, as this is a production Cluster, which at present, has no data and is not being written too, its being built to go live shortly, so with no data, a Keyspace with a table designed but no data in it, only system databases, what on earth could be causing load on the cluster to cause the error message??
>
>
>
> From: Jacob Shadix [mailto:jacobshadix@gmail.com]
> Sent: 21 September 2017 12:34
> To: user@cassandra.apache.org
> Subject: Re: Understanding Messages in the Debug.log
>
> check out this thread - https://www.mail-archive.com/user@cassandra.apache.org/msg52313.html
>
> -- Jacob Shadix
>
> On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <Jo...@tradeweb.com> wrote:
> Good Morning Community,
>
> Can anyone help me understand what these messages (and I have a lot of them in the log) mean, they must be a symptom of something but I am unsure how to starting working this out.
>
> I’m using C* 3.0.11, CQL 5.0.1.
>
> I have 6 node ring over 2 DC’s (3 in each DC) each of the nodes has this message in the log.
>
> The message in the debug log is
> DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 - Ignoring interval time of 2000275419
>
>
> There are so many of these messages, I’d like to find the root cause of this, if anyone could help that would be great. Or if they can be ignored, how do I supress these?
>
> Thanks
> J
>
> Jonathan Baynes
> DBA
> Tradeweb Europe Limited
> Moor Place • 1 Fore Street Avenue • London EC2Y 9DT
> P +44 (0)20 77760988 • F +44 (0)20 7776 3201 • M +44 (0)xxxx xxxxxx
> Jonathan.Baynes@tradeweb.com
>
> <image001.jpg> follow us: <image002.jpg> <image003.jpg>
> —————————————————————————————————————————
> A leading marketplace for electronic fixed income, derivatives and ETF trading
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>
RE: Understanding Messages in the Debug.log
Posted by Jonathan Baynes <Jo...@tradeweb.com>.
Thanks Jacob,
I’ve already seen this, but it doesn’t answer my question, as this is a production Cluster, which at present, has no data and is not being written too, its being built to go live shortly, so with no data, a Keyspace with a table designed but no data in it, only system databases, what on earth could be causing load on the cluster to cause the error message??
From: Jacob Shadix [mailto:jacobshadix@gmail.com]
Sent: 21 September 2017 12:34
To: user@cassandra.apache.org
Subject: Re: Understanding Messages in the Debug.log
check out this thread - https://www.mail-archive.com/user@cassandra.apache.org/msg52313.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mail-2Darchive.com_user-40cassandra.apache.org_msg52313.html&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=T5vuzfn8ElJgnkkZ4sQTkQUMwN-_GEOacqRbeVayQUQ&e=>
-- Jacob Shadix
On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <Jo...@tradeweb.com>> wrote:
Good Morning Community,
Can anyone help me understand what these messages (and I have a lot of them in the log) mean, they must be a symptom of something but I am unsure how to starting working this out.
I’m using C* 3.0.11, CQL 5.0.1.
I have 6 node ring over 2 DC’s (3 in each DC) each of the nodes has this message in the log.
The message in the debug log is
DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 - Ignoring interval time of 2000275419
There are so many of these messages, I’d like to find the root cause of this, if anyone could help that would be great. Or if they can be ignored, how do I supress these?
Thanks
J
Jonathan Baynes
DBA
Tradeweb Europe Limited
Moor Place • 1 Fore Street Avenue<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=> • London EC2Y 9DT<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D1-2BFore-2BStreet-2BAvenue-25C2-25A0-25C2-25A0-25E2-2580-25A2-25C2-25A0-25C2-25A0London-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=8MElkv-02VTEMtwP2-AvmBTjsk3O-VMBNd3JtOBzp1o&e=>
P +44 (0)20 77760988 • F +44 (0)20 7776 3201 • M +44 (0)xxxx xxxxxx
Jonathan.Baynes@tradeweb.com<ma...@tradeweb.com>
[cid:image001.jpg@01CD26AD.4165F110]<http://www.tradeweb.com/> follow us: [cid:image002.jpg@01CD26AD.4165F110] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_company_tradeweb-3Ftrk-3Dtop-5Fnav-5Fhome&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=qEj7m_M2g7N23-PtTGZmpbyO13Zqu-0qypSZg4i3qlA&e=> [cid:image003.jpg@01CD26AD.4165F110] <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.twitter.com_Tradeweb&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=B5FSJ8OU-4_K4gemgZ7FC02TH3tkIGnZ6oSynGFEyNs&e=>
—————————————————————————————————————————
A leading marketplace<http://www.tradeweb.com/About-Us/Awards/> for electronic fixed income, derivatives and ETF trading
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com<ma...@tradeweb.com> or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3Dnumber-2B3912826-29-2C-2B1-2BFore-2BStreet-2BAvenue-2BLondon-2BEC2Y-2B9DT-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=sA0VaJZJFLZREu2pbPeqjXHJ-Wd9NNzgHW3gpUOLSSk&r=CNKccIKIKCVbYTu1VxR8dIOP6NLpf4fYjidpNm-diQ8&m=0LpreILsiIHqijRbJ-hSyC7R1Hq3hMhquXS3xcIDLjo&s=713g1JML8nBUcMKnA8iqjQ1xaCSJ999h4dJQJenCsB4&e=>. To see our privacy policy, visit our website @ www.tradeweb.com<http://www.tradeweb.com>.
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
Re: Understanding Messages in the Debug.log
Posted by Jacob Shadix <ja...@gmail.com>.
check out this thread -
https://www.mail-archive.com/user@cassandra.apache.org/msg52313.html
-- Jacob Shadix
On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <
Jonathan.Baynes@tradeweb.com> wrote:
> Good Morning Community,
>
>
>
> Can anyone help me understand what these messages (and I have a lot of
> them in the log) mean, they must be a symptom of something but I am unsure
> how to starting working this out.
>
>
>
> I’m using C* 3.0.11, CQL 5.0.1.
>
>
>
> I have 6 node ring over 2 DC’s (3 in each DC) each of the nodes has this
> message in the log.
>
>
>
> The message in the debug log is
>
> DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 -
> Ignoring interval time of 2000275419
>
>
>
>
>
> There are so many of these messages, I’d like to find the root cause of
> this, if anyone could help that would be great. Or if they can be ignored,
> how do I supress these?
>
>
>
> Thanks
>
> J
>
>
>
> *Jonathan Baynes*
>
> DBA
> Tradeweb Europe Limited
>
> Moor Place • 1 Fore Street Avenue
> <https://maps.google.com/?q=1+Fore+Street+Avenue%C2%A0%C2%A0%E2%80%A2%C2%A0%C2%A0London+EC2Y+9DT&entry=gmail&source=g>
> • London EC2Y 9DT
> <https://maps.google.com/?q=1+Fore+Street+Avenue%C2%A0%C2%A0%E2%80%A2%C2%A0%C2%A0London+EC2Y+9DT&entry=gmail&source=g>
> P +44 (0)20 77760988 • F +44 (0)20 7776 3201 • M +44 (0)xxxx xxxxxx
>
> Jonathan.Baynes@tradeweb.com
>
>
>
> [image: cid:image001.jpg@01CD26AD.4165F110] <http://www.tradeweb.com/>
> follow us: *[image: cid:image002.jpg@01CD26AD.4165F110]*
> <https://www.linkedin.com/company/tradeweb?trk=top_nav_home> [image:
> cid:image003.jpg@01CD26AD.4165F110] <http://www.twitter.com/Tradeweb>
>
> —————————————————————————————————————————
>
> A leading marketplace <http://www.tradeweb.com/About-Us/Awards/> for
> electronic fixed income, derivatives and ETF trading
>
>
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contactus@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT
> <https://maps.google.com/?q=number+3912826),+1+Fore+Street+Avenue+London+EC2Y+9DT&entry=gmail&source=g>.
> To see our privacy policy, visit our website @ www.tradeweb.com.
>