You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Durity, Sean R" <SE...@homedepot.com> on 2018/10/12 13:40:04 UTC
RE: [EXTERNAL] Re: Tracing in cassandra
I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
Sean Durity
From: Abdul Patel <ab...@gmail.com>
Sent: Thursday, October 11, 2018 5:54 PM
To: adam@instaclustr.com
Cc: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Tracing in cassandra
Query :
SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
-Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 9754 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10343 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10480 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 5745 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 587 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7038 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 95 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 12361 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 206 | RequestResponseStage-7
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d32f1-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND token(user_id) >= token(769281382) AND token(user_id) <= token(769281382) LIMIT 12; | 10.54.145.32 | 519 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d5a00-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 906 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8110-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 1820 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8111-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 1884 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8112-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 1968 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8113-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 2023 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8114-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 2076 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8115-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 2133 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8116-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 2179 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8117-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 2290 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8118-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 2371 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8119-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 2494 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da820-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 2560 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da821-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 2618 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da822-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 2689 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da823-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 2750 | ReadStage-3
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df640-cd9e-11e8-8e99-15807bff4dfd | Index mean cardinalities are status_idx:1. Scanning with status_idx. | 10.54.145.32 | 4656 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df641-cd9e-11e8-8e99-15807bff4dfd | Computing ranges to query | 10.54.145.32 | 4844 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df642-cd9e-11e8-8e99-15807bff4dfd | Submitting range requests on 1 ranges with a concurrency of 1 (2.9296876E-4 rows per range expected) | 10.54.145.32 | 5109 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df643-cd9e-11e8-8e99-15807bff4dfd | Submitted 1 concurrent range requests | 10.54.145.32 | 5286 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df644-cd9e-11e8-8e99-15807bff4dfd | Executing read on keysoace.tablenameusing index status_idx | 10.54.145.32 | 5386 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df645-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session.status_idx | 10.54.145.32 | 5476 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df646-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 5499 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df647-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 42279 | 10.54.145.32 | 5539 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d50-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 42267 | 10.54.145.32 | 5574 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d51-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 41890 | 10.54.145.32 | 5601 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d52-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 37159 | 10.54.145.32 | 5704 | ReadStage-6
---MORE---
session_id | event_id | activity | source | source_elapsed | thread
--------------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+--------------+----------------+-----------------------------------------------
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d53-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 5800 | ReadStage-6
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | fc533b00-cd9e-11e8-8e99-15807bff4dfd | Timed out; received 0 of 1 responses | 10.54.145.32 | 10000610 | Native-Transport-Requests-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eac6e0-cd6c-11e8-af38-373d9c5cfc0c | Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND token(user_id) >= token(371384583) AND token(user_id) <= token(371384583) LIMIT 11\n; | 10.153.42.83 | 3819 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eaedf0-cd6c-11e8-af38-373d9c5cfc0c | Preparing statement | 10.153.42.83 | 5486 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ebff60-cd6c-11e8-af38-373d9c5cfc0c | reading data from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 12093 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ec4d80-cd6c-11e8-af38-373d9c5cfc0c | Sending READ message to /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 13801 | MessagingService-Outgoing-/10.153.42.84-Small
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76edad10-cd6c-11e8-a6c8-79961efb2cb8 | READ message received from /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 928 | MessagingService-Incoming-/10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=>
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee7060-cd6c-11e8-a6c8-79961efb2cb8 | Executing single-partition query on roles | 10.153.42.84 | 5919 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee7061-cd6c-11e8-a6c8-79961efb2cb8 | Acquiring sstable references | 10.153.42.84 | 6327 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee9770-cd6c-11e8-a6c8-79961efb2cb8 | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.153.42.84 | 6802 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee590-cd6c-11e8-a6c8-79961efb2cb8 | Key cache hit for sstable 477 | 10.153.42.84 | 8557 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee591-cd6c-11e8-a6c8-79961efb2cb8 | Bloom filter allows skipping sstable 476 | 10.153.42.84 | 8836 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee592-cd6c-11e8-a6c8-79961efb2cb8 | Merged data from memtables and 2 sstables | 10.153.42.84 | 9252 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca0-cd6c-11e8-a6c8-79961efb2cb8 | Read 1 live and 0 tombstone cells | 10.153.42.84 | 9428 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca1-cd6c-11e8-a6c8-79961efb2cb8 | Enqueuing response to /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 9554 | ReadStage-7
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca2-cd6c-11e8-a6c8-79961efb2cb8 | Sending REQUEST_RESPONSE message to /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 10130 | MessagingService-Outgoing-/10.153.42.83-Small
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef33b0-cd6c-11e8-af38-373d9c5cfc0c | REQUEST_RESPONSE message received from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 32954 | MessagingService-Incoming-/10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=>
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac0-cd6c-11e8-af38-373d9c5cfc0c | Processing response from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 34053 | RequestResponseStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac1-cd6c-11e8-af38-373d9c5cfc0c | reading data from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 34499 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac2-cd6c-11e8-af38-373d9c5cfc0c | Sending READ message to /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 34775 | MessagingService-Outgoing-/10.153.42.84-Small
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d0-cd6c-11e8-a6c8-79961efb2cb8 | READ message received from /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 14 | MessagingService-Incoming-/10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=>
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d1-cd6c-11e8-a6c8-79961efb2cb8 | Executing single-partition query on roles | 10.153.42.84 | 399 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d2-cd6c-11e8-a6c8-79961efb2cb8 | Acquiring sstable references | 10.153.42.84 | 521 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d3-cd6c-11e8-a6c8-79961efb2cb8 | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.153.42.84 | 607 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d4-cd6c-11e8-a6c8-79961efb2cb8 | Key cache hit for sstable 477 | 10.153.42.84 | 736 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e0-cd6c-11e8-a6c8-79961efb2cb8 | Bloom filter allows skipping sstable 476 | 10.153.42.84 | 810 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e0-cd6c-11e8-af38-373d9c5cfc0c | REQUEST_RESPONSE message received from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 36504 | MessagingService-Incoming-/10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=>
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e1-cd6c-11e8-a6c8-79961efb2cb8 | Merged data from memtables and 2 sstables | 10.153.42.84 | 1021 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e1-cd6c-11e8-af38-373d9c5cfc0c | Processing response from /10.153.42.84<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.84&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=c1KkhSnlslC4k-A5mgLTBjQzLE5h7F42WSyOClHxxDU&e=> | 10.153.42.83 | 36729 | RequestResponseStage-5
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e2-cd6c-11e8-a6c8-79961efb2cb8 | Read 1 live and 0 tombstone cells | 10.153.42.84 | 1132 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e3-cd6c-11e8-a6c8-79961efb2cb8 | Enqueuing response to /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 1180 | ReadStage-4
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e4-cd6c-11e8-a6c8-79961efb2cb8 | Sending REQUEST_RESPONSE message to /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.84 | 1279 | MessagingService-Outgoing-/10.153.42.83-Small
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0ba50-cd6c-11e8-af38-373d9c5cfc0c | Index mean cardinalities are status_idx:1. Scanning with status_idx. | 10.153.42.83 | 43356 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0ba51-cd6c-11e8-af38-373d9c5cfc0c | Computing ranges to query | 10.153.42.83 | 43791 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0e160-cd6c-11e8-af38-373d9c5cfc0c | Submitting range requests on 1 ranges with a concurrency of 1 (2.9296876E-4 rows per range expected) | 10.153.42.83 | 44274 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0e161-cd6c-11e8-af38-373d9c5cfc0c | Enqueuing request to /10.153.42.85<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.85&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=kSy9KK2_XRx2wI2_aZkr64v1JjegI6CsOmxc3XNY-Uw&e=> | 10.153.42.83 | 44631 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f10870-cd6c-11e8-af38-373d9c5cfc0c | Submitted 1 concurrent range requests | 10.153.42.83 | 45063 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f10871-cd6c-11e8-af38-373d9c5cfc0c | Sending RANGE_SLICE message to /10.153.42.85<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.85&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=kSy9KK2_XRx2wI2_aZkr64v1JjegI6CsOmxc3XNY-Uw&e=> | 10.153.42.83 | 45074 | MessagingService-Outgoing-/10.153.42.85-Small
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1cbc0-cd6c-11e8-9866-ade6e01c2040 | RANGE_SLICE message received from /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> | 10.153.42.85 | 605 | MessagingService-Incoming-/10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=>
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d0-cd6c-11e8-9866-ade6e01c2040 | Executing read on keysoace.tablenameusing index status_idx | 10.153.42.85 | 2375 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d1-cd6c-11e8-9866-ade6e01c2040 | Executing single-partition query on session.status_idx | 10.153.42.85 | 2599 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d2-cd6c-11e8-9866-ade6e01c2040 | Acquiring sstable references | 10.153.42.85 | 2709 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e0-cd6c-11e8-9866-ade6e01c2040 | Partition index with 456 entries found for sstable 1386 | 10.153.42.85 | 2954 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e1-cd6c-11e8-9866-ade6e01c2040 | Partition index with 11352 entries found for sstable 1367 | 10.153.42.85 | 3240 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e2-cd6c-11e8-9866-ade6e01c2040 | Partition index with 16589 entries found for sstable 838 | 10.153.42.85 | 3413 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e3-cd6c-11e8-9866-ade6e01c2040 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.153.42.85 | 3599 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 7cdf4850-cd6c-11e8-af38-373d9c5cfc0c | Timed out; received 0 of 1 responses | 10.153.42.83 | 9995766 | Native-Transport-Requests-3
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa0-cd6c-11e8-9866-ade6e01c2040 | Executing single-partition query on session | 10.153.42.85 | 45774985 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa1-cd6c-11e8-9866-ade6e01c2040 | Acquiring sstable references | 10.153.42.85 | 45775171 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa2-cd6c-11e8-9866-ade6e01c2040 | Merging memtable contents | 10.153.42.85 | 45775241 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa3-cd6c-11e8-9866-ade6e01c2040 | Merging data from sstable 2489 | 10.153.42.85 | 45775454 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa4-cd6c-11e8-9866-ade6e01c2040 | Partition index with 0 entries found for sstable 2489 | 10.153.42.85 | 45775602 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a81b0-cd6c-11e8-9866-ade6e01c2040 | Merging data from sstable 2455 | 10.153.42.85 | 45775894 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a81b1-cd6c-11e8-9866-ade6e01c2040 | Partition index with 2 entries found for sstable 2455 | 10.153.42.85 | 45776026 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923aa8c0-cd6c-11e8-9866-ade6e01c2040 | Merging data from sstable 2335 | 10.153.42.85 | 45776836 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923aa8c1-cd6c-11e8-9866-ade6e01c2040 | Key cache hit for sstable 2335 | 10.153.42.85 | 45776958 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923be140-cd6c-11e8-9866-ade6e01c2040 | Merging data from sstable 2293 | 10.153.42.85 | 45785346 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923be141-cd6c-11e8-9866-ade6e01c2040 | Key cache hit for sstable 2293 | 10.153.42.85 | 45785532 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923ccba0-cd6c-11e8-9866-ade6e01c2040 | Read 0 live and 0 tombstone cells | 10.153.42.85 | 45791600 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923cf2b0-cd6c-11e8-9866-ade6e01c2040 | Merged data from memtables and 3 sstables | 10.153.42.85 | 45791761 | ReadStage-2
76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923cf2b1-cd6c-11e8-9866-ade6e01c2040 | Discarding partial response to /10.153.42.83<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.153.42.83&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=NHKyznVC_QxysSO0bvJkbLDm-9r7TxkW_Bs7s1rQJZI&e=> (timed out) | 10.153.42.85 | 45791858 | ReadStage-2
On Thu, Oct 11, 2018 at 4:36 PM Adam Zegelin <ad...@instaclustr.com>> wrote:
Hi Abdul,
Can you post the timeout error and also the trace output?
The trace output is detailed, but often refers to Cassandra internals. If we can see the error and the trace it'd be easier to pin-point the problem.
- Adam
On Thu, 11 Oct 2018 at 08:53, Abdul Patel <ab...@gmail.com>> wrote:
Hi,
We have multiple timeouts with select queries.
I ran trace to see why its timing out ..do we jave a good document to interpretate whats causing the queries to timeout?
________________________________
The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: Tracing in cassandra
Posted by Nitan Kainth <ni...@gmail.com>.
Using partition key is best way to query
Sent from my iPhone
> On Oct 12, 2018, at 6:49 PM, Abdul Patel <ab...@gmail.com> wrote:
>
> Yes with range queries its timing out, one question was the where condition is primary key rather than clustering key.
>
>> On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
>> Did it still timeout?
>>
>> Sent from my iPhone
>>
>>> On Oct 12, 2018, at 1:11 PM, Abdul Patel <ab...@gmail.com> wrote:
>>>
>>> With limit 11 this is query..
>>> Select * from table where status=0 and tojen(user_id) >=token(126838) and token(user_id) <= token<somenumber) LIMIT 11;
>>>
>>>> On Friday, October 12, 2018, Abdul Patel <ab...@gmail.com> wrote:
>>>> Let me try with limit 11 ..we have 18 node cluster ..no nodes down..
>>>>
>>>>> On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
>>>>> Try query with partition key selection in where clause. But time for limit 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay sstable?
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>>> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>>>>>>
>>>>>> Sean,
>>>>>>
>>>>>> here it is :
>>>>>> CREATE TABLE Keyspave.tblname (
>>>>>> user_id bigint,
>>>>>> session_id text,
>>>>>> application_guid text,
>>>>>> last_access_time timestamp,
>>>>>> login_time timestamp,
>>>>>> status int,
>>>>>> terminated_by text,
>>>>>> update_time timestamp,
>>>>>> PRIMARY KEY (user_id, session_id)
>>>>>> ) WITH CLUSTERING ORDER BY (session_id ASC)
>>>>>>
>>>>>> also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema?
>>>>>>
>>>>>>> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SE...@homedepot.com> wrote:
>>>>>>> Cross-partition = multiple partitions
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Simple example:
>>>>>>>
>>>>>>> Create table customer (
>>>>>>>
>>>>>>> Customerid int,
>>>>>>>
>>>>>>> Name text,
>>>>>>>
>>>>>>> Lastvisit date,
>>>>>>>
>>>>>>> Phone text,
>>>>>>>
>>>>>>> Primary key (customerid) );
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Query
>>>>>>>
>>>>>>> Select customerid from customer limit 5000;
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sean Durity
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> From: Abdul Patel <ab...@gmail.com>
>>>>>>> Sent: Friday, October 12, 2018 10:04 AM
>>>>>>> To: user@cassandra.apache.org
>>>>>>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Cpuld you elaborate cross partition query?
>>>>>>>
>>>>>>> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com> wrote:
>>>>>>>
>>>>>>> I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sean Durity
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> From: Abdul Patel <ab...@gmail.com>
>>>>>>> Sent: Thursday, October 11, 2018 5:54 PM
>>>>>>> To: adam@instaclustr.com
>>>>>>> Cc: user@cassandra.apache.org
>>>>>>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Query :
>>>>>>>
>>>>>>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>>>>>>
>>>>>>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31 | 10.54.145.32 | 9754 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56 | 10.54.145.32 | 10343 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57 | 10.54.145.32 | 10480 | Native-Transport-Requests-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31 | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56 | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57 | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32 | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32 | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32 | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32 | 10.54.101.57 | 5745 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57 | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57 | 10.54.145.32 | 587 | RequestResponseStage-1
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32 | 10.54.145.31 | 7038 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31 | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31 | 10.54.145.32 | 95 | RequestResponseStage-1
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32 | 10.54.101.56 | 12361 | ReadStage-3
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56 | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56
>>>>>>>
>>>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd |
Re: Tracing in cassandra
Posted by Abdul Patel <ab...@gmail.com>.
Yes with range queries its timing out, one question was the where condition
is primary key rather than clustering key.
On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
> Did it still timeout?
>
> Sent from my iPhone
>
> On Oct 12, 2018, at 1:11 PM, Abdul Patel <ab...@gmail.com> wrote:
>
> With limit 11 this is query..
> Select * from table where status=0 and tojen(user_id) >=token(126838) and
> token(user_id) <= token<somenumber) LIMIT 11;
>
> On Friday, October 12, 2018, Abdul Patel <ab...@gmail.com> wrote:
>
>> Let me try with limit 11 ..we have 18 node cluster ..no nodes down..
>>
>> On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
>>
>>> Try query with partition key selection in where clause. But time for
>>> limit 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay
>>> sstable?
>>>
>>> Sent from my iPhone
>>>
>>> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>>>
>>> Sean,
>>>
>>> here it is :
>>> CREATE TABLE Keyspave.tblname (
>>> user_id bigint,
>>> session_id text,
>>> application_guid text,
>>> last_access_time timestamp,
>>> login_time timestamp,
>>> status int,
>>> terminated_by text,
>>> update_time timestamp,
>>> PRIMARY KEY (user_id, session_id)
>>> ) WITH CLUSTERING ORDER BY (session_id ASC)
>>>
>>> also they see timeouts with limit 11 as well, so is it better to remove
>>> with limit option ? or whats best to query such schema?
>>>
>>> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <
>>> SEAN_R_DURITY@homedepot.com> wrote:
>>>
>>>> Cross-partition = multiple partitions
>>>>
>>>>
>>>>
>>>> Simple example:
>>>>
>>>> Create table customer (
>>>>
>>>> Customerid int,
>>>>
>>>> Name text,
>>>>
>>>> Lastvisit date,
>>>>
>>>> Phone text,
>>>>
>>>> Primary key (customerid) );
>>>>
>>>>
>>>>
>>>> Query
>>>>
>>>> Select customerid from customer limit 5000;
>>>>
>>>>
>>>>
>>>> The query is asking for 5000 different partitions to be selected across
>>>> the cluster. This is a very EXPENSIVE query for Cassandra, especially as
>>>> the number of nodes goes up. Typically, you want to query a single
>>>> partition. Read timeouts are usually caused by queries that are selecting
>>>> many partitions or a very large partition. That is why a schema for the
>>>> involved table could help.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Sean Durity
>>>>
>>>>
>>>>
>>>> *From:* Abdul Patel <ab...@gmail.com>
>>>> *Sent:* Friday, October 12, 2018 10:04 AM
>>>> *To:* user@cassandra.apache.org
>>>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>>>
>>>>
>>>>
>>>> Cpuld you elaborate cross partition query?
>>>>
>>>> On Friday, October 12, 2018, Durity, Sean R <
>>>> SEAN_R_DURITY@homedepot.com> wrote:
>>>>
>>>> I suspect you are doing a cross-partition query, which will not scale
>>>> well (as you can see). What is the schema for the table involved?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Sean Durity
>>>>
>>>>
>>>>
>>>> *From:* Abdul Patel <ab...@gmail.com>
>>>> *Sent:* Thursday, October 11, 2018 5:54 PM
>>>> *To:* adam@instaclustr.com
>>>> *Cc:* user@cassandra.apache.org
>>>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>>>
>>>>
>>>>
>>>> Query :
>>>>
>>>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>>>
>>>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node
>>>> timed out waiting for replica nodes' responses] message="Operation timed
>>>> out - received only 0 responses." info={'received_responses': 0,
>>>> 'required_responses': 1, 'consistency': 'ONE'}
>>>>
>>>>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
>>>> 5000; | 10.54.145.32 | 4020 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Preparing
>>>> statement | 10.54.145.32 | 5065 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Executing
>>>> single-partition query on roles | 10.54.145.32 | 6171
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.145.32 | 6362
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>>>> 10.54.145.32 | 6641 | Re
>>>> adStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Key cache
>>>> hit for sstable 346 | 10.54.145.32 | 6955
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Bloom filter allows
>>>> skipping sstable 347 | 10.54.145.32 | 7202
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Merged
>>>> data from memtables and 2 sstables | 10.54.145.32 | 7386
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Read 1 live and
>>>> 0 tombstone cells | 10.54.145.32 | 7519
>>>> | ReadStage-2
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Executing
>>>> single-partition query on roles | 10.54.145.32 | 7826
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.145.32 | 7924
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>>>> 10.54.145.32 | 8060 |
>>>> ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Key cache
>>>> hit for sstable 346 | 10.54.145.32 | 8137
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Bloom
>>>> filter allows skipping sstable 347 | 10.54.145.32 | 8187
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Merged data from
>>>> memtables and 2 sstables | 10.54.145.32 | 8318
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Read 1 live and
>>>> 0 tombstone cells | 10.54.145.32 | 8941
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>>
>>>> Read-repair DC_LOCAL | 10.54.145.32 | 9468 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> reading data from /10.54.145.31
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>> | 10.54.145.32 | 9754 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Executing
>>>> single-partition query on session | 10.54.145.32 | 10310
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> reading digest
>>>> from /10.54.101.56
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>>> | 10.54.145.32 | 10343 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.145.32 | 10356
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Key cache hit
>>>> for sstable 68642 | 10.54.145.32 | 10445
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> reading digest from /10.54.101.57
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>> | 10.54.145.32 | 10480 |
>>>> Native-Transport-Requests-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Bloom filter allows skipping
>>>> sstable 68397 | 10.54.145.32 | 10496 |
>>>> ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>>>> 10.54.145.32 | 10547 |
>>>> ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Sending READ message
>>>> to /10.54.145.31
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.
>>>> 54.145.31-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Key cache hit
>>>> for sstable 68655 | 10.54.145.32 | 10631
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Sending READ message
>>>> to /10.54.101.56
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>>> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.
>>>> 54.101.56-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Sending READ
>>>> message to /10.54.101.57
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.
>>>> 54.101.57-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Merged data from memtables
>>>> and 3 sstables | 10.54.145.32 | 10863
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Read 13 live and
>>>> 0 tombstone cells | 10.54.145.32 | 10914
>>>> | ReadStage-4
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> READ message received
>>>> from /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.
>>>> 54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> READ message received
>>>> from /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.
>>>> 54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> READ message received from /
>>>> 10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.
>>>> 54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Executing
>>>> single-partition query on session | 10.54.145.31 | 3208
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Executing single-partition
>>>> query on session | 10.54.101.57 | 3258
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.145.31 | 3483
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.101.57 | 3549
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Key cache hit for sstable 74713 |
>>>> 10.54.145.31 | 3732 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Key cache hit for
>>>> sstable 85409 | 10.54.101.57 | 3978 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Executing single-partition
>>>> query on session | 10.54.101.56 | 3497
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Key cache hit
>>>> for sstable 74445 | 10.54.145.31 | 3893 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Bloom filter allows
>>>> skipping sstable 85201 | 10.54.101.57 | 4193
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Bloom filter allows
>>>> skipping sstable 74383 | 10.54.145.31 | 3975
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Skipped 0/3 non-slice-intersecting sstables, included
>>>> 0 due to tombstones | 10.54.101.57 | 4354
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> | Skipped
>>>> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>>>> 10.54.145.31 | 4142 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Key cache hit
>>>> for sstable 85418 | 10.54.101.57 | 4512
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Merged data from
>>>> memtables and 3 sstables | 10.54.101.57 | 4989
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Acquiring
>>>> sstable references | 10.54.101.56 | 3751
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Read 13 live and 0
>>>> tombstone cells | 10.54.101.57 | 5336
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Key
>>>> cache hit for sstable 101147 | 10.54.101.56 | 4091
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Read 13 live and
>>>> 0 tombstone cells | 10.54.101.57 | 5586
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Enqueuing response
>>>> to /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.57 | 5745 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
>>>> |
>>>> Sending REQUEST_RESPONSE message
>>>> to /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.
>>>> 54.145.32-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> REQUEST_RESPONSE message received from /10.54.101.57
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.
>>>> 54.101.57
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Merged data from memtables and
>>>> 3 sstables | 10.54.145.31 | 6600 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Key cache hit
>>>> for sstable 100896 | 10.54.101.56 | 6496
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Processing response
>>>> from /10.54.101.57
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>> | 10.54.145.32 | 587 |
>>>> RequestResponseStage-1
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Read 13 live and
>>>> 0 tombstone cells | 10.54.145.31 | 6892
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Enqueuing response
>>>> to /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.145.31 | 7038 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> REQUEST_RESPONSE message
>>>> received from /10.54.145.31
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.
>>>> 54.145.31
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
>>>> |
>>>> Sending REQUEST_RESPONSE message to /
>>>> 10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.
>>>> 54.145.32-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Bloom filter allows
>>>> skipping sstable 100887 | 10.54.101.56 | 6652
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> Processing response
>>>> from /10.54.145.31
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>> | 10.54.145.32 | 95 |
>>>> RequestResponseStage-1
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
>>>> 10.54.101.56 | 6828 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Key cache hit for
>>>> sstable 101149 | 10.54.101.56 | 7005
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Merged data
>>>> from memtables and 4 sstables | 10.54.101.56 | 9897
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Read 13 live and
>>>> 0 tombstone cells | 10.54.101.56 | 11969
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Read 13 live and
>>>> 0 tombstone cells | 10.54.101.56 | 12067
>>>> | ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Enqueuing response
>>>> to /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.56 | 12361 |
>>>> ReadStage-3
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
>>>> |
>>>> Sending REQUEST_RESPONSE message to /10.54.145.32
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.
>>>> 54.145.32-Small
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>> REQUEST_RESPONSE message received from /
>>>> 10.54.101.56
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>>> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.
>>>> 54.101.56
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>>>
>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
>>>> |
>>>>
>>>>
Re: Tracing in cassandra
Posted by Nitan Kainth <ni...@gmail.com>.
Did it still timeout?
Sent from my iPhone
> On Oct 12, 2018, at 1:11 PM, Abdul Patel <ab...@gmail.com> wrote:
>
> With limit 11 this is query..
> Select * from table where status=0 and tojen(user_id) >=token(126838) and token(user_id) <= token<somenumber) LIMIT 11;
>
>> On Friday, October 12, 2018, Abdul Patel <ab...@gmail.com> wrote:
>> Let me try with limit 11 ..we have 18 node cluster ..no nodes down..
>>
>>> On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
>>> Try query with partition key selection in where clause. But time for limit 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay sstable?
>>>
>>> Sent from my iPhone
>>>
>>>> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>>>>
>>>> Sean,
>>>>
>>>> here it is :
>>>> CREATE TABLE Keyspave.tblname (
>>>> user_id bigint,
>>>> session_id text,
>>>> application_guid text,
>>>> last_access_time timestamp,
>>>> login_time timestamp,
>>>> status int,
>>>> terminated_by text,
>>>> update_time timestamp,
>>>> PRIMARY KEY (user_id, session_id)
>>>> ) WITH CLUSTERING ORDER BY (session_id ASC)
>>>>
>>>> also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema?
>>>>
>>>>> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SE...@homedepot.com> wrote:
>>>>> Cross-partition = multiple partitions
>>>>>
>>>>>
>>>>>
>>>>> Simple example:
>>>>>
>>>>> Create table customer (
>>>>>
>>>>> Customerid int,
>>>>>
>>>>> Name text,
>>>>>
>>>>> Lastvisit date,
>>>>>
>>>>> Phone text,
>>>>>
>>>>> Primary key (customerid) );
>>>>>
>>>>>
>>>>>
>>>>> Query
>>>>>
>>>>> Select customerid from customer limit 5000;
>>>>>
>>>>>
>>>>>
>>>>> The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Sean Durity
>>>>>
>>>>>
>>>>>
>>>>> From: Abdul Patel <ab...@gmail.com>
>>>>> Sent: Friday, October 12, 2018 10:04 AM
>>>>> To: user@cassandra.apache.org
>>>>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>>>>
>>>>>
>>>>>
>>>>> Cpuld you elaborate cross partition query?
>>>>>
>>>>> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com> wrote:
>>>>>
>>>>> I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Sean Durity
>>>>>
>>>>>
>>>>>
>>>>> From: Abdul Patel <ab...@gmail.com>
>>>>> Sent: Thursday, October 11, 2018 5:54 PM
>>>>> To: adam@instaclustr.com
>>>>> Cc: user@cassandra.apache.org
>>>>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>>>>
>>>>>
>>>>>
>>>>> Query :
>>>>>
>>>>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>>>>
>>>>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
>>>>>
>>>>>
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31 | 10.54.145.32 | 9754 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56 | 10.54.145.32 | 10343 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57 | 10.54.145.32 | 10480 | Native-Transport-Requests-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31 | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56 | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57 | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32 | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32 | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32 | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32 | 10.54.101.57 | 5745 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57 | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57 | 10.54.145.32 | 587 | RequestResponseStage-1
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32 | 10.54.145.31 | 7038 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31 | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31 | 10.54.145.32 | 95 | RequestResponseStage-1
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32 | 10.54.101.56 | 12361 | ReadStage-3
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56 | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56
>>>>>
>>>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd |
Re: Tracing in cassandra
Posted by Abdul Patel <ab...@gmail.com>.
With limit 11 this is query..
Select * from table where status=0 and tojen(user_id) >=token(126838) and
token(user_id) <= token<somenumber) LIMIT 11;
On Friday, October 12, 2018, Abdul Patel <ab...@gmail.com> wrote:
> Let me try with limit 11 ..we have 18 node cluster ..no nodes down..
>
> On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
>
>> Try query with partition key selection in where clause. But time for
>> limit 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay
>> sstable?
>>
>> Sent from my iPhone
>>
>> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>>
>> Sean,
>>
>> here it is :
>> CREATE TABLE Keyspave.tblname (
>> user_id bigint,
>> session_id text,
>> application_guid text,
>> last_access_time timestamp,
>> login_time timestamp,
>> status int,
>> terminated_by text,
>> update_time timestamp,
>> PRIMARY KEY (user_id, session_id)
>> ) WITH CLUSTERING ORDER BY (session_id ASC)
>>
>> also they see timeouts with limit 11 as well, so is it better to remove
>> with limit option ? or whats best to query such schema?
>>
>> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <
>> SEAN_R_DURITY@homedepot.com> wrote:
>>
>>> Cross-partition = multiple partitions
>>>
>>>
>>>
>>> Simple example:
>>>
>>> Create table customer (
>>>
>>> Customerid int,
>>>
>>> Name text,
>>>
>>> Lastvisit date,
>>>
>>> Phone text,
>>>
>>> Primary key (customerid) );
>>>
>>>
>>>
>>> Query
>>>
>>> Select customerid from customer limit 5000;
>>>
>>>
>>>
>>> The query is asking for 5000 different partitions to be selected across
>>> the cluster. This is a very EXPENSIVE query for Cassandra, especially as
>>> the number of nodes goes up. Typically, you want to query a single
>>> partition. Read timeouts are usually caused by queries that are selecting
>>> many partitions or a very large partition. That is why a schema for the
>>> involved table could help.
>>>
>>>
>>>
>>>
>>>
>>> Sean Durity
>>>
>>>
>>>
>>> *From:* Abdul Patel <ab...@gmail.com>
>>> *Sent:* Friday, October 12, 2018 10:04 AM
>>> *To:* user@cassandra.apache.org
>>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>>
>>>
>>>
>>> Cpuld you elaborate cross partition query?
>>>
>>> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>
>>> wrote:
>>>
>>> I suspect you are doing a cross-partition query, which will not scale
>>> well (as you can see). What is the schema for the table involved?
>>>
>>>
>>>
>>>
>>>
>>> Sean Durity
>>>
>>>
>>>
>>> *From:* Abdul Patel <ab...@gmail.com>
>>> *Sent:* Thursday, October 11, 2018 5:54 PM
>>> *To:* adam@instaclustr.com
>>> *Cc:* user@cassandra.apache.org
>>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>>
>>>
>>>
>>> Query :
>>>
>>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>>
>>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node
>>> timed out waiting for replica nodes' responses] message="Operation timed
>>> out - received only 0 responses." info={'received_responses': 0,
>>> 'required_responses': 1, 'consistency': 'ONE'}
>>>
>>>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
>>> 5000; | 10.54.145.32 | 4020 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Preparing statement
>>> | 10.54.145.32 | 5065 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Executing
>>> single-partition query on roles | 10.54.145.32 | 6171
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Acquiring
>>> sstable references | 10.54.145.32 | 6362
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>>> 10.54.145.32 | 6641 | Re
>>> adStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Key cache hit
>>> for sstable 346 | 10.54.145.32 | 6955
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Bloom filter allows
>>> skipping sstable 347 | 10.54.145.32 | 7202
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Merged
>>> data from memtables and 2 sstables | 10.54.145.32 | 7386
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Read 1 live and 0
>>> tombstone cells | 10.54.145.32 | 7519
>>> | ReadStage-2
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Executing
>>> single-partition query on roles | 10.54.145.32 | 7826
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Acquiring
>>> sstable references | 10.54.145.32 | 7924
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>>> 10.54.145.32 | 8060 |
>>> ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Key cache hit
>>> for sstable 346 | 10.54.145.32 | 8137
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Bloom filter allows skipping sstable
>>> 347 | 10.54.145.32 | 8187 |
>>> ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Merged data from
>>> memtables and 2 sstables | 10.54.145.32 | 8318
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Read 1 live and 0
>>> tombstone cells | 10.54.145.32 | 8941
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>>
>>> Read-repair DC_LOCAL | 10.54.145.32 | 9468 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> reading data from /10.54.145.31
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>> | 10.54.145.32 | 9754 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Executing
>>> single-partition query on session | 10.54.145.32 | 10310
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> reading digest
>>> from /10.54.101.56
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>> | 10.54.145.32 | 10343 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Acquiring
>>> sstable references | 10.54.145.32 | 10356
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Key cache hit
>>> for sstable 68642 | 10.54.145.32 | 10445
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> reading digest from /10.54.101.57
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>> | 10.54.145.32 | 10480 |
>>> Native-Transport-Requests-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Bloom filter allows skipping
>>> sstable 68397 | 10.54.145.32 | 10496 |
>>> ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>>> 10.54.145.32 | 10547 |
>>> ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Sending READ message
>>> to /10.54.145.31
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.
>>> 54.145.31-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Key cache hit
>>> for sstable 68655 | 10.54.145.32 | 10631
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Sending READ message
>>> to /10.54.101.56
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.
>>> 54.101.56-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Sending READ
>>> message to /10.54.101.57
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.
>>> 54.101.57-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Merged data from memtables
>>> and 3 sstables | 10.54.145.32 | 10863
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.145.32 | 10914
>>> | ReadStage-4
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> READ message received
>>> from /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.
>>> 54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> READ message received
>>> from /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.
>>> 54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> READ message received from /
>>> 10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.
>>> 54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Executing
>>> single-partition query on session | 10.54.145.31 | 3208
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Executing single-partition
>>> query on session | 10.54.101.57 | 3258
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Acquiring
>>> sstable references | 10.54.145.31 | 3483
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Acquiring
>>> sstable references | 10.54.101.57 | 3549
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Key cache hit for sstable 74713 |
>>> 10.54.145.31 | 3732 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Key cache hit for
>>> sstable 85409 | 10.54.101.57 | 3978 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Executing single-partition
>>> query on session | 10.54.101.56 | 3497
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Key cache hit
>>> for sstable 74445 | 10.54.145.31 | 3893 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Bloom filter allows
>>> skipping sstable 85201 | 10.54.101.57 | 4193
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Bloom filter allows
>>> skipping sstable 74383 | 10.54.145.31 | 3975
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Skipped 0/3 non-slice-intersecting sstables, included
>>> 0 due to tombstones | 10.54.101.57 | 4354
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
>>> | Skipped
>>> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>>> 10.54.145.31 | 4142 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Key cache hit
>>> for sstable 85418 | 10.54.101.57 | 4512
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Merged data from
>>> memtables and 3 sstables | 10.54.101.57 | 4989
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Acquiring
>>> sstable references | 10.54.101.56 | 3751
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.101.57 | 5336
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Key
>>> cache hit for sstable 101147 | 10.54.101.56 | 4091
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.101.57 | 5586
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Enqueuing response
>>> to /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.57 | 5745 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
>>> |
>>> Sending REQUEST_RESPONSE message
>>> to /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.
>>> 54.145.32-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> REQUEST_RESPONSE message received from /10.54.101.57
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.
>>> 54.101.57
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Merged data from memtables and
>>> 3 sstables | 10.54.145.31 | 6600 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Key cache hit
>>> for sstable 100896 | 10.54.101.56 | 6496
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Processing response
>>> from /10.54.101.57
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>> | 10.54.145.32 | 587 |
>>> RequestResponseStage-1
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.145.31 | 6892
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Enqueuing response
>>> to /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.145.31 | 7038 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> REQUEST_RESPONSE message
>>> received from /10.54.145.31
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.
>>> 54.145.31
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
>>> |
>>> Sending REQUEST_RESPONSE message to /
>>> 10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.
>>> 54.145.32-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Bloom filter allows
>>> skipping sstable 100887 | 10.54.101.56 | 6652
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> Processing response
>>> from /10.54.145.31
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>> | 10.54.145.32 | 95 |
>>> RequestResponseStage-1
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
>>> 10.54.101.56 | 6828 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Key cache hit for
>>> sstable 101149 | 10.54.101.56 | 7005
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Merged data
>>> from memtables and 4 sstables | 10.54.101.56 | 9897
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.101.56 | 11969
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Read 13 live and 0
>>> tombstone cells | 10.54.101.56 | 12067
>>> | ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Enqueuing response
>>> to /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.56 | 12361 |
>>> ReadStage-3
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
>>> |
>>> Sending REQUEST_RESPONSE message to /10.54.145.32
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.
>>> 54.145.32-Small
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>> REQUEST_RESPONSE message received from /
>>> 10.54.101.56
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.
>>> 54.101.56
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>>
>>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
>>> |
>>>
>>>
Re: Tracing in cassandra
Posted by Abdul Patel <ab...@gmail.com>.
Let me try with limit 11 ..we have 18 node cluster ..no nodes down..
On Friday, October 12, 2018, Nitan Kainth <ni...@gmail.com> wrote:
> Try query with partition key selection in where clause. But time for limit
> 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay
> sstable?
>
> Sent from my iPhone
>
> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>
> Sean,
>
> here it is :
> CREATE TABLE Keyspave.tblname (
> user_id bigint,
> session_id text,
> application_guid text,
> last_access_time timestamp,
> login_time timestamp,
> status int,
> terminated_by text,
> update_time timestamp,
> PRIMARY KEY (user_id, session_id)
> ) WITH CLUSTERING ORDER BY (session_id ASC)
>
> also they see timeouts with limit 11 as well, so is it better to remove
> with limit option ? or whats best to query such schema?
>
> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <
> SEAN_R_DURITY@homedepot.com> wrote:
>
>> Cross-partition = multiple partitions
>>
>>
>>
>> Simple example:
>>
>> Create table customer (
>>
>> Customerid int,
>>
>> Name text,
>>
>> Lastvisit date,
>>
>> Phone text,
>>
>> Primary key (customerid) );
>>
>>
>>
>> Query
>>
>> Select customerid from customer limit 5000;
>>
>>
>>
>> The query is asking for 5000 different partitions to be selected across
>> the cluster. This is a very EXPENSIVE query for Cassandra, especially as
>> the number of nodes goes up. Typically, you want to query a single
>> partition. Read timeouts are usually caused by queries that are selecting
>> many partitions or a very large partition. That is why a schema for the
>> involved table could help.
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> *From:* Abdul Patel <ab...@gmail.com>
>> *Sent:* Friday, October 12, 2018 10:04 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Cpuld you elaborate cross partition query?
>>
>> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>
>> wrote:
>>
>> I suspect you are doing a cross-partition query, which will not scale
>> well (as you can see). What is the schema for the table involved?
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> *From:* Abdul Patel <ab...@gmail.com>
>> *Sent:* Thursday, October 11, 2018 5:54 PM
>> *To:* adam@instaclustr.com
>> *Cc:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Query :
>>
>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>
>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node
>> timed out waiting for replica nodes' responses] message="Operation timed
>> out - received only 0 responses." info={'received_responses': 0,
>> 'required_responses': 1, 'consistency': 'ONE'}
>>
>>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
>> 5000; | 10.54.145.32 | 4020 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Preparing statement
>> | 10.54.145.32 | 5065 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Executing
>> single-partition query on roles | 10.54.145.32 | 6171
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Acquiring
>> sstable references | 10.54.145.32 | 6362
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 | 6641 |
>> ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Key cache hit
>> for sstable 346 | 10.54.145.32 | 6955
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Bloom filter allows
>> skipping sstable 347 | 10.54.145.32 | 7202
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Merged
>> data from memtables and 2 sstables | 10.54.145.32 | 7386
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Read 1 live and 0
>> tombstone cells | 10.54.145.32 | 7519
>> | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Executing single-partition
>> query on roles | 10.54.145.32 | 7826 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Acquiring
>> sstable references | 10.54.145.32 | 7924
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 | 8060 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Key cache hit
>> for sstable 346 | 10.54.145.32 | 8137
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Bloom filter allows skipping sstable
>> 347 | 10.54.145.32 | 8187 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Merged data from memtables
>> and 2 sstables | 10.54.145.32 | 8318
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Read 1 live and 0
>> tombstone cells | 10.54.145.32 | 8941
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
>> |
>>
>> Read-repair DC_LOCAL | 10.54.145.32 | 9468 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
>> |
>> reading data from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 | 9754 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Executing single-partition
>> query on session | 10.54.145.32 | 10310
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
>> |
>> reading digest
>> from /10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 | 10343 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Acquiring
>> sstable references | 10.54.145.32 | 10356
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Key cache hit
>> for sstable 68642 | 10.54.145.32 | 10445
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
>> |
>> reading digest from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 | 10480 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Bloom filter allows skipping
>> sstable 68397 | 10.54.145.32 | 10496 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 | 10547 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Sending READ message
>> to /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.
>> 54.145.31-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Key cache hit
>> for sstable 68655 | 10.54.145.32 | 10631
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Sending READ message
>> to /10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.
>> 54.101.56-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Sending READ
>> message to /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.
>> 54.101.57-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Merged data from memtables and
>> 3 sstables | 10.54.145.32 | 10863 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.145.32 | 10914
>> | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> READ message received
>> from /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> READ message received
>> from /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> READ message received from /
>> 10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Executing
>> single-partition query on session | 10.54.145.31 | 3208
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Executing single-partition
>> query on session | 10.54.101.57 | 3258
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Acquiring
>> sstable references | 10.54.145.31 | 3483
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Acquiring
>> sstable references | 10.54.101.57 | 3549
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Key cache hit for sstable 74713 |
>> 10.54.145.31 | 3732 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Key cache hit for
>> sstable 85409 | 10.54.101.57 | 3978 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Executing single-partition
>> query on session | 10.54.101.56 | 3497
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Key cache hit
>> for sstable 74445 | 10.54.145.31 | 3893 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Bloom filter allows
>> skipping sstable 85201 | 10.54.101.57 | 4193
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Bloom filter allows
>> skipping sstable 74383 | 10.54.145.31 | 3975
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Skipped 0/3 non-slice-intersecting sstables, included
>> 0 due to tombstones | 10.54.101.57 | 4354
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
>> | Skipped
>> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.31 | 4142 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Key cache hit
>> for sstable 85418 | 10.54.101.57 | 4512
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Merged data from memtables
>> and 3 sstables | 10.54.101.57 | 4989
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Acquiring
>> sstable references | 10.54.101.56 | 3751
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.101.57 | 5336
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Key
>> cache hit for sstable 101147 | 10.54.101.56 | 4091
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.101.57 | 5586
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 | 5745 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>> Sending REQUEST_RESPONSE message
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> REQUEST_RESPONSE message received from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.
>> 54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Merged data from memtables and 3
>> sstables | 10.54.145.31 | 6600 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Key cache hit for
>> sstable 100896 | 10.54.101.56 | 6496
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Processing response
>> from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 | 587 |
>> RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.145.31 | 6892
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 | 7038 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> REQUEST_RESPONSE message
>> received from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.
>> 54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>> Sending REQUEST_RESPONSE message to /
>> 10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Bloom filter allows skipping
>> sstable 100887 | 10.54.101.56 | 6652
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Processing response
>> from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 | 95 |
>> RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.101.56 | 6828 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Key cache hit for sstable
>> 101149 | 10.54.101.56 | 7005 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Merged data from
>> memtables and 4 sstables | 10.54.101.56 | 9897
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.101.56 | 11969
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Read 13 live and 0
>> tombstone cells | 10.54.101.56 | 12067
>> | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 | 12361 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Sending REQUEST_RESPONSE message to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
>> |
>> REQUEST_RESPONSE message received from /
>> 10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.
>> 54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
>> |
>>
>>
Re: [EXTERNAL] Re: Tracing in cassandra
Posted by Nitan Kainth <ni...@gmail.com>.
Try query with partition key selection in where clause. But time for limit 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay sstable?
Sent from my iPhone
> On Oct 12, 2018, at 11:40 AM, Abdul Patel <ab...@gmail.com> wrote:
>
> Sean,
>
> here it is :
> CREATE TABLE Keyspave.tblname (
> user_id bigint,
> session_id text,
> application_guid text,
> last_access_time timestamp,
> login_time timestamp,
> status int,
> terminated_by text,
> update_time timestamp,
> PRIMARY KEY (user_id, session_id)
> ) WITH CLUSTERING ORDER BY (session_id ASC)
>
> also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema?
>
>> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SE...@homedepot.com> wrote:
>> Cross-partition = multiple partitions
>>
>>
>>
>> Simple example:
>>
>> Create table customer (
>>
>> Customerid int,
>>
>> Name text,
>>
>> Lastvisit date,
>>
>> Phone text,
>>
>> Primary key (customerid) );
>>
>>
>>
>> Query
>>
>> Select customerid from customer limit 5000;
>>
>>
>>
>> The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> From: Abdul Patel <ab...@gmail.com>
>> Sent: Friday, October 12, 2018 10:04 AM
>> To: user@cassandra.apache.org
>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Cpuld you elaborate cross partition query?
>>
>> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com> wrote:
>>
>> I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> From: Abdul Patel <ab...@gmail.com>
>> Sent: Thursday, October 11, 2018 5:54 PM
>> To: adam@instaclustr.com
>> Cc: user@cassandra.apache.org
>> Subject: [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Query :
>>
>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>
>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
>>
>>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31 | 10.54.145.32 | 9754 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56 | 10.54.145.32 | 10343 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57 | 10.54.145.32 | 10480 | Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31 | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56 | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57 | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32 | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32 | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32 | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32 | 10.54.101.57 | 5745 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57 | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57 | 10.54.145.32 | 587 | RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32 | 10.54.145.31 | 7038 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31 | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31 | 10.54.145.32 | 95 | RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32 | 10.54.101.56 | 12361 | ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56 | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd |
RE: [EXTERNAL] Re: Tracing in cassandra
Posted by "Durity, Sean R" <SE...@homedepot.com>.
So, querying for a status column that is not part of the partition key will not scale – especially if there are only a few status codes. Every node in the cluster will be reading all the data in the table and trying to send to the coordinator node.
With some possibly erroneous assumptions on what you are trying to do, I would suggest looking at something more like:
Create table active_sessions
Primary key ((user, session),session_start_time DESC)
(perhaps using a TTL to expire data)
(And if you need the historical information)
Create table inactive_sessions
Primary key ((user, session), session_end_time DESC)
(also consider adding a time bucket to the primary key, if this will get large)
Much depends on volume, how you actually use the data, how long it can last, etc. But the place to start is to have a table for each query upon which you can execute at least a partition key query. Table scans in Cassandra (alone) are not the right access plan. (Spark is one way around this for certain analytics use cases.)
Sean Durity
lord of the (C*) rings (Staff Systems Engineer – Cassandra)
MTC 2250
#cassandra - for the latest news and updates
From: Abdul Patel <ab...@gmail.com>
Sent: Friday, October 12, 2018 12:41 PM
To: user@cassandra.apache.org
Subject: Re: [EXTERNAL] Re: Tracing in cassandra
Sean,
here it is :
CREATE TABLE Keyspave.tblname (
user_id bigint,
session_id text,
application_guid text,
last_access_time timestamp,
login_time timestamp,
status int,
terminated_by text,
update_time timestamp,
PRIMARY KEY (user_id, session_id)
) WITH CLUSTERING ORDER BY (session_id ASC)
also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema?
On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SE...@homedepot.com>> wrote:
Cross-partition = multiple partitions
Simple example:
Create table customer (
Customerid int,
Name text,
Lastvisit date,
Phone text,
Primary key (customerid) );
Query
Select customerid from customer limit 5000;
The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.
Sean Durity
From: Abdul Patel <ab...@gmail.com>>
Sent: Friday, October 12, 2018 10:04 AM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: [EXTERNAL] Re: Tracing in cassandra
Cpuld you elaborate cross partition query?
On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>> wrote:
I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
Sean Durity
From: Abdul Patel <ab...@gmail.com>>
Sent: Thursday, October 11, 2018 5:54 PM
To: adam@instaclustr.com<ma...@instaclustr.com>
Cc: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: [EXTERNAL] Re: Tracing in cassandra
Query :
SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
-Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 9754 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10343 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10480 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 5745 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 587 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7038 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 95 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 12361 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd |
________________________________
The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: [EXTERNAL] Re: Tracing in cassandra
Posted by Abdul Patel <ab...@gmail.com>.
Sean,
here it is :
CREATE TABLE Keyspave.tblname (
user_id bigint,
session_id text,
application_guid text,
last_access_time timestamp,
login_time timestamp,
status int,
terminated_by text,
update_time timestamp,
PRIMARY KEY (user_id, session_id)
) WITH CLUSTERING ORDER BY (session_id ASC)
also they see timeouts with limit 11 as well, so is it better to remove
with limit option ? or whats best to query such schema?
On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SE...@homedepot.com>
wrote:
> Cross-partition = multiple partitions
>
>
>
> Simple example:
>
> Create table customer (
>
> Customerid int,
>
> Name text,
>
> Lastvisit date,
>
> Phone text,
>
> Primary key (customerid) );
>
>
>
> Query
>
> Select customerid from customer limit 5000;
>
>
>
> The query is asking for 5000 different partitions to be selected across
> the cluster. This is a very EXPENSIVE query for Cassandra, especially as
> the number of nodes goes up. Typically, you want to query a single
> partition. Read timeouts are usually caused by queries that are selecting
> many partitions or a very large partition. That is why a schema for the
> involved table could help.
>
>
>
>
>
> Sean Durity
>
>
>
> *From:* Abdul Patel <ab...@gmail.com>
> *Sent:* Friday, October 12, 2018 10:04 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>
>
>
> Cpuld you elaborate cross partition query?
>
> On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>
> wrote:
>
> I suspect you are doing a cross-partition query, which will not scale well
> (as you can see). What is the schema for the table involved?
>
>
>
>
>
> Sean Durity
>
>
>
> *From:* Abdul Patel <ab...@gmail.com>
> *Sent:* Thursday, October 11, 2018 5:54 PM
> *To:* adam@instaclustr.com
> *Cc:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>
>
>
> Query :
>
> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>
> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed
> out waiting for replica nodes' responses] message="Operation timed out -
> received only 0 responses." info={'received_responses': 0,
> 'required_responses': 1, 'consistency': 'ONE'}
>
>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
> |
> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
> 5000; | 10.54.145.32 | 4020 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70bfed0-cd9e-11e8-8e99-15807bff4dfd
> |
> Preparing statement |
> 10.54.145.32 | 5065 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c25e0-cd9e-11e8-8e99-15807bff4dfd |
> Executing
> single-partition query on roles | 10.54.145.32 | 6171
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring sstable references | 10.54.145.32 | 6362
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 6641 |
> ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for sstable 346 | 10.54.145.32 | 6955
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
> |
> Bloom filter allows skipping sstable 347 | 10.54.145.32
> | 7202 | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7400-cd9e-11e8-8e99-15807bff4dfd
> |
> Merged data
> from memtables and 2 sstables | 10.54.145.32 | 7386
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7401-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7402-cd9e-11e8-8e99-15807bff4dfd
> |
> Executing single-partition query on roles | 10.54.145.32 | 7826
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7403-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring sstable references | 10.54.145.32 | 7924
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7404-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 8060 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7405-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for
> sstable 346 | 10.54.145.32 | 8137
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7406-cd9e-11e8-8e99-15807bff4dfd |
> Bloom
> filter allows skipping sstable 347 | 10.54.145.32 | 8187
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c7407-cd9e-11e8-8e99-15807bff4dfd
> |
> Merged data from memtables and 2 sstables | 10.54.145.32 | 8318
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70c9b10-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70cc220-cd9e-11e8-8e99-15807bff4dfd
> |
> Read-repair DC_LOCAL | 10.54.145.32 | 9468 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70cc221-cd9e-11e8-8e99-15807bff4dfd
> |
> reading data from /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 9754 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70cc222-cd9e-11e8-8e99-15807bff4dfd |
> Executing
> single-partition query on session | 10.54.145.32 | 10310
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce930-cd9e-11e8-8e99-15807bff4dfd
> |
> reading digest from /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 10343 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce931-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring sstable references | 10.54.145.32 | 10356
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce932-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for sstable 68642 | 10.54.145.32 | 10445
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce933-cd9e-11e8-8e99-15807bff4dfd
> |
> reading digest from /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 10480 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce934-cd9e-11e8-8e99-15807bff4dfd |
> Bloom
> filter allows skipping sstable 68397 | 10.54.145.32 | 10496
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce935-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 10547 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce936-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ message to /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 10575 |
> MessagingService-Outgoing-/10.54.145.31-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce937-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for sstable 68655 | 10.54.145.32 | 10631
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce938-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ message to /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 10695 |
> MessagingService-Outgoing-/10.54.101.56-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce939-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ
> message to /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 10823 |
> MessagingService-Outgoing-/10.54.101.57-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce93a-cd9e-11e8-8e99-15807bff4dfd |
> Merged
> data from memtables and 3 sstables | 10.54.145.32 | 10863
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ce93b-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> READ message received from /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 883 | MessagingService-Incoming-/
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dac80-cd9e-11e8-ba8b-bbf19da89f90
> |
> READ message received from /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 803 | MessagingService-Incoming-/
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
> |
> READ message received from /
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 961 | MessagingService-Incoming-/
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba |
> Executing
> single-partition query on session | 10.54.145.31 | 3208
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Executing single-partition query on session | 10.54.101.57 | 3258
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Acquiring sstable references | 10.54.145.31 | 3483
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Acquiring sstable references | 10.54.101.57 | 3549
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Key cache hit for sstable 74713 |
> 10.54.145.31 | 3732 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 |
> Key
> cache hit for sstable 85409 | 10.54.101.57 | 3978
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Executing single-partition query on session | 10.54.101.56 | 3497
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Key cache hit for sstable 74445 | 10.54.145.31 | 3893
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Bloom filter allows skipping sstable 74383 | 10.54.145.31 |
> 3975 | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
> |
> Skipped 0/3 non-slice-intersecting sstables, included
> 0 due to tombstones | 10.54.101.57 | 4354
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba |
> Skipped
> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.31 | 4142 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
> |
> Key cache hit for sstable 85418 | 10.54.101.57 | 4512
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Merged data from memtables and 3 sstables | 10.54.101.57 | 4989
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Acquiring sstable references | 10.54.101.56 | 3751
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Read 13 live and 0 tombstone
> cells | 10.54.101.57 | 5336 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c |
> Key
> cache hit for sstable 101147 | 10.54.101.56 | 4091
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
> |
> Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Enqueuing response to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 5745 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Sending REQUEST_RESPONSE message to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 6473 |
> MessagingService-Outgoing-/10.54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e0-cd9e-11e8-8e99-15807bff4dfd
> |
> REQUEST_RESPONSE message received from /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 57 | MessagingService-Incoming-/
> 10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba |
> Merged
> data from memtables and 3 sstables | 10.54.145.31 | 6600
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Key cache hit for sstable 100896 | 10.54.101.56 | 6496
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e1-cd9e-11e8-8e99-15807bff4dfd
> |
> Processing response from /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 587 |
> RequestResponseStage-1
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Enqueuing response to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 7038 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
> |
> REQUEST_RESPONSE message
> received from /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 15 | MessagingService-Incoming-/
> 10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba |
> Sending
> REQUEST_RESPONSE message to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 7641 |
> MessagingService-Outgoing-/10.54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
> |
> Processing response from /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 95 |
> RequestResponseStage-1
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.101.56 | 6828 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Key cache hit for sstable
> 101149 | 10.54.101.56 | 7005 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70f3320-cd9e-11e8-bed7-f13ab36bfa4c |
> Merged
> data from memtables and 4 sstables | 10.54.101.56 | 9897
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Enqueuing response to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 12361 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Sending REQUEST_RESPONSE message to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 14161 |
> MessagingService-Outgoing-/10.54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ff670-cd9e-11e8-8e99-15807bff4dfd |
> REQUEST_RESPONSE
> message received from /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 51 | MessagingService-Incoming-/
> 10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd |
> e70ff671-cd9e-11e8-8e99-15807bff4dfd |
>
>
RE: [EXTERNAL] Re: Tracing in cassandra
Posted by "Durity, Sean R" <SE...@homedepot.com>.
Cross-partition = multiple partitions
Simple example:
Create table customer (
Customerid int,
Name text,
Lastvisit date,
Phone text,
Primary key (customerid) );
Query
Select customerid from customer limit 5000;
The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.
Sean Durity
From: Abdul Patel <ab...@gmail.com>
Sent: Friday, October 12, 2018 10:04 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Tracing in cassandra
Cpuld you elaborate cross partition query?
On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>> wrote:
I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?
Sean Durity
From: Abdul Patel <ab...@gmail.com>>
Sent: Thursday, October 11, 2018 5:54 PM
To: adam@instaclustr.com<ma...@instaclustr.com>
Cc: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: [EXTERNAL] Re: Tracing in cassandra
Query :
SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
-Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 | 4020 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 5065 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 6171 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 6362 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 6641 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 6955 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 7202 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 7386 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 | ReadStage-2
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 7826 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 7924 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 8060 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 8137 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 8187 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd | Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd | Read-repair DC_LOCAL | 10.54.145.32 | 9468 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd | reading data from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 9754 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on session | 10.54.145.32 | 10310 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10343 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 10356 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68642 | 10.54.145.32 | 10445 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd | reading digest from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10480 | Native-Transport-Requests-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 68397 | 10.54.145.32 | 10496 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 10547 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.54.145.31-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 68655 | 10.54.145.32 | 10631 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.54.101.56-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd | Sending READ message to /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.54.101.57-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd | Merged data from memtables and 3 sstables | 10.54.145.32 | 10863 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd | Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 | ReadStage-4
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c | READ message received from /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | Executing single-partition query on session | 10.54.145.31 | 3208 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 | Executing single-partition query on session | 10.54.101.57 | 3258 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba | Acquiring sstable references | 10.54.145.31 | 3483 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 | Acquiring sstable references | 10.54.101.57 | 3549 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74713 | 10.54.145.31 | 3732 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85409 | 10.54.101.57 | 3978 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c | Executing single-partition query on session | 10.54.101.56 | 3497 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba | Key cache hit for sstable 74445 | 10.54.145.31 | 3893 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 | Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba | Bloom filter allows skipping sstable 74383 | 10.54.145.31 | 3975 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 | 4354 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 | 4142 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 | Key cache hit for sstable 85418 | 10.54.101.57 | 4512 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 | Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c | Acquiring sstable references | 10.54.101.56 | 3751 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5336 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101147 | 10.54.101.56 | 4091 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 | Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 5745 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | Merged data from memtables and 3 sstables | 10.54.145.31 | 6600 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 100896 | 10.54.101.56 | 6496 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.57<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> | 10.54.145.32 | 587 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba | Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7038 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c | Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.145.31<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> | 10.54.145.32 | 95 | RequestResponseStage-1
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c | Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 | 6828 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c | Key cache hit for sstable 101149 | 10.54.101.56 | 7005 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | Merged data from memtables and 4 sstables | 10.54.101.56 | 9897 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c | Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c | Enqueuing response to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 12361 | ReadStage-3
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c | Sending REQUEST_RESPONSE message to /10.54.145.32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.54.145.32-Small
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd | REQUEST_RESPONSE message received from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd | Processing response from /10.54.101.56<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> | 10.54.145.32 | 206 | RequestResponseStage-7
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d32f1-cd9e-11e8-8e99-15807bff4dfd | Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND token(user_id) >= token(769281382) AND token(user_id) <= token(769281382) LIMIT 12; | 10.54.145.32 | 519 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d5a00-cd9e-11e8-8e99-15807bff4dfd | Preparing statement | 10.54.145.32 | 906 | Native-Transport-Requests-2
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8110-cd9e-11e8-8e99-15807bff4dfd | Executing single-partition query on roles | 10.54.145.32 | 1820 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8111-cd9e-11e8-8e99-15807bff4dfd | Acquiring sstable references | 10.54.145.32 | 1884 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8112-cd9e-11e8-8e99-15807bff4dfd | Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 | 1968 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8113-cd9e-11e8-8e99-15807bff4dfd | Key cache hit for sstable 346 | 10.54.145.32 | 2023 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8114-cd9e-11e8-8e99-15807bff4dfd | Bloom filter allows skipping sstable 347 | 10.54.145.32 | 2076 | ReadStage-1
f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8115-cd9e-11e8-8e99-15807bff4dfd |
________________________________
The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: Tracing in cassandra
Posted by Abdul Patel <ab...@gmail.com>.
Cpuld you elaborate cross partition query?
On Friday, October 12, 2018, Durity, Sean R <SE...@homedepot.com>
wrote:
> I suspect you are doing a cross-partition query, which will not scale well
> (as you can see). What is the schema for the table involved?
>
>
>
>
>
> Sean Durity
>
>
>
> *From:* Abdul Patel <ab...@gmail.com>
> *Sent:* Thursday, October 11, 2018 5:54 PM
> *To:* adam@instaclustr.com
> *Cc:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>
>
>
> Query :
>
> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>
> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed
> out waiting for replica nodes' responses] message="Operation timed out -
> received only 0 responses." info={'received_responses': 0,
> 'required_responses': 1, 'consistency': 'ONE'}
>
>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
> |
> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
> 5000; | 10.54.145.32 | 4020 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
> |
> Preparing statement |
> 10.54.145.32 | 5065 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
> |
> Executing
> single-partition query on roles | 10.54.145.32 | 6171
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring
> sstable references | 10.54.145.32 | 6362
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 6641 |
> ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit
> for sstable 346 | 10.54.145.32 | 6955
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
> |
> Bloom filter allows
> skipping sstable 347 | 10.54.145.32 | 7202
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
> |
> Merged data from memtables and 2 sstables
> | 10.54.145.32 | 7386 |
> ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 1 live and 0
> tombstone cells | 10.54.145.32 | 7519
> | ReadStage-2
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
> |
> Executing single-partition
> query on roles | 10.54.145.32 | 7826 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring
> sstable references | 10.54.145.32 | 7924
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 8060 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit
> for sstable 346 | 10.54.145.32 | 8137
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
> |
> Bloom filter allows skipping sstable
> 347 | 10.54.145.32 | 8187 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
> |
> Merged data from memtables
> and 2 sstables | 10.54.145.32 | 8318
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 1 live and 0
> tombstone cells | 10.54.145.32 | 8941
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
> |
>
> Read-repair DC_LOCAL | 10.54.145.32 | 9468 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
> |
> reading data from /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 9754 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
> |
> Executing single-partition
> query on session | 10.54.145.32 | 10310
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
> |
> reading digest from
> /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 10343 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring
> sstable references | 10.54.145.32 | 10356
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for
> sstable 68642 | 10.54.145.32 | 10445 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
> |
> reading digest from /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 10480 |
> Native-Transport-Requests-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
> |
> Bloom filter allows skipping
> sstable 68397 | 10.54.145.32 | 10496 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
> |
> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 10547 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ message to
> /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 10575 | MessagingService-Outgoing-/10.
> 54.145.31-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit for
> sstable 68655 | 10.54.145.32 | 10631 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ message to
> /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 10695 | MessagingService-Outgoing-/10.
> 54.101.56-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
> |
> Sending READ
> message to /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 10823 | MessagingService-Outgoing-/10.
> 54.101.57-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
> |
> Merged data from memtables and
> 3 sstables | 10.54.145.32 | 10863 |
> ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
> |
> Read 13 live and 0
> tombstone cells | 10.54.145.32 | 10914
> | ReadStage-4
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> READ message received from
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 883 | MessagingService-Incoming-/10.
> 54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
> |
> READ message received from
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 803 | MessagingService-Incoming-/10.
> 54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
> |
> READ message received from /
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 961 | MessagingService-Incoming-/10.
> 54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Executing
> single-partition query on session | 10.54.145.31 | 3208
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Executing single-partition
> query on session | 10.54.101.57 | 3258
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Acquiring
> sstable references | 10.54.145.31 | 3483
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Acquiring
> sstable references | 10.54.101.57 | 3549
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Key cache hit for sstable 74713 |
> 10.54.145.31 | 3732 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Key cache hit for
> sstable 85409 | 10.54.101.57 | 3978 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Executing single-partition
> query on session | 10.54.101.56 | 3497
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Key cache hit for
> sstable 74445 | 10.54.145.31 | 3893 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Bloom filter allows skipping
> sstable 85201 | 10.54.101.57 | 4193 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Bloom filter allows skipping
> sstable 74383 | 10.54.145.31 | 3975 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
> |
> Skipped 0/3 non-slice-intersecting sstables, included
> 0 due to tombstones | 10.54.101.57 | 4354
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
> | Skipped
> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.31 | 4142 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
> |
> Key cache hit for
> sstable 85418 | 10.54.101.57 | 4512 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Merged data from memtables
> and 3 sstables | 10.54.101.57 | 4989
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Acquiring
> sstable references | 10.54.101.56 | 3751
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Read 13 live and 0 tombstone
> cells | 10.54.101.57 | 5336 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Key
> cache hit for sstable 101147 | 10.54.101.56 | 4091
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
> |
> Read 13 live and 0
> tombstone cells | 10.54.101.57 | 5586
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
> |
> Enqueuing response to
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 5745 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
> |
> Sending REQUEST_RESPONSE message to
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.57 | 6473 | MessagingService-Outgoing-/10.
> 54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
> |
> REQUEST_RESPONSE message received from /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 57 | MessagingService-Incoming-/10.
> 54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Merged data from memtables and 3
> sstables | 10.54.145.31 | 6600 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Key cache hit for
> sstable 100896 | 10.54.101.56 | 6496
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
> |
> Processing response from
> /10.54.101.57
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
> | 10.54.145.32 | 587 |
> RequestResponseStage-1
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Read 13 live and 0
> tombstone cells | 10.54.145.31 | 6892
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Enqueuing response to
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 7038 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
> |
> REQUEST_RESPONSE message
> received from /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 15 | MessagingService-Incoming-/10.
> 54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
> |
> Sending REQUEST_RESPONSE message to /
> 10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.145.31 | 7641 | MessagingService-Outgoing-/10.
> 54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Bloom filter allows skipping
> sstable 100887 | 10.54.101.56 | 6652
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
> |
> Processing response from
> /10.54.145.31
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
> | 10.54.145.32 | 95 |
> RequestResponseStage-1
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.101.56 | 6828 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Key cache hit for sstable
> 101149 | 10.54.101.56 | 7005 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Merged data from
> memtables and 4 sstables | 10.54.101.56 | 9897
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Read 13 live and 0
> tombstone cells | 10.54.101.56 | 11969
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Read 13 live and 0
> tombstone cells | 10.54.101.56 | 12067
> | ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Enqueuing response to
> /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 12361 |
> ReadStage-3
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
> |
> Sending REQUEST_RESPONSE message to /10.54.145.32
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
> | 10.54.101.56 | 14161 | MessagingService-Outgoing-/10.
> 54.145.32-Small
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
> |
> REQUEST_RESPONSE message received from /
> 10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 51 | MessagingService-Incoming-/10.
> 54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>
> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
> |
> Processing response from
> /10.54.101.56
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
> | 10.54.145.32 | 206 |
> RequestResponseStage-7
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d32f1-cd9e-11e8-8e99-15807bff4dfd
> | Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND
> token(user_id) >= token(769281382) AND token(user_id) <= token(769281382)
> LIMIT 12; | 10.54.145.32 | 519 |
> Native-Transport-Requests-2
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d5a00-cd9e-11e8-8e99-15807bff4dfd
> |
>
> Preparing statement | 10.54.145.32 | 906 |
> Native-Transport-Requests-2
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8110-cd9e-11e8-8e99-15807bff4dfd
> |
> Executing single-partition
> query on roles | 10.54.145.32 | 1820
> | ReadStage-1
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8111-cd9e-11e8-8e99-15807bff4dfd
> |
> Acquiring
> sstable references | 10.54.145.32 | 1884
> | ReadStage-1
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8112-cd9e-11e8-8e99-15807bff4dfd
> | Skipped
> 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
> 10.54.145.32 | 1968 |
> ReadStage-1
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8113-cd9e-11e8-8e99-15807bff4dfd
> |
> Key cache hit
> for sstable 346 | 10.54.145.32 | 2023
> | ReadStage-1
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8114-cd9e-11e8-8e99-15807bff4dfd
> |
> Bloom filter allows
> skipping sstable 347 | 10.54.145.32 | 2076
> | ReadStage-1
>
> f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8115-cd9e-11e8-8e99-15807bff4dfd
> |
>
>