You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Andrew Cobley <a....@dundee.ac.uk> on 2013/07/16 10:16:14 UTC
V2 Beta 1 bootstrap behaviour ?
Hi all
I'm setting up a new test cluster using 2.0.0-beta1 and I noticed the following behaviour with vnodes turned on. I bring up one node all well and good. however if I bring up a second node, that can't contact the first (the first being the seed for the second) after a short period of time, the second goes ahead and assumes it's the only node and bootstraps with all tokens. Is this correct behaviour? I'd have thought it would halt with a "Can't contact seed" message to avoid the node starting in a possibly mis-confgured state ?
Startup trace (from the can't contact the seed messages below).
Regards
Andy
DEBUG 17:57:46,134 Target max version is -2147483648; no version information yet, will retry
INFO 17:57:46,139 Cannot handshake version with /192.168.0.10
INFO 17:57:46,147 Handshaking version with /192.168.0.10
DEBUG 17:57:51,147 Target max version is -2147483648; no version information yet, will retry
INFO 17:57:51,151 Cannot handshake version with /192.168.0.10
DEBUG 17:57:51,156 attempting to connect to /192.168.0.10
INFO 17:57:51,164 Handshaking version with /192.168.0.10
DEBUG 17:57:56,163 Target max version is -2147483648; no version information yet, will retry
INFO 17:57:56,168 Cannot handshake version with /192.168.0.10
INFO 17:57:56,177 Handshaking version with /192.168.0.10
DEBUG 17:58:01,176 Target max version is -2147483648; no version information yet, will retry
INFO 17:58:01,195 Cannot handshake version with /192.168.0.10
DEBUG 17:58:01,199 attempting to connect to /192.168.0.10
INFO 17:58:01,211 Handshaking version with /192.168.0.10
DEBUG 17:58:06,210 Target max version is -2147483648; no version information yet, will retry
INFO 17:58:06,229 Cannot handshake version with /192.168.0.10
INFO 17:58:06,243 Handshaking version with /192.168.0.10
DEBUG 17:58:11,240 Target max version is -2147483648; no version information yet, will retry
INFO 17:58:11,257 Cannot handshake version with /192.168.0.10
DEBUG 17:58:11,265 attempting to connect to /192.168.0.10
DEBUG 17:58:11,249 Disseminating load info ...
INFO 17:58:11,277 Handshaking version with /192.168.0.10
INFO 17:58:12,918 JOINING: Starting to bootstrap...
DEBUG 17:58:12,922 Beginning bootstrap process
INFO 17:58:13,065 Bootstrap completed! for the tokens [-6323000812485711377, 7540920246740048508, 8343754522398236724, 8408024349763410671, 2958176648006585896, -4558504792860463383, 6505045068820279515, -2723585437376250373, 6295277530571293136, -632036933490110017, 7076829522639601545, -972040818133908807, -8032509273357338597, -8195415716479930448, 2815056297233670800, -8514945661303914275, -1404986336850055825, 1902287316263855857, 2271127848797136099, 2253027225668615491, -4596682821656889301, -3829603337268068014, 6732451001921723049, -7525322146388049675, -7520469144418445567, -42294142306824200, -3583505036108061780, 2037651260804506174, -8857382013269212764, -287803492046061484, -2387955156962752445, 5851040736860755910, -2566990238894069943, -4862528385953509677, -6934815927114679513, 436339434870862426, -9024405707727859684, -5712896424965939691, -7306151990958768726, 1972562868127181045, 4322111133230504092, 3274187527528688964, -170723646511099548, -5215516480259793274, -5433009735911985650, 1611629741105319295, -1003574501709202508, -342125947698355485, -6982367747787992083, -1265530805719316699, -3787322892048394796, 4335532059375005778, -4955726035129474058, -3371084287244267978, -1356714364316547864, 2509890377680956235, -1688681099182070115, 4105757083923252042, 6251650102550794764, 6814340446137923947, 2350550282350068760, -5827526901688128468, 6417310693620297152, -3072998771907077372, -1401160349313134159, 5689378233308186158, -4579053362657493044, -1974282572872555977, 6030416333994244776, -6459751443485456668, -7053683753232252424, 3313304008102967685, -3323360202881321267, 1777100483859232105, 644078824255348160, 9174385969124367717, -3152648150979214720, -5810274955975592409, -2417697781941443512, 2097979287894624485, 1306279112226821961, -5471073413421323422, -899964525929809433, 5721993461796585132, 6287709076354945844, -2618973873461192043, -4901233632190356360, 2810307178808259886, 414264499406348099, 9204881637988352346, 2756156004752906240, 4264704223664142541, 6966682145360577932, 456907978019976961, 3526133307919328434, 6663482402885505474, 1208521381475842799, 3709424135702812997, 2664447310834129793, 2184366192501394990, -9184383021035672777, 1764656501925028031, -8960542797122524994, 5425128251068131909, -5995018354184249099, 7807241096173494918, -4950974144156281148, 5174743236438194862, 1244619037182110640, -2546311664945762019, -5553188653211189061, 8804167436403071449, -3529889955411356312, 6118046371500892857, 3320865090004456676, -6903778681629605976, 6386743816636437221, -9071719352823462138, -6576758648871647588, 5987037537853395238, 986125783873493733, 2560513262779074155, 2664163134953574068, -4785699651647692687, -2423779828439331130, 3204264553240839859, -3032730799546050734, -1119983597542134724, 8581101667680406496, -5853780338109789104, 6618776578435553577, 8127528839792128464, 3981688355788388288, -3276943142712007267, -5553559033592503934, -565586245499042749, 2698146986917107680, -4118747694524450398, -94197050448104434, -7041621829760205318, -1204017316777816708, 3229847261799003880, -6279870462178797449, -990289430614639535, 8443909287132310249, -4274511997658117637, 3785578998658569690, -6359618184454198961, 2665969438726830287, 8054750200135920151, -3934821671677811317, -485635553910904013, -623380094895579520, -3472488394961304835, -7526106638135734955, 1890693942329731243, 5244101280950037554, 4583546775549106643, 3895567705889066228, 5334628314720458147, -8596631607607319391, 3740554219906914028, 9095268733721810466, 2758206461555504574, -6162081928909388617, 8708338036502321790, 5183092910492165645, 1829498305562316123, -2572585900623957771, -8260878088704282035, 7734738546550767507, 1832837517438053898, 4324222608062794450, -474214797246154002, -6069598122091042081, -9142441383540253639, 2815706000764693552, 5967442704636348366, 2087292969799552844, 5828375876428174510, 6379276420430849033, -8271396753288625388, 7780395417475412741, -1390444458813154553, -5453290611375819442, 6083896792337057679, -4257643687424988922, 770735604922246946, 2216654002368848583, -7748279779200518330, 1944728945767071214, -2294194973999153845, -3603821754103775284, 4721521319267790087, 5372973717379594525, -8275676737706078573, 5897461205541082929, -695930662834055090, 2719486353185233220, -392331656470549574, -1568658082513590058, -5648478850489989928, -4812696232891045407, 2401615522584549554, -6464420048640302972, -5046677986523856430, 780995344937361068, -1233002335258848816, -6735843204069900975, 8044399031264580270, -4871656601660382401, 8297269945276814669, -1666262333518459589, 1442708638138645005, -101325079897371283, -3146272811424619794, -2142311001363252112, 3924382339535466270, -6761931011837249238, 6504024115050419271, -3429178218050542445, 4270740334962704112, -385458615439681688, 4381980869381678991, -953904416719284833, 8298012175524755475, 7844241744998944762, -9084311131189837541, 8001607762992085251, -7252657913919874500, -4710023932873815473, -8312370105536886283, 591824176525945018, -3603135508858098035, -8594324420901678749, -7624329750282992340, 3109508350842625585, 3086473628596783752, -7805741803336097762, 3652336128698228111, 9151260251441783444, -2571481641165315710, -3983064571066933508, -4414430870012910693, 7041889069507587063, -3533275297825790658, 2274185594739201969, -6219740351350975938, 5603158837900577920, 8831369449751982065, 1992022020144601219, -3286196597800579880, 2977787458295061172, -8243451741655531306, 1066786986080576383, -1069941784245844754]
INFO 17:58:13,097 Enqueuing flush of Memtable-local@6452952(43/43 serialized/live bytes, 2 ops)
INFO 17:58:13,103 Writing Memtable-local@6452952(43/43 serialized/live bytes, 2 ops)
DEBUG 17:58:13,375 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-CompressionInfo.db to /var/lib/cassandra/data/system/local/system-local-ja-6-CompressionInfo.db
DEBUG 17:58:13,380 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-Statistics.db to /var/lib/cassandra/data/system/local/system-local-ja-6-Statistics.db
DEBUG 17:58:13,385 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-Filter.db to /var/lib/cassandra/data/system/local/system-local-ja-6-Filter.db
DEBUG 17:58:13,390 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-Index.db to /var/lib/cassandra/data/system/local/system-local-ja-6-Index.db
DEBUG 17:58:13,395 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-TOC.txt to /var/lib/cassandra/data/system/local/system-local-ja-6-TOC.txt
DEBUG 17:58:13,400 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-6-Data.db to /var/lib/cassandra/data/system/local/system-local-ja-6-Data.db
INFO 17:58:13,415 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-6-Data.db (70 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=93668)
DEBUG 17:58:13,420 adding /var/lib/cassandra/data/system/local/system-local-ja-6 to list of files tracked for system.local
DEBUG 17:58:13,425 Scheduling a background task check for system.local with SizeTieredCompactionStrategy
DEBUG 17:58:13,428 Checking system.local
DEBUG 17:58:13,432 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-6-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-5-Data.db')]]
DEBUG 17:58:13,434 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=93668), column family 7ad54392-bcdd-35a6-8417-4e047860b377
DEBUG 17:58:13,440 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:13,443 No tasks available
DEBUG 17:58:13,453 Setting tokens to [-6323000812485711377, 7540920246740048508, 8343754522398236724, 8408024349763410671, 2958176648006585896, -4558504792860463383, 6505045068820279515, -2723585437376250373, 6295277530571293136, -632036933490110017, 7076829522639601545, -972040818133908807, -8032509273357338597, -8195415716479930448, 2815056297233670800, -8514945661303914275, -1404986336850055825, 1902287316263855857, 2271127848797136099, 2253027225668615491, -4596682821656889301, -3829603337268068014, 6732451001921723049, -7525322146388049675, -7520469144418445567, -42294142306824200, -3583505036108061780, 2037651260804506174, -8857382013269212764, -287803492046061484, -2387955156962752445, 5851040736860755910, -2566990238894069943, -4862528385953509677, -6934815927114679513, 436339434870862426, -9024405707727859684, -5712896424965939691, -7306151990958768726, 1972562868127181045, 4322111133230504092, 3274187527528688964, -170723646511099548, -5215516480259793274, -5433009735911985650, 1611629741105319295, -1003574501709202508, -342125947698355485, -6982367747787992083, -1265530805719316699, -3787322892048394796, 4335532059375005778, -4955726035129474058, -3371084287244267978, -1356714364316547864, 2509890377680956235, -1688681099182070115, 4105757083923252042, 6251650102550794764, 6814340446137923947, 2350550282350068760, -5827526901688128468, 6417310693620297152, -3072998771907077372, -1401160349313134159, 5689378233308186158, -4579053362657493044, -1974282572872555977, 6030416333994244776, -6459751443485456668, -7053683753232252424, 3313304008102967685, -3323360202881321267, 1777100483859232105, 644078824255348160, 9174385969124367717, -3152648150979214720, -5810274955975592409, -2417697781941443512, 2097979287894624485, 1306279112226821961, -5471073413421323422, -899964525929809433, 5721993461796585132, 6287709076354945844, -2618973873461192043, -4901233632190356360, 2810307178808259886, 414264499406348099, 9204881637988352346, 2756156004752906240, 4264704223664142541, 6966682145360577932, 456907978019976961, 3526133307919328434, 6663482402885505474, 1208521381475842799, 3709424135702812997, 2664447310834129793, 2184366192501394990, -9184383021035672777, 1764656501925028031, -8960542797122524994, 5425128251068131909, -5995018354184249099, 7807241096173494918, -4950974144156281148, 5174743236438194862, 1244619037182110640, -2546311664945762019, -5553188653211189061, 8804167436403071449, -3529889955411356312, 6118046371500892857, 3320865090004456676, -6903778681629605976, 6386743816636437221, -9071719352823462138, -6576758648871647588, 5987037537853395238, 986125783873493733, 2560513262779074155, 2664163134953574068, -4785699651647692687, -2423779828439331130, 3204264553240839859, -3032730799546050734, -1119983597542134724, 8581101667680406496, -5853780338109789104, 6618776578435553577, 8127528839792128464, 3981688355788388288, -3276943142712007267, -5553559033592503934, -565586245499042749, 2698146986917107680, -4118747694524450398, -94197050448104434, -7041621829760205318, -1204017316777816708, 3229847261799003880, -6279870462178797449, -990289430614639535, 8443909287132310249, -4274511997658117637, 3785578998658569690, -6359618184454198961, 2665969438726830287, 8054750200135920151, -3934821671677811317, -485635553910904013, -623380094895579520, -3472488394961304835, -7526106638135734955, 1890693942329731243, 5244101280950037554, 4583546775549106643, 3895567705889066228, 5334628314720458147, -8596631607607319391, 3740554219906914028, 9095268733721810466, 2758206461555504574, -6162081928909388617, 8708338036502321790, 5183092910492165645, 1829498305562316123, -2572585900623957771, -8260878088704282035, 7734738546550767507, 1832837517438053898, 4324222608062794450, -474214797246154002, -6069598122091042081, -9142441383540253639, 2815706000764693552, 5967442704636348366, 2087292969799552844, 5828375876428174510, 6379276420430849033, -8271396753288625388, 7780395417475412741, -1390444458813154553, -5453290611375819442, 6083896792337057679, -4257643687424988922, 770735604922246946, 2216654002368848583, -7748279779200518330, 1944728945767071214, -2294194973999153845, -3603821754103775284, 4721521319267790087, 5372973717379594525, -8275676737706078573, 5897461205541082929, -695930662834055090, 2719486353185233220, -392331656470549574, -1568658082513590058, -5648478850489989928, -4812696232891045407, 2401615522584549554, -6464420048640302972, -5046677986523856430, 780995344937361068, -1233002335258848816, -6735843204069900975, 8044399031264580270, -4871656601660382401, 8297269945276814669, -1666262333518459589, 1442708638138645005, -101325079897371283, -3146272811424619794, -2142311001363252112, 3924382339535466270, -6761931011837249238, 6504024115050419271, -3429178218050542445, 4270740334962704112, -385458615439681688, 4381980869381678991, -953904416719284833, 8298012175524755475, 7844241744998944762, -9084311131189837541, 8001607762992085251, -7252657913919874500, -4710023932873815473, -8312370105536886283, 591824176525945018, -3603135508858098035, -8594324420901678749, -7624329750282992340, 3109508350842625585, 3086473628596783752, -7805741803336097762, 3652336128698228111, 9151260251441783444, -2571481641165315710, -3983064571066933508, -4414430870012910693, 7041889069507587063, -3533275297825790658, 2274185594739201969, -6219740351350975938, 5603158837900577920, 8831369449751982065, 1992022020144601219, -3286196597800579880, 2977787458295061172, -8243451741655531306, 1066786986080576383, -1069941784245844754]
INFO 17:58:13,783 Enqueuing flush of Memtable-local@15259753(10098/10098 serialized/live bytes, 257 ops)
INFO 17:58:13,789 Writing Memtable-local@15259753(10098/10098 serialized/live bytes, 257 ops)
DEBUG 17:58:14,835 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-CompressionInfo.db to /var/lib/cassandra/data/system/local/system-local-ja-7-CompressionInfo.db
DEBUG 17:58:14,841 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-Statistics.db to /var/lib/cassandra/data/system/local/system-local-ja-7-Statistics.db
DEBUG 17:58:14,846 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-Filter.db to /var/lib/cassandra/data/system/local/system-local-ja-7-Filter.db
DEBUG 17:58:14,853 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-Index.db to /var/lib/cassandra/data/system/local/system-local-ja-7-Index.db
DEBUG 17:58:14,859 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-TOC.txt to /var/lib/cassandra/data/system/local/system-local-ja-7-TOC.txt
DEBUG 17:58:14,864 Renaming /var/lib/cassandra/data/system/local/system-local-tmp-ja-7-Data.db to /var/lib/cassandra/data/system/local/system-local-ja-7-Data.db
INFO 17:58:14,876 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-7-Data.db (5264 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=105667)
DEBUG 17:58:14,881 adding /var/lib/cassandra/data/system/local/system-local-ja-7 to list of files tracked for system.local
DEBUG 17:58:14,887 Scheduling a background task check for system.local with SizeTieredCompactionStrategy
DEBUG 17:58:14,890 Checking system.local
DEBUG 17:58:14,894 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=105667), column family 7ad54392-bcdd-35a6-8417-4e047860b377
DEBUG 17:58:14,897 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:14,909 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-6-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-7-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-5-Data.db')]]
DEBUG 17:58:14,923 No tasks available
DEBUG 17:58:14,953 clearing cached endpoints
DEBUG 17:58:14,957 clearing cached endpoints
DEBUG 17:58:15,834 Node /192.168.0.11 state normal, token [-1003574501709202508, -101325079897371283, -1069941784245844754, -1119983597542134724, -1204017316777816708, -1233002335258848816, -1265530805719316699, -1356714364316547864, -1390444458813154553, -1401160349313134159, -1404986336850055825, -1568658082513590058, -1666262333518459589, -1688681099182070115, -170723646511099548, -1974282572872555977, -2142311001363252112, -2294194973999153845, -2387955156962752445, -2417697781941443512, -2423779828439331130, -2546311664945762019, -2566990238894069943, -2571481641165315710, -2572585900623957771, -2618973873461192043, -2723585437376250373, -287803492046061484, -3032730799546050734, -3072998771907077372, -3146272811424619794, -3152648150979214720, -3276943142712007267, -3286196597800579880, -3323360202881321267, -3371084287244267978, -342125947698355485, -3429178218050542445, -3472488394961304835, -3529889955411356312, -3533275297825790658, -3583505036108061780, -3603135508858098035, -3603821754103775284, -3787322892048394796, -3829603337268068014, -385458615439681688, -392331656470549574, -3934821671677811317, -3983064571066933508, -4118747694524450398, -42294142306824200, -4257643687424988922, -4274511997658117637, -4414430870012910693, -4558504792860463383, -4579053362657493044, -4596682821656889301, -4710023932873815473, -474214797246154002, -4785699651647692687, -4812696232891045407, -485635553910904013, -4862528385953509677, -4871656601660382401, -4901233632190356360, -4950974144156281148, -4955726035129474058, -5046677986523856430, -5215516480259793274, -5433009735911985650, -5453290611375819442, -5471073413421323422, -5553188653211189061, -5553559033592503934, -5648478850489989928, -565586245499042749, -5712896424965939691, -5810274955975592409, -5827526901688128468, -5853780338109789104, -5995018354184249099, -6069598122091042081, -6162081928909388617, -6219740351350975938, -623380094895579520, -6279870462178797449, -632036933490110017, -6323000812485711377, -6359618184454198961, -6459751443485456668, -6464420048640302972, -6576758648871647588, -6735843204069900975, -6761931011837249238, -6903778681629605976, -6934815927114679513, -695930662834055090, -6982367747787992083, -7041621829760205318, -7053683753232252424, -7252657913919874500, -7306151990958768726, -7520469144418445567, -7525322146388049675, -7526106638135734955, -7624329750282992340, -7748279779200518330, -7805741803336097762, -8032509273357338597, -8195415716479930448, -8243451741655531306, -8260878088704282035, -8271396753288625388, -8275676737706078573, -8312370105536886283, -8514945661303914275, -8594324420901678749, -8596631607607319391, -8857382013269212764, -8960542797122524994, -899964525929809433, -9024405707727859684, -9071719352823462138, -9084311131189837541, -9142441383540253639, -9184383021035672777, -94197050448104434, -953904416719284833, -972040818133908807, -990289430614639535, 1066786986080576383, 1208521381475842799, 1244619037182110640, 1306279112226821961, 1442708638138645005, 1611629741105319295, 1764656501925028031, 1777100483859232105, 1829498305562316123, 1832837517438053898, 1890693942329731243, 1902287316263855857, 1944728945767071214, 1972562868127181045, 1992022020144601219, 2037651260804506174, 2087292969799552844, 2097979287894624485, 2184366192501394990, 2216654002368848583, 2253027225668615491, 2271127848797136099, 2274185594739201969, 2350550282350068760, 2401615522584549554, 2509890377680956235, 2560513262779074155, 2664163134953574068, 2664447310834129793, 2665969438726830287, 2698146986917107680, 2719486353185233220, 2756156004752906240, 2758206461555504574, 2810307178808259886, 2815056297233670800, 2815706000764693552, 2958176648006585896, 2977787458295061172, 3086473628596783752, 3109508350842625585, 3204264553240839859, 3229847261799003880, 3274187527528688964, 3313304008102967685, 3320865090004456676, 3526133307919328434, 3652336128698228111, 3709424135702812997, 3740554219906914028, 3785578998658569690, 3895567705889066228, 3924382339535466270, 3981688355788388288, 4105757083923252042, 414264499406348099, 4264704223664142541, 4270740334962704112, 4322111133230504092, 4324222608062794450, 4335532059375005778, 436339434870862426, 4381980869381678991, 456907978019976961, 4583546775549106643, 4721521319267790087, 5174743236438194862, 5183092910492165645, 5244101280950037554, 5334628314720458147, 5372973717379594525, 5425128251068131909, 5603158837900577920, 5689378233308186158, 5721993461796585132, 5828375876428174510, 5851040736860755910, 5897461205541082929, 591824176525945018, 5967442704636348366, 5987037537853395238, 6030416333994244776, 6083896792337057679, 6118046371500892857, 6251650102550794764, 6287709076354945844, 6295277530571293136, 6379276420430849033, 6386743816636437221, 6417310693620297152, 644078824255348160, 6504024115050419271, 6505045068820279515, 6618776578435553577, 6663482402885505474, 6732451001921723049, 6814340446137923947, 6966682145360577932, 7041889069507587063, 7076829522639601545, 7540920246740048508, 770735604922246946, 7734738546550767507, 7780395417475412741, 7807241096173494918, 780995344937361068, 7844241744998944762, 8001607762992085251, 8044399031264580270, 8054750200135920151, 8127528839792128464, 8297269945276814669, 8298012175524755475, 8343754522398236724, 8408024349763410671, 8443909287132310249, 8581101667680406496, 8708338036502321790, 8804167436403071449, 8831369449751982065, 9095268733721810466, 9151260251441783444, 9174385969124367717, 9204881637988352346, 986125783873493733]
INFO 17:58:15,841 Node /192.168.0.11 state jump to normal
DEBUG 17:58:15,893 clearing cached endpoints
DEBUG 17:58:15,898 clearing cached endpoints
DEBUG 17:58:15,932 NORMAL
INFO 17:58:15,936 Startup completed! Now serving reads.
INFO 17:58:15,963 Create new Keyspace: system_auth, rep strategy:SimpleStrategy{}, strategy_options: {replication_factor=1}, durable_writes: true
INFO 17:58:16,106 Enqueuing flush of Memtable-schema_keyspaces@7950298(389/389 serialized/live bytes, 11 ops)
INFO 17:58:16,124 Writing Memtable-schema_keyspaces@7950298(389/389 serialized/live bytes, 11 ops)
DEBUG 17:58:16,235 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-CompressionInfo.db to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-CompressionInfo.db
DEBUG 17:58:16,240 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-Statistics.db to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Statistics.db
DEBUG 17:58:16,246 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-Filter.db to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Filter.db
DEBUG 17:58:16,253 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-Index.db to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Index.db
DEBUG 17:58:16,259 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-TOC.txt to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-TOC.txt
DEBUG 17:58:16,264 Renaming /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-tmp-ja-1-Data.db to /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Data.db
INFO 17:58:16,276 Completed flushing /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Data.db (268 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=105900)
DEBUG 17:58:16,277 Target max version is -2147483648; no version information yet, will retry
DEBUG 17:58:16,286 adding /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1 to list of files tracked for system.schema_keyspaces
INFO 17:58:16,288 Cannot handshake version with /192.168.0.10
DEBUG 17:58:16,302 Scheduling a background task check for system.schema_keyspaces with SizeTieredCompactionStrategy
INFO 17:58:16,305 Handshaking version with /192.168.0.10
DEBUG 17:58:16,311 Checking system.schema_keyspaces
DEBUG 17:58:16,316 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-1-Data.db')]]
DEBUG 17:58:16,319 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=105900), column family b0f22357-4458-3cdb-9631-c43e59ce3676
DEBUG 17:58:16,335 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:16,336 No tasks available
INFO 17:58:16,343 Enqueuing flush of Memtable-schema_columnfamilies@24893745(25088/25088 serialized/live bytes, 402 ops)
INFO 17:58:16,352 Writing Memtable-schema_columnfamilies@24893745(25088/25088 serialized/live bytes, 402 ops)
DEBUG 17:58:16,658 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-CompressionInfo.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-CompressionInfo.db
DEBUG 17:58:16,664 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-Statistics.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Statistics.db
DEBUG 17:58:16,669 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-Filter.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Filter.db
DEBUG 17:58:16,674 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-Index.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Index.db
DEBUG 17:58:16,680 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-TOC.txt to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-TOC.txt
DEBUG 17:58:16,686 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-1-Data.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Data.db
INFO 17:58:16,698 Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Data.db (5140 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=105900)
DEBUG 17:58:16,703 adding /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1 to list of files tracked for system.schema_columnfamilies
DEBUG 17:58:16,709 Scheduling a background task check for system.schema_columnfamilies with SizeTieredCompactionStrategy
DEBUG 17:58:16,714 Checking system.schema_columnfamilies
DEBUG 17:58:16,717 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Data.db')]]
DEBUG 17:58:16,720 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=105900), column family 45f5b360-24bc-3f83-a363-1034ea4fa697
DEBUG 17:58:16,727 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:16,728 No tasks available
INFO 17:58:16,736 Enqueuing flush of Memtable-schema_columns@9968064(46022/46022 serialized/live bytes, 765 ops)
INFO 17:58:16,745 Writing Memtable-schema_columns@9968064(46022/46022 serialized/live bytes, 765 ops)
DEBUG 17:58:17,635 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-CompressionInfo.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-CompressionInfo.db
DEBUG 17:58:17,655 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-Statistics.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Statistics.db
DEBUG 17:58:17,660 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-Filter.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Filter.db
DEBUG 17:58:17,667 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-Index.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Index.db
DEBUG 17:58:17,673 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-TOC.txt to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-TOC.txt
DEBUG 17:58:17,678 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-1-Data.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Data.db
INFO 17:58:17,690 Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Data.db (9178 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=105900)
DEBUG 17:58:17,695 adding /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1 to list of files tracked for system.schema_columns
DEBUG 17:58:17,700 Scheduling a background task check for system.schema_columns with SizeTieredCompactionStrategy
DEBUG 17:58:17,704 Checking system.schema_columns
DEBUG 17:58:17,709 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Data.db')]]
DEBUG 17:58:17,711 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=105900), column family 296e9c04-9bec-3085-827d-c17d3df2122a
DEBUG 17:58:17,718 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:17,720 No tasks available
DEBUG 17:58:18,067 Gossiping my schema version 3647ed87-133a-32f1-86e9-c6392aba2d88
DEBUG 17:58:18,316 GC for ParNew: 119 ms for 1 collections, 18127656 used; max is 118620160
INFO 17:58:18,924 Create new ColumnFamily: org.apache.cassandra.config.CFMetaData@158c08[cfId=473588ad-9c79-38be-8b59-e06c10456ba0,ksName=system_auth,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,replicateOnWrite=true,gcGraceSeconds=7776000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=7375706572, validator=org.apache.cassandra.db.marshal.BooleanType, index_type=null, index_name='null', component_index=0, type=REGULAR}, java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]=ColumnDefinition{name=6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null', type=PARTITION_KEY}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtable_flush_period_in_ms=0,caching=KEYS_ONLY,defaultTimeToLive=0,speculative_retry=NONE,indexInterval=128,populateIoCacheOnFlush=false,droppedColumns={},triggers={}]
DEBUG 17:58:19,043 forceFlush requested but everything is clean in schema_keyspaces
INFO 17:58:19,058 Enqueuing flush of Memtable-schema_columnfamilies@25052400(1283/1283 serialized/live bytes, 25 ops)
INFO 17:58:19,069 Writing Memtable-schema_columnfamilies@25052400(1283/1283 serialized/live bytes, 25 ops)
DEBUG 17:58:19,568 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-CompressionInfo.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-CompressionInfo.db
DEBUG 17:58:19,573 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-Statistics.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Statistics.db
DEBUG 17:58:19,578 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-Filter.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Filter.db
DEBUG 17:58:19,583 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-Index.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Index.db
DEBUG 17:58:19,588 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-TOC.txt to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-TOC.txt
DEBUG 17:58:19,593 Renaming /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-tmp-ja-2-Data.db to /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Data.db
INFO 17:58:19,606 Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Data.db (789 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=108329)
DEBUG 17:58:19,611 adding /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2 to list of files tracked for system.schema_columnfamilies
DEBUG 17:58:19,624 Scheduling a background task check for system.schema_columnfamilies with SizeTieredCompactionStrategy
DEBUG 17:58:19,627 Checking system.schema_columnfamilies
DEBUG 17:58:19,632 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-2-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-1-Data.db')]]
DEBUG 17:58:19,635 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=108329), column family 45f5b360-24bc-3f83-a363-1034ea4fa697
DEBUG 17:58:19,642 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
INFO 17:58:19,649 Enqueuing flush of Memtable-schema_columns@261549(628/628 serialized/live bytes, 14 ops)
DEBUG 17:58:19,644 No tasks available
INFO 17:58:19,656 Writing Memtable-schema_columns@261549(628/628 serialized/live bytes, 14 ops)
DEBUG 17:58:19,765 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-CompressionInfo.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-CompressionInfo.db
DEBUG 17:58:19,770 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-Statistics.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Statistics.db
DEBUG 17:58:19,775 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-Filter.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Filter.db
DEBUG 17:58:19,780 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-Index.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Index.db
DEBUG 17:58:19,786 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-TOC.txt to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-TOC.txt
DEBUG 17:58:19,791 Renaming /var/lib/cassandra/data/system/schema_columns/system-schema_columns-tmp-ja-2-Data.db to /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Data.db
INFO 17:58:19,803 Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Data.db (319 bytes) for commitlog position ReplayPosition(segmentId=1373911018903, position=108329)
DEBUG 17:58:19,809 adding /var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2 to list of files tracked for system.schema_columns
DEBUG 17:58:19,814 Scheduling a background task check for system.schema_columns with SizeTieredCompactionStrategy
DEBUG 17:58:19,818 Checking system.schema_columns
DEBUG 17:58:19,823 Compaction buckets are [[SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-2-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ja-1-Data.db')]]
DEBUG 17:58:19,825 discard completed log segments for ReplayPosition(segmentId=1373911018903, position=108329), column family 296e9c04-9bec-3085-827d-c17d3df2122a
DEBUG 17:58:19,832 Not deleting active commitlog segment CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-3-1373911018903.log)
DEBUG 17:58:19,834 No tasks available
DEBUG 17:58:20,094 Gossiping my schema version 1dfa6b7c-d627-3308-81b3-1a977d961c38
INFO 17:58:20,593 Loading org.apache.cassandra.config.CFMetaData@1b659f[cfId=473588ad-9c79-38be-8b59-e06c10456ba0,ksName=system_auth,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,replicateOnWrite=true,gcGraceSeconds=7776000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=7375706572, validator=org.apache.cassandra.db.marshal.BooleanType, index_type=null, index_name='null', component_index=0, type=REGULAR}, java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]=ColumnDefinition{name=6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null', type=PARTITION_KEY}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtable_flush_period_in_ms=0,caching=KEYS_ONLY,defaultTimeToLive=0,speculative_retry=NONE,indexInterval=128,populateIoCacheOnFlush=false,droppedColumns={},triggers={}]
DEBUG 17:58:20,603 Adding org.apache.cassandra.config.CFMetaData@1b659f[cfId=473588ad-9c79-38be-8b59-e06c10456ba0,ksName=system_auth,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,replicateOnWrite=true,gcGraceSeconds=7776000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=7375706572, validator=org.apache.cassandra.db.marshal.BooleanType, index_type=null, index_name='null', component_index=0, type=REGULAR}, java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]=ColumnDefinition{name=6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null', type=PARTITION_KEY}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtable_flush_period_in_ms=0,caching=KEYS_ONLY,defaultTimeToLive=0,speculative_retry=NONE,indexInterval=128,populateIoCacheOnFlush=false,droppedColumns={},triggers={}] to cfIdMap
DEBUG 17:58:20,644 Starting CFS users
DEBUG 17:58:20,670 completed reading (0 ms; 0 keys) saved cache /var/lib/cassandra/saved_caches/system_auth-users-KeyCache-b.db
DEBUG 17:58:20,732 Will try to load mx4j now, if it's in the classpath
DEBUG 17:58:20,745 Will not load MX4J, mx4j-tools.jar is not in the classpath
The University of Dundee is a registered Scottish Charity, No: SC015096
Re: V2 Beta 1 bootstrap behaviour ?
Posted by Andrew Cobley <a....@dundee.ac.uk>.
Righto !
Will do a JIRA report..
Many thanks
Andy
On 16 Jul 2013, at 18:50, Robert Coli <rc...@eventbrite.com>>
wrote:
On Tue, Jul 16, 2013 at 1:16 AM, Andrew Cobley <a....@dundee.ac.uk>> wrote:
I'm setting up a new test cluster using 2.0.0-beta1 and I noticed the following behaviour with vnodes turned on. I bring up one node all well and good. however if I bring up a second node, that can't contact the first (the first being the seed for the second) after a short period of time, the second goes ahead and assumes it's the only node and bootstraps with all tokens. Is this correct behaviour? I'd have thought it would halt with a "Can't contact seed" message to avoid the node starting in a possibly mis-confgured state ?
It should do this, but currently does not. Obviously if you have defined a seed and cannot contact it, the node should not start as a cluster of one. I have a to-do list item to file a JIRA on the subject, but if you wanted to file and link us, that'd be super. :)
=Rob
The University of Dundee is a registered Scottish Charity, No: SC015096
Re: V2 Beta 1 bootstrap behaviour ?
Posted by Robert Coli <rc...@eventbrite.com>.
On Tue, Jul 16, 2013 at 1:16 AM, Andrew Cobley <a....@dundee.ac.uk>wrote:
> I'm setting up a new test cluster using 2.0.0-beta1 and I noticed the
> following behaviour with vnodes turned on. I bring up one node all well
> and good. however if I bring up a second node, that can't contact the
> first (the first being the seed for the second) after a short period of
> time, the second goes ahead and assumes it's the only node and bootstraps
> with all tokens. Is this correct behaviour? I'd have thought it would
> halt with a "Can't contact seed" message to avoid the node starting in a
> possibly mis-confgured state ?
>
It should do this, but currently does not. Obviously if you have defined a
seed and cannot contact it, the node should not start as a cluster of one.
I have a to-do list item to file a JIRA on the subject, but if you wanted
to file and link us, that'd be super. :)
=Rob