You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by qf zhou <zh...@gmail.com> on 2017/08/31 09:52:28 UTC
Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError:
Direct buffer memory
I am running a cluster with 3 nodes. Then it occurs the following errors. After I restart one node, the error happens again.
I don’t know why. Who Can help me ? Thank you !!
ERROR [ReadStage-31] 2017-08-31 15:08:20,878 JVMStabilityInspector.java:141 - JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693) ~[na:1.8.0_111]
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) ~[na:1.8.0_111]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[na:1.8.0_111]
at org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:109) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:46) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool$LocalPool.allocate(BufferPool.java:406) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:333) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.takeFromPool(BufferPool.java:121) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:93) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:156) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:40) ~[apache-cassandra-3.9.jar:3.9]
at com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807) ~[caffeine-2.2.6.jar:na]
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1892) ~[na:1.8.0_111]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66) ~[caffeine-2.2.6.jar:na]
at org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:78) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:220) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.SegmentedFile.createReader(SegmentedFile.java:138) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1709) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(AbstractSSTableIterator.java:102) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:49) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.initializeIterator(UnfilteredRowIteratorWithLowerBound.java:105) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.partitionLevelDeletion(LazilyInitializedUnfilteredRowIterator.java:81) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.partitionLevelDeletion(UnfilteredRowIteratorWithLowerBound.java:165) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:585) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:492) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:397) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) ~[apache-cassandra-3.9.jar:3.9]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_111]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.9.jar:3.9]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError:
Direct buffer memory
Posted by John Sanda <jo...@gmail.com>.
I am not sure which version of Netty is in 3.9, but maybe you are hitting
https://issues.apache.org/jira/browse/CASSANDRA-13114. I hit this in
Cassandra 3.0.9 which uses Netty 4.0.23. Here is the upstream netty ticket
https://github.com/netty/netty/issues/3057.
On Thu, Aug 31, 2017 at 10:15 AM, Chris Lohfink <cl...@gmail.com>
wrote:
> What version of java are you running? There is a "kinda leak" in jvm
> around this you may run into, can try with -Djdk.nio.maxCachedBufferSize=
> 262144 if above 8u102. You can also try increasing the size allowed for
> direct byte buffers. It defaults to size of heap -XX:MaxDirectMemorySize=?
> G
>
> Some NIO channel operations use temporary DirectByteBuffers which are
>> cached in thread-local caches to avoid having to allocate / free a buffer
>> at every operation.
>> Unfortunately, there is no bound imposed on the size of buffers added to
>> the thread-local caches. So, infrequent channel operations that require a
>> very large buffer can create a native memory leak.
>
>
>
>> *Ability to limit the capacity of buffers that can be held in the
>> temporary buffer cache*The system property jdk.nio.maxCachedBufferSize has
>> been introduced in 8u102 to limit the memory used by the "temporary buffer
>> cache." The temporary buffer cache is a per-thread cache of direct memory
>> used by the NIO implementation to support applications that do I/O with
>> buffers backed by arrays in the java heap. The value of the property is the
>> maximum capacity of a direct buffer that can be cached. If the property is
>> not set, then no limit is put on the size of buffers that are cached.
>> Applications with certain patterns of I/O usage may benefit from using this
>> property. In particular, an application that does I/O with large
>> multi-megabyte buffers at startup but does I/O with small buffers may see a
>> benefit to using this property. Applications that do I/O using direct
>> buffers will not see any benefit to using this system property.
>> See JDK-8147468 <http://bugs.openjdk.java.net/browse/JDK-8147468>
>
>
> Chris
>
> On Thu, Aug 31, 2017 at 4:59 AM, Jonathan Baynes <
> Jonathan.Baynes@tradeweb.com> wrote:
>
>> I wonder if its related to this bug (below) that’s currently unresolved,
>> albeit it being reproduced way back in 2.1.11
>>
>>
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-10689
>>
>>
>>
>>
>>
>> *From:* qf zhou [mailto:zhouqf2013@gmail.com]
>> *Sent:* 31 August 2017 10:58
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: Cassandra - Nodes can't restart due to
>> java.lang.OutOfMemoryError: Direct buffer memory
>>
>>
>>
>> I am using Cassandra 3.9 with cqlsh 5.0.1.
>>
>> 在 2017年8月31日，下午5:54，Jonathan Baynes <Jo...@tradeweb.com> 写道：
>>
>>
>>
>> again
>>
>>
>>
>> ________________________________________________________________________
>>
>> This e-mail may contain confidential and/or privileged information. If
>> you are not the intended recipient (or have received this e-mail in error)
>> please notify the sender immediately and destroy it. Any unauthorized
>> copying, disclosure or distribution of the material in this e-mail is
>> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
>> communications through its networks. If you do not wish to receive
>> marketing emails about our products / services, please let us know by
>> contacting us, either by email at contactus@tradeweb.com or by writing
>> to us at the registered office of Tradeweb in the UK, which is: Tradeweb
>> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
>> 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>>
>
>
--
- John
Re: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError:
Direct buffer memory
Posted by Chris Lohfink <cl...@gmail.com>.
What version of java are you running? There is a "kinda leak" in jvm around
this you may run into, can try with -Djdk.nio.maxCachedBufferSize=262144 if
above 8u102. You can also try increasing the size allowed for direct byte
buffers. It defaults to size of heap -XX:MaxDirectMemorySize=?G
Some NIO channel operations use temporary DirectByteBuffers which are
> cached in thread-local caches to avoid having to allocate / free a buffer
> at every operation.
> Unfortunately, there is no bound imposed on the size of buffers added to
> the thread-local caches. So, infrequent channel operations that require a
> very large buffer can create a native memory leak.
> *Ability to limit the capacity of buffers that can be held in the
> temporary buffer cache*The system property jdk.nio.maxCachedBufferSize has
> been introduced in 8u102 to limit the memory used by the "temporary buffer
> cache." The temporary buffer cache is a per-thread cache of direct memory
> used by the NIO implementation to support applications that do I/O with
> buffers backed by arrays in the java heap. The value of the property is the
> maximum capacity of a direct buffer that can be cached. If the property is
> not set, then no limit is put on the size of buffers that are cached.
> Applications with certain patterns of I/O usage may benefit from using this
> property. In particular, an application that does I/O with large
> multi-megabyte buffers at startup but does I/O with small buffers may see a
> benefit to using this property. Applications that do I/O using direct
> buffers will not see any benefit to using this system property.
> See JDK-8147468 <http://bugs.openjdk.java.net/browse/JDK-8147468>
Chris
On Thu, Aug 31, 2017 at 4:59 AM, Jonathan Baynes <
Jonathan.Baynes@tradeweb.com> wrote:
> I wonder if its related to this bug (below) that’s currently unresolved,
> albeit it being reproduced way back in 2.1.11
>
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-10689
>
>
>
>
>
> *From:* qf zhou [mailto:zhouqf2013@gmail.com]
> *Sent:* 31 August 2017 10:58
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra - Nodes can't restart due to
> java.lang.OutOfMemoryError: Direct buffer memory
>
>
>
> I am using Cassandra 3.9 with cqlsh 5.0.1.
>
> 在 2017年8月31日，下午5:54，Jonathan Baynes <Jo...@tradeweb.com> 写道：
>
>
>
> again
>
>
>
> ________________________________________________________________________
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contactus@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>
RE: Cassandra - Nodes can't restart due to
java.lang.OutOfMemoryError: Direct buffer memory
Posted by Jonathan Baynes <Jo...@tradeweb.com>.
I wonder if its related to this bug (below) that’s currently unresolved, albeit it being reproduced way back in 2.1.11
https://issues.apache.org/jira/browse/CASSANDRA-10689
From: qf zhou [mailto:zhouqf2013@gmail.com]
Sent: 31 August 2017 10:58
To: user@cassandra.apache.org
Subject: Re: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory
I am using Cassandra 3.9 with cqlsh 5.0.1.
在 2017年8月31日，下午5:54，Jonathan Baynes <Jo...@tradeweb.com>> 写道：
again
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
Re: Cassandra - Nodes can't restart due to
java.lang.OutOfMemoryError: Direct buffer memory
Posted by qf zhou <zh...@gmail.com>.
I am using Cassandra 3.9 with cqlsh 5.0.1.
> 在 2017年8月31日，下午5:54，Jonathan Baynes <Jo...@tradeweb.com> 写道：
>
> again
RE: Cassandra - Nodes can't restart due to
java.lang.OutOfMemoryError: Direct buffer memory
Posted by Jonathan Baynes <Jo...@tradeweb.com>.
Can you tell us what version you are on?
-----Original Message-----
From: qf zhou [mailto:zhouqf2013@gmail.com]
Sent: 31 August 2017 10:52
To: user@cassandra.apache.org
Subject: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory
I am running a cluster with 3 nodes. Then it occurs the following errors. After I restart one node, the error happens again.
I don't know why. Who Can help me ? Thank you !!
ERROR [ReadStage-31] 2017-08-31 15:08:20,878 JVMStabilityInspector.java:141 - JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693) ~[na:1.8.0_111]
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) ~[na:1.8.0_111]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[na:1.8.0_111]
at org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:109) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:46) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool$LocalPool.allocate(BufferPool.java:406) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:333) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.takeFromPool(BufferPool.java:121) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:93) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:156) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:40) ~[apache-cassandra-3.9.jar:3.9]
at com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807) ~[caffeine-2.2.6.jar:na]
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1892) ~[na:1.8.0_111]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97) ~[caffeine-2.2.6.jar:na]
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66) ~[caffeine-2.2.6.jar:na]
at org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:236) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:214) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:78) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:220) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.util.SegmentedFile.createReader(SegmentedFile.java:138) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1709) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(AbstractSSTableIterator.java:102) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:49) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.initializeIterator(UnfilteredRowIteratorWithLowerBound.java:105) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.partitionLevelDeletion(LazilyInitializedUnfilteredRowIterator.java:81) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.partitionLevelDeletion(UnfilteredRowIteratorWithLowerBound.java:165) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:585) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:492) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:397) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) ~[apache-cassandra-3.9.jar:3.9]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_111]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.9.jar:3.9]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
________________________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy it. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Tradeweb reserves the right to monitor all e-mail communications through its networks. If you do not wish to receive marketing emails about our products / services, please let us know by contacting us, either by email at contactus@tradeweb.com or by writing to us at the registered office of Tradeweb in the UK, which is: Tradeweb Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org