You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Simon Chemouil <sc...@gmail.com> on 2014/06/20 10:18:58 UTC
Bug on 2.1-rc1 with BLOBs?
Hi,
When I am sending BLOBs _below_ the max query size (blob size=0.6MB), on
Cassandra 2.0, it works fine, but on 2.1-rc1 I get the following error
within the Cassandra server (from the logs) and the query just dies:
WARN [SharedPool-Worker-2] 2014-06-20 10:06:00,263
AbstractTracingAwareExecutorService.java:166 - Uncaught exception on
thread Thread[SharedPool-Worker-2,5,main]: {}
java.lang.RuntimeException: java.lang.IllegalArgumentException: Mutation
of 122880122 bytes is too large for the maxiumum size of 16777216
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2052)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_05]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:162)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at
org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103)
[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
Caused by: java.lang.IllegalArgumentException: Mutation of 122880122
bytes is too large for the maxiumum size of 16777216
at
org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:205)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at
org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:192)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:354)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at
org.apache.cassandra.service.StorageProxy$7.runMayThrow(StorageProxy.java:958)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2048)
~[apache-cassandra-2.1.0-rc1.jar:2.1.0-rc1]
... 4 common frames omitted
I checked on JIRA to see something similar but didn't find (then again,
not sure which keywords to search for). Should I open an issue?
Cheers,
Simon
Re: Bug on 2.1-rc1 with BLOBs?
Posted by DuyHai Doan <do...@gmail.com>.
Thanks Simon for the info. I didn't know that the maximum payload size is
related to commit log config, interesting ...
On Fri, Jun 20, 2014 at 11:39 AM, Simon Chemouil <sc...@gmail.com>
wrote:
> OK, so Cassandra 2.1 now rejects writes it considers too big. It is
> possible to increase the value by changing commitlog_segment_size_in_mb
> in cassandra.yaml. It defaults to 32MB, and the maximum segment size for
> a write is half that value:
>
> from CommitLog.java:
> // we only permit records HALF the size of a commit log, to ensure we
> don't spin allocating many mostly
> // empty segments when writing large records
> private static final long MAX_MUTATION_SIZE =
> DatabaseDescriptor.getCommitLogSegmentSize() >> 1;
>
>
> which explains (with the request overhead) why my ~30,5MB blob was
> rejected.
>
> Simon
>
> Le 20/06/2014 11:24, Simon Chemouil a écrit :
> > For the record, I could reproduce the problem with blobs of size below
> 64MB.
> >
> > Caused by: java.lang.IllegalArgumentException: Mutation of 32000122
> > bytes is too large for the maxiumum size of 16777216
> >
> > 32000122 is just ~30MB and fails on 2.1-rc1 while it works on 2.0.X for
> > even larger values (up to 64MB works fine)
> >
> > Simon
> >
> > Le 20/06/2014 11:00, Simon Chemouil a écrit :
> >> Le 20/06/2014 10:41, Duncan Sands a écrit :
> >>> Hi Simon,
> >>> 122880122 bytes is a lot more than 0.6MB... How are you sending your
> blob?
> >>
> >> Turns out there was a mistake in my code. The blob in this case was
> >> actually 122MB!
> >> Still the same code works fine on Cassandra 2.0.x so there might be a
> >> bug lurking. Even if it's definitely above the recommended limit.
> >>
> >> Simon
> >>
>
Re: Bug on 2.1-rc1 with BLOBs?
Posted by Robert Coli <rc...@eventbrite.com>.
On Fri, Jun 20, 2014 at 2:39 AM, Simon Chemouil <sc...@gmail.com> wrote:
> OK, so Cassandra 2.1 now rejects writes it considers too big. It is
> possible to increase the value by changing commitlog_segment_size_in_mb
> in cassandra.yaml. It defaults to 32MB, and the maximum segment size for
> a write is half that value:
>
The previous behavior, IIRC, was to just not commitlog the gigantic
thing... so that's probably a good change. :)
=Rob
Re: Bug on 2.1-rc1 with BLOBs?
Posted by Simon Chemouil <sc...@gmail.com>.
OK, so Cassandra 2.1 now rejects writes it considers too big. It is
possible to increase the value by changing commitlog_segment_size_in_mb
in cassandra.yaml. It defaults to 32MB, and the maximum segment size for
a write is half that value:
from CommitLog.java:
// we only permit records HALF the size of a commit log, to ensure we
don't spin allocating many mostly
// empty segments when writing large records
private static final long MAX_MUTATION_SIZE =
DatabaseDescriptor.getCommitLogSegmentSize() >> 1;
which explains (with the request overhead) why my ~30,5MB blob was
rejected.
Simon
Le 20/06/2014 11:24, Simon Chemouil a écrit :
> For the record, I could reproduce the problem with blobs of size below 64MB.
>
> Caused by: java.lang.IllegalArgumentException: Mutation of 32000122
> bytes is too large for the maxiumum size of 16777216
>
> 32000122 is just ~30MB and fails on 2.1-rc1 while it works on 2.0.X for
> even larger values (up to 64MB works fine)
>
> Simon
>
> Le 20/06/2014 11:00, Simon Chemouil a écrit :
>> Le 20/06/2014 10:41, Duncan Sands a écrit :
>>> Hi Simon,
>>> 122880122 bytes is a lot more than 0.6MB... How are you sending your blob?
>>
>> Turns out there was a mistake in my code. The blob in this case was
>> actually 122MB!
>> Still the same code works fine on Cassandra 2.0.x so there might be a
>> bug lurking. Even if it's definitely above the recommended limit.
>>
>> Simon
>>
Re: Bug on 2.1-rc1 with BLOBs?
Posted by Simon Chemouil <sc...@gmail.com>.
For the record, I could reproduce the problem with blobs of size below 64MB.
Caused by: java.lang.IllegalArgumentException: Mutation of 32000122
bytes is too large for the maxiumum size of 16777216
32000122 is just ~30MB and fails on 2.1-rc1 while it works on 2.0.X for
even larger values (up to 64MB works fine)
Simon
Le 20/06/2014 11:00, Simon Chemouil a écrit :
> Le 20/06/2014 10:41, Duncan Sands a écrit :
>> Hi Simon,
>> 122880122 bytes is a lot more than 0.6MB... How are you sending your blob?
>
> Turns out there was a mistake in my code. The blob in this case was
> actually 122MB!
> Still the same code works fine on Cassandra 2.0.x so there might be a
> bug lurking. Even if it's definitely above the recommended limit.
>
> Simon
>
Re: Bug on 2.1-rc1 with BLOBs?
Posted by Simon Chemouil <sc...@gmail.com>.
Le 20/06/2014 10:41, Duncan Sands a écrit :
> Hi Simon,
> 122880122 bytes is a lot more than 0.6MB... How are you sending your blob?
Turns out there was a mistake in my code. The blob in this case was
actually 122MB!
Still the same code works fine on Cassandra 2.0.x so there might be a
bug lurking. Even if it's definitely above the recommended limit.
Simon
Re: Bug on 2.1-rc1 with BLOBs?
Posted by Duncan Sands <du...@gmail.com>.
Hi Simon,
On 20/06/14 10:18, Simon Chemouil wrote:
> Hi,
>
> When I am sending BLOBs _below_ the max query size (blob size=0.6MB), on
> Cassandra 2.0, it works fine, but on 2.1-rc1 I get the following error
> within the Cassandra server (from the logs) and the query just dies:
>
> WARN [SharedPool-Worker-2] 2014-06-20 10:06:00,263
> AbstractTracingAwareExecutorService.java:166 - Uncaught exception on
> thread Thread[SharedPool-Worker-2,5,main]: {}
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Mutation
> of 122880122 bytes is too large for the maxiumum size of 16777216
122880122 bytes is a lot more than 0.6MB... How are you sending your blob?
Ciao, Duncan.