You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Terje Marthinussen <tm...@gmail.com> on 2011/03/04 20:36:56 UTC
2GB rows and errros
Hi,
Any good reason this guy
public int bytesPastMark(FileMark mark)
{
assert mark instanceof BufferedRandomAccessFileMark;
long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
mark).pointer;
assert bytes >= 0;
if (bytes > Integer.MAX_VALUE)
throw new UnsupportedOperationException("Overflow: " + bytes);
return (int) bytes;
}
does not show an error more like "Overflow: Maximum row size 2GB.
Currently:" + bytes?
Error you get today is not exactly self explaining :)
Terje
Re: 2GB rows and errros
Posted by Jonathan Ellis <jb...@gmail.com>.
You're right, I was looking at the no-arg bytesPastMark overload. Sorry!
On Fri, Mar 4, 2011 at 2:32 PM, Terje Marthinussen
<tm...@gmail.com> wrote:
> Are you entirely sure?
> My Eclipse disagree and the Exception we had a day ago in a 0.7.3 install
> disagree too :)
>
> Terje
>
> On Sat, Mar 5, 2011 at 5:01 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> That method is unused, so it's a little academic.
>>
>> On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
>> <tm...@gmail.com> wrote:
>> > Hi,
>> >
>> > Any good reason this guy
>> > public int bytesPastMark(FileMark mark)
>> > {
>> > assert mark instanceof BufferedRandomAccessFileMark;
>> > long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
>> > mark).pointer;
>> >
>> > assert bytes >= 0;
>> > if (bytes > Integer.MAX_VALUE)
>> > throw new UnsupportedOperationException("Overflow: " +
>> > bytes);
>> > return (int) bytes;
>> > }
>> >
>> > does not show an error more like "Overflow: Maximum row size 2GB.
>> > Currently:" + bytes?
>> >
>> > Error you get today is not exactly self explaining :)
>> >
>> > Terje
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: 2GB rows and errros
Posted by Jonathan Ellis <jb...@gmail.com>.
That method is unused, so it's a little academic.
On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
<tm...@gmail.com> wrote:
> Hi,
>
> Any good reason this guy
> public int bytesPastMark(FileMark mark)
> {
> assert mark instanceof BufferedRandomAccessFileMark;
> long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
> mark).pointer;
>
> assert bytes >= 0;
> if (bytes > Integer.MAX_VALUE)
> throw new UnsupportedOperationException("Overflow: " + bytes);
> return (int) bytes;
> }
>
> does not show an error more like "Overflow: Maximum row size 2GB.
> Currently:" + bytes?
>
> Error you get today is not exactly self explaining :)
>
> Terje
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: 2GB rows and errros
Posted by Terje Marthinussen <tm...@gmail.com>.
Ah, yes, I should have noticed that distinction.
We actually hit this overflow on a row that was more than 60GB (yes, we had
to count the number of digits a few times to make sure).
On Sat, Mar 5, 2011 at 5:41 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> Second try:
>
> - this isn't used in row size (which is not limited to 2GB)
> - it's used both for the column index summary and index-block reading,
> both of which should be well under 2GB
> - however, I don't see any technical reason this method should return
> an int instead of a long
> - if we make that change we should probably do additional sanity
> checks in the callers, which will have the necessary context to
> provide better error messages
>
> On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
> <tm...@gmail.com> wrote:
> > Hi,
> >
> > Any good reason this guy
> > public int bytesPastMark(FileMark mark)
> > {
> > assert mark instanceof BufferedRandomAccessFileMark;
> > long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
> > mark).pointer;
> >
> > assert bytes >= 0;
> > if (bytes > Integer.MAX_VALUE)
> > throw new UnsupportedOperationException("Overflow: " + bytes);
> > return (int) bytes;
> > }
> >
> > does not show an error more like "Overflow: Maximum row size 2GB.
> > Currently:" + bytes?
> >
> > Error you get today is not exactly self explaining :)
> >
> > Terje
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: 2GB rows and errros
Posted by Muga Nishizawa <mu...@gmail.com>.
Hi,
I filed a ticket on this problem.
https://issues.apache.org/jira/browse/CASSANDRA-2297
Thanks
Muga Nishizawa,
On Sat, Mar 5, 2011 at 6:10 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> Can you open a ticket with the exception you saw?
>
> On Fri, Mar 4, 2011 at 2:51 PM, Terje Marthinussen
> <tm...@gmail.com> wrote:
>> Ah, yes, I should have noticed that distinction.
>> We actually hit this overflow on a row that was more than 60GB (yes, we had
>> to count the number of digits a few times to make sure).
>> Terje
>> On Sat, Mar 5, 2011 at 5:41 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>>>
>>> Second try:
>>>
>>> - this isn't used in row size (which is not limited to 2GB)
>>> - it's used both for the column index summary and index-block reading,
>>> both of which should be well under 2GB
>>> - however, I don't see any technical reason this method should return
>>> an int instead of a long
>>> - if we make that change we should probably do additional sanity
>>> checks in the callers, which will have the necessary context to
>>> provide better error messages
>>>
>>> On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
>>> <tm...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > Any good reason this guy
>>> > public int bytesPastMark(FileMark mark)
>>> > {
>>> > assert mark instanceof BufferedRandomAccessFileMark;
>>> > long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
>>> > mark).pointer;
>>> >
>>> > assert bytes >= 0;
>>> > if (bytes > Integer.MAX_VALUE)
>>> > throw new UnsupportedOperationException("Overflow: " +
>>> > bytes);
>>> > return (int) bytes;
>>> > }
>>> >
>>> > does not show an error more like "Overflow: Maximum row size 2GB.
>>> > Currently:" + bytes?
>>> >
>>> > Error you get today is not exactly self explaining :)
>>> >
>>> > Terje
>>> >
>>>
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>>
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: 2GB rows and errros
Posted by Jonathan Ellis <jb...@gmail.com>.
Can you open a ticket with the exception you saw?
On Fri, Mar 4, 2011 at 2:51 PM, Terje Marthinussen
<tm...@gmail.com> wrote:
> Ah, yes, I should have noticed that distinction.
> We actually hit this overflow on a row that was more than 60GB (yes, we had
> to count the number of digits a few times to make sure).
> Terje
> On Sat, Mar 5, 2011 at 5:41 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> Second try:
>>
>> - this isn't used in row size (which is not limited to 2GB)
>> - it's used both for the column index summary and index-block reading,
>> both of which should be well under 2GB
>> - however, I don't see any technical reason this method should return
>> an int instead of a long
>> - if we make that change we should probably do additional sanity
>> checks in the callers, which will have the necessary context to
>> provide better error messages
>>
>> On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
>> <tm...@gmail.com> wrote:
>> > Hi,
>> >
>> > Any good reason this guy
>> > public int bytesPastMark(FileMark mark)
>> > {
>> > assert mark instanceof BufferedRandomAccessFileMark;
>> > long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
>> > mark).pointer;
>> >
>> > assert bytes >= 0;
>> > if (bytes > Integer.MAX_VALUE)
>> > throw new UnsupportedOperationException("Overflow: " +
>> > bytes);
>> > return (int) bytes;
>> > }
>> >
>> > does not show an error more like "Overflow: Maximum row size 2GB.
>> > Currently:" + bytes?
>> >
>> > Error you get today is not exactly self explaining :)
>> >
>> > Terje
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: 2GB rows and errros
Posted by Jonathan Ellis <jb...@gmail.com>.
Second try:
- this isn't used in row size (which is not limited to 2GB)
- it's used both for the column index summary and index-block reading,
both of which should be well under 2GB
- however, I don't see any technical reason this method should return
an int instead of a long
- if we make that change we should probably do additional sanity
checks in the callers, which will have the necessary context to
provide better error messages
On Fri, Mar 4, 2011 at 1:36 PM, Terje Marthinussen
<tm...@gmail.com> wrote:
> Hi,
>
> Any good reason this guy
> public int bytesPastMark(FileMark mark)
> {
> assert mark instanceof BufferedRandomAccessFileMark;
> long bytes = getFilePointer() - ((BufferedRandomAccessFileMark)
> mark).pointer;
>
> assert bytes >= 0;
> if (bytes > Integer.MAX_VALUE)
> throw new UnsupportedOperationException("Overflow: " + bytes);
> return (int) bytes;
> }
>
> does not show an error more like "Overflow: Maximum row size 2GB.
> Currently:" + bytes?
>
> Error you get today is not exactly self explaining :)
>
> Terje
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com