Re: Binary Loading Question

[Chris Goffinet <cg...@chrisgoffinet.com>]

It would be helpful if you could attach your example. Also did you  
look at the example in contrib by chance?


On Sep 30, 2009, at 2:05 PM, Dan Di Spaltro wrote:

> Okay i get what its doing now... its pretty confusing until I saw
> jbellis' comments.  now when i flush binary it is throwing on the
> key.toString line because the token is null.  When i setup the
> rowmutation on the client i used StringToken as opposed to big integer
> but thats because i am not sure what to put there so i used the tokens
> chose by cassandra
>
> On 9/30/09, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
>> It's very jank. I kinda understand where they were coming from on  
>> that. Let
>> me think through it some more and open a  ticket to improve some of  
>> that
>> code.
>>
>>
>> On Wed, Sep 30, 2009 at 12:22 PM, Jonathan Ellis  
>> <jb...@gmail.com> wrote:
>>
>>> So it looks like it takes multiple rows, one per "column" in the
>>> rowmutation.
>>>
>>> Which is confusing and weird if you ask me.
>>>
>>> On Wed, Sep 30, 2009 at 1:57 PM, Jonathan Ellis  
>>> <jb...@gmail.com> wrote:
>>>> Yeah, just eyeballing it
>>>>
>>>>               ColumnFamilyStore cfStore =
>>>> columnFamilyStores_.get(columnFamily.name());
>>>>
>>>> would make more sense to me.
>>>>
>>>> Got any insight for us, Chris? :)
>>>>
>>>> -Jonathan
>>>>
>>>> On Wed, Sep 30, 2009 at 1:47 PM, Dan Di Spaltro
>>>> <da...@gmail.com>
>>> wrote:
>>>>> I was trying to binary load some data into an instance and ran  
>>>>> into a
>>>>> null exception in the process.  So I began looking at the code,  
>>>>> and it
>>>>> looks like around Table.java:653 of trunk, that piece of code  
>>>>> doesn't
>>>>> really make sense.  I don't understand the internals of  
>>>>> Cassandra all
>>>>> that well, but this doesn't seem right.  The null exception came  
>>>>> from
>>>>> null cfStore.
>>>>>
>>>>> Either way I am just trying to use the binarymemtable option to  
>>>>> load
>>>>> values into cassandra, since I have about 240 million rows to  
>>>>> load.
>>>>>
>>>>> {
>>>>>               ColumnFamilyStore cfStore =
>>>>> columnFamilyStores_.get(new String(column.name(), "UTF-8"));
>>>>>               cfStore.applyBinary(key, column.value());
>>>>> }
>>>>>
>>>>> --
>>>>> Dan Di Spaltro
>>>>>
>>>>
>>>
>>
>
>
> -- 
> Dan Di Spaltro

Re: Binary Loading Question

[Dan Di Spaltro <da...@gmail.com>]

Chris,

Sorry for the delay.  Jbellis figured it out, I am using the OPP and
during serialization the Token was null and throwing.

I will say after looking through the contrib code, the BMT is pretty
odd and especially the way it gets serialized with indexes; then the
payload dropped into the value and the column name is loaded with the
actual ColumnFamily name; however, we have about half a billion points
to migrate and ended up getting like 35k sec from one pretty weak
machine.

Thanks for the help,

On Wed, Sep 30, 2009 at 7:15 PM, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
> It would be helpful if you could attach your example. Also did you look at
> the example in contrib by chance?
>
>
> On Sep 30, 2009, at 2:05 PM, Dan Di Spaltro wrote:
>
>> Okay i get what its doing now... its pretty confusing until I saw
>> jbellis' comments.  now when i flush binary it is throwing on the
>> key.toString line because the token is null.  When i setup the
>> rowmutation on the client i used StringToken as opposed to big integer
>> but thats because i am not sure what to put there so i used the tokens
>> chose by cassandra
>>
>> On 9/30/09, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
>>>
>>> It's very jank. I kinda understand where they were coming from on that.
>>> Let
>>> me think through it some more and open a  ticket to improve some of that
>>> code.
>>>
>>>
>>> On Wed, Sep 30, 2009 at 12:22 PM, Jonathan Ellis <jb...@gmail.com>
>>> wrote:
>>>
>>>> So it looks like it takes multiple rows, one per "column" in the
>>>> rowmutation.
>>>>
>>>> Which is confusing and weird if you ask me.
>>>>
>>>> On Wed, Sep 30, 2009 at 1:57 PM, Jonathan Ellis <jb...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Yeah, just eyeballing it
>>>>>
>>>>>              ColumnFamilyStore cfStore =
>>>>> columnFamilyStores_.get(columnFamily.name());
>>>>>
>>>>> would make more sense to me.
>>>>>
>>>>> Got any insight for us, Chris? :)
>>>>>
>>>>> -Jonathan
>>>>>
>>>>> On Wed, Sep 30, 2009 at 1:47 PM, Dan Di Spaltro
>>>>> <da...@gmail.com>
>>>>
>>>> wrote:
>>>>>>
>>>>>> I was trying to binary load some data into an instance and ran into a
>>>>>> null exception in the process.  So I began looking at the code, and it
>>>>>> looks like around Table.java:653 of trunk, that piece of code doesn't
>>>>>> really make sense.  I don't understand the internals of Cassandra all
>>>>>> that well, but this doesn't seem right.  The null exception came from
>>>>>> null cfStore.
>>>>>>
>>>>>> Either way I am just trying to use the binarymemtable option to load
>>>>>> values into cassandra, since I have about 240 million rows to load.
>>>>>>
>>>>>> {
>>>>>>              ColumnFamilyStore cfStore =
>>>>>> columnFamilyStores_.get(new String(column.name(), "UTF-8"));
>>>>>>              cfStore.applyBinary(key, column.value());
>>>>>> }
>>>>>>
>>>>>> --
>>>>>> Dan Di Spaltro
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Dan Di Spaltro
>
>



-- 
Dan Di Spaltro