You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by shyla deshpande <de...@gmail.com> on 2017/02/03 08:45:22 UTC

saveToCassandra issue. Please help

Hello All,

This is the content of my RDD which I am saving to Cassandra table.

But looks like the 2nd row is written first and then the first row
overwrites it. So I end up with bad output.

(494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H9M30S, WEDNESDAY)
(494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H10M0S, WEDNESDAY)

Is there a way to force the order of the rows written to Cassandra.

Please help.

Thanks

Re: saveToCassandra issue. Please help

Posted by shyla deshpande <de...@gmail.com>.

Thanks Fernando.  But I need to have only 1 row for a given user, date with
very low latency. So none of your options work for me.



On Fri, Feb 3, 2017 at 10:34 AM, Fernando Avalos <ga...@gmail.com> wrote:

> Hi Shyla,
>
> Maybe I am wrong, but I can see two options here.
>
> 1.- Do some grouping before insert to Cassandra.
> 2.- Insert to cassandra all the entries and add some logic to your
> request to get the most recent.
>
> Regards,
>
> 2017-02-03 10:26 GMT-08:00 shyla deshpande <de...@gmail.com>:
> > Hi All,
> >
> > I wanted to add more info ..
> > The first column is the user and the third is the period. and my key is
> > (userid, date) For a given user and date combination I want to see only 1
> > row. My problem is that PT0H10M0S is overwritten by PT0H9M30S, even
> though
> > the order of the rows in the RDD is PT0H9M30S and then PT0H10M0S.
> >
> > Appreciate your input. Thanks
> >
> > On Fri, Feb 3, 2017 at 12:45 AM, shyla deshpande <
> deshpandeshyla@gmail.com>
> > wrote:
> >>
> >> Hello All,
> >>
> >> This is the content of my RDD which I am saving to Cassandra table.
> >>
> >> But looks like the 2nd row is written first and then the first row
> >> overwrites it. So I end up with bad output.
> >>
> >> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H9M30S, WEDNESDAY)
> >> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H10M0S, WEDNESDAY)
> >>
> >> Is there a way to force the order of the rows written to Cassandra.
> >>
> >> Please help.
> >>
> >> Thanks
> >
> >
>

Re: saveToCassandra issue. Please help

Posted by shyla deshpande <de...@gmail.com>.

Hi All,

I wanted to add more info ..
The first column is the user and the third is the period. and my key is
(userid, date) For a given user and date combination I want to see only 1
row. My problem is that PT0H10M0S is overwritten by PT0H9M30S, even though
the order of the rows in the RDD is PT0H9M30S and then PT0H10M0S.

Appreciate your input. Thanks

On Fri, Feb 3, 2017 at 12:45 AM, shyla deshpande <de...@gmail.com>
wrote:

> Hello All,
>
> This is the content of my RDD which I am saving to Cassandra table.
>
> But looks like the 2nd row is written first and then the first row overwrites it. So I end up with bad output.
>
> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H9M30S, WEDNESDAY)
> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H10M0S, WEDNESDAY)
>
> Is there a way to force the order of the rows written to Cassandra.
>
> Please help.
>
> Thanks
>
>