You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by ziju feng <pk...@gmail.com> on 2014/05/16 05:14:11 UTC

Data modeling for Pinterest-like application

Hello,

I'm working on data modeling for a Pinterest-like project. There are
basically two main concepts: Pin and Board, just like Pinterest, where pin
is an item containing an image, description and some other information such
as a like count, and each board should contain a sorted list of Pins.

The board can be modeled with primary key (board_id, created_at, pin_id)
where created_at is used to sort the pins of the board by date. The problem
is whether I should denormalize details of pins into the board table or
just retrieve pins by page (page size can be 10~20) and then multi-get by
pin_ids to obtain details.

Since there are some boards that are accessed very often (like the home
board), denormalization seems to be a reasonable choice to enhance read
performance. However, we then have to update not only the pin table be also
each row in the board table that contains the pin whenever a pin is
updated, which sometimes could be quite frequent (such as updating the like
count). Since a pin may be contained by many boards (could be thousands),
denormalization seems to bring a lot of load on the write side as well as
application code complexity.

Any suggestion to whether our data model should go denormalized or the
normalized/multi-get way which then perhaps need a separate cached layer
for read?

Thanks,

Ziju

Re: Data modeling for Pinterest-like application

Posted by ziju feng <pk...@gmail.com>.
I was thinking to use counter type a separate pin counter table and, when I
need to update the like count, I would use read-after-write to get the
current value and timestamp and then denormalize into pin's detail table and
board tables. 

Is it a viable solution in this case?

Thanks



--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-modeling-for-Pinterest-like-application-tp7594481p7594539.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Re: Data modeling for Pinterest-like application

Posted by DuyHai Doan <do...@gmail.com>.
"A related question is whether it is a good idea to denormalize on
read-heavy part of data while normalize on other less frequently-accessed
data?"

 Heavy read -> denormalize
 Less frequently accessed data -> it depends how "less frequent" it is and
whether it's complicated to denormalize in your code.

"We will also have a like board for each user containing pins that they
like, which can be somewhat private and only viewed by the owner."

"Since a pin can be potentially liked by thousands of user, if we also
denormalize the like board, everytime that pin is liked by another user we
would have to update the like count in thousands of like boards."

If I understand your use case, a pin consist of a "description" and a "like
count" isn't it ?  It makes sense then to use counter type for the "like
count" but in this case you cannot denormalize the counter type because you
cannot mix counter column family with normal column family (containing the
pin description and properties).

*If you are sure* the the like board  is accessed rarely or not very
frequently by the users, then normalization could be the answer. You can
mitigate further the effect of N+1 select in the like board by paging pins
(not showing all of them at once but by page of 10 for example)








On Sat, May 17, 2014 at 2:37 AM, ziju feng <pk...@gmail.com> wrote:

> Thanks for your answer, I really like the frequency of update vs read way
> of
> thinking.
>
> A related question is whether it is a good idea to denormalize on
> read-heavy
> part of data while normalize on other less frequently-accessed data?
>
> Our app will have a limited number of system managed boards that are viewed
> by every user so it makes sense to denormalize and propagate updates of
> pins
> to these boards.
>
> We will also have a like board for each user containing pins that they
> like,
> which can be somewhat private and only viewed by the owner.
>
> Since a pin can be potentially liked by thousands of user, if we also
> denormalize the like board, everytime that pin is liked by another user we
> would have to update the like count in thousands of like boards.
>
> Does normalize work better in this case or cassandra can handle this kind
> of
> write load?
>
>
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-modeling-for-Pinterest-like-application-tp7594481p7594517.html
> Sent from the cassandra-user@incubator.apache.org mailing list archive at
> Nabble.com.
>

Re: Data modeling for Pinterest-like application

Posted by ziju feng <pk...@gmail.com>.
Thanks for your answer, I really like the frequency of update vs read way of
thinking. 

A related question is whether it is a good idea to denormalize on read-heavy
part of data while normalize on other less frequently-accessed data? 

Our app will have a limited number of system managed boards that are viewed
by every user so it makes sense to denormalize and propagate updates of pins
to these boards. 

We will also have a like board for each user containing pins that they like,
which can be somewhat private and only viewed by the owner. 

Since a pin can be potentially liked by thousands of user, if we also
denormalize the like board, everytime that pin is liked by another user we
would have to update the like count in thousands of like boards. 

Does normalize work better in this case or cassandra can handle this kind of
write load?



--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-modeling-for-Pinterest-like-application-tp7594481p7594517.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Re: Data modeling for Pinterest-like application

Posted by DuyHai Doan <do...@gmail.com>.
"The problem is whether I should denormalize details of pins into the board
table or just retrieve pins by page (page size can be 10~20) and then
multi-get by pin_ids to obtain details"

--> Denormalize is the best way to go in your case. Otherwise, for 1 board
read, you'll have 10-20 subsequent reads to load the pins. Multiply it by
the number of users listing boards and you'll be quickly in trouble...

 For the update of pins like count, you'll need to use counter type.

 "denormalization seems to bring a lot of load on the write side as well as
application code complexity" --> first C* copes quite well with write load.
Second, you should ask yourself: "how often is the update scenario vs the
read scenario" ? Usually the read pattern is predominant.

 About update code complexity it's the price to pay for read performance.
CQRS pattern will help you to separate the write and read stages, also
heavy unit and integration testing.



On Fri, May 16, 2014 at 5:14 AM, ziju feng <pk...@gmail.com> wrote:

> Hello,
>
> I'm working on data modeling for a Pinterest-like project. There are
> basically two main concepts: Pin and Board, just like Pinterest, where pin
> is an item containing an image, description and some other information such
> as a like count, and each board should contain a sorted list of Pins.
>
> The board can be modeled with primary key (board_id, created_at, pin_id)
> where created_at is used to sort the pins of the board by date. The problem
> is whether I should denormalize details of pins into the board table or
> just retrieve pins by page (page size can be 10~20) and then multi-get by
> pin_ids to obtain details.
>
> Since there are some boards that are accessed very often (like the home
> board), denormalization seems to be a reasonable choice to enhance read
> performance. However, we then have to update not only the pin table be also
> each row in the board table that contains the pin whenever a pin is
> updated, which sometimes could be quite frequent (such as updating the like
> count). Since a pin may be contained by many boards (could be thousands),
> denormalization seems to bring a lot of load on the write side as well as
> application code complexity.
>
> Any suggestion to whether our data model should go denormalized or the
> normalized/multi-get way which then perhaps need a separate cached layer
> for read?
>
> Thanks,
>
> Ziju
>