Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

Hi all,

I am new to big data technologies and design so looking for help from java
world.

I have concept of tags and tagcombinations.
For example U.S.A and Pen are two tags AND if they come together in some
definition then register a tagcombination(U.S.A-Pen) for that..

*tags *(U.S.A, Pen, Pencil, India, Shampoo)
*tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
India-Pen-Shampoo)

- millions of tags
- billions of tagcombinations
- one tagcombination generally have 2-8 tags....
- Every day we get lakhs of new tagcombinations to write
- daily crores of queries to find matching combination by set of tags

Query need to support :
one tag or set of tags appears in how many tagcombinationids ????
If i query for Pen,India then it should return two tagcombinaions
(India-Pen, India-Pen-Shampoo))..Query will be fired by application in
realtime.

Please suggest a solution which is distributed with java client and can
handle scale of data  i am looking for..

Thanks
Naresh

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

This is what a friend of mine that knows elastic search had to say about
this:

o   Their tagcombinations are no different than say a category or similar
grouping for data

o   A search can then be executed on the index using a mixture of search
functions

§  Search on index for the tags category

§  Then refine the search based on filters such as "India" + "Pen"

§  This would return the IMP002 result





On Tue, Jan 28, 2014 at 9:52 AM, Naresh Yadav <ny...@gmail.com> wrote:

> i had tried on cassandra, that attempt was not convincing, but not used
> distributed counters....i actually needed tagcombination ids in output, not
> the no of matches, for the given set of tags..
>
> please illustrate a little your thought by taking my tag combination table
> design..
>
>
> On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:
>
>> No-sql solution with real-time counters would work, e.g. Cassandra or
>> hbase. But I think elastic search or Solr would be simpler and can do the
>> counting on access. There are solutions that are the combination of both
>> these approaches.
>>
>>
>> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> please give suggestions on this...
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am new to big data technologies and design so looking for help from
>>>> java world.
>>>>
>>>> I have concept of tags and tagcombinations.
>>>> For example U.S.A and Pen are two tags AND if they come together in
>>>> some definition then register a tagcombination(U.S.A-Pen) for that..
>>>>
>>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>>> India-Pen-Shampoo)
>>>>
>>>> - millions of tags
>>>> - billions of tagcombinations
>>>> - one tagcombination generally have 2-8 tags....
>>>> - Every day we get lakhs of new tagcombinations to write
>>>> - daily crores of queries to find matching combination by set of tags
>>>>
>>>> Query need to support :
>>>> one tag or set of tags appears in how many tagcombinationids ????
>>>> If i query for Pen,India then it should return two tagcombinaions
>>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>>> realtime.
>>>>
>>>> Please suggest a solution which is distributed with java client and can
>>>> handle scale of data  i am looking for..
>>>>
>>>> Thanks
>>>> Naresh
>>>>
>>>
>>>
>>>
>>>
>>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

This is what a friend of mine that knows elastic search had to say about
this:

o   Their tagcombinations are no different than say a category or similar
grouping for data

o   A search can then be executed on the index using a mixture of search
functions

§  Search on index for the tags category

§  Then refine the search based on filters such as "India" + "Pen"

§  This would return the IMP002 result





On Tue, Jan 28, 2014 at 9:52 AM, Naresh Yadav <ny...@gmail.com> wrote:

> i had tried on cassandra, that attempt was not convincing, but not used
> distributed counters....i actually needed tagcombination ids in output, not
> the no of matches, for the given set of tags..
>
> please illustrate a little your thought by taking my tag combination table
> design..
>
>
> On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:
>
>> No-sql solution with real-time counters would work, e.g. Cassandra or
>> hbase. But I think elastic search or Solr would be simpler and can do the
>> counting on access. There are solutions that are the combination of both
>> these approaches.
>>
>>
>> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> please give suggestions on this...
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am new to big data technologies and design so looking for help from
>>>> java world.
>>>>
>>>> I have concept of tags and tagcombinations.
>>>> For example U.S.A and Pen are two tags AND if they come together in
>>>> some definition then register a tagcombination(U.S.A-Pen) for that..
>>>>
>>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>>> India-Pen-Shampoo)
>>>>
>>>> - millions of tags
>>>> - billions of tagcombinations
>>>> - one tagcombination generally have 2-8 tags....
>>>> - Every day we get lakhs of new tagcombinations to write
>>>> - daily crores of queries to find matching combination by set of tags
>>>>
>>>> Query need to support :
>>>> one tag or set of tags appears in how many tagcombinationids ????
>>>> If i query for Pen,India then it should return two tagcombinaions
>>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>>> realtime.
>>>>
>>>> Please suggest a solution which is distributed with java client and can
>>>> handle scale of data  i am looking for..
>>>>
>>>> Thanks
>>>> Naresh
>>>>
>>>
>>>
>>>
>>>
>>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

This is what a friend of mine that knows elastic search had to say about
this:

o   Their tagcombinations are no different than say a category or similar
grouping for data

o   A search can then be executed on the index using a mixture of search
functions

§  Search on index for the tags category

§  Then refine the search based on filters such as "India" + "Pen"

§  This would return the IMP002 result





On Tue, Jan 28, 2014 at 9:52 AM, Naresh Yadav <ny...@gmail.com> wrote:

> i had tried on cassandra, that attempt was not convincing, but not used
> distributed counters....i actually needed tagcombination ids in output, not
> the no of matches, for the given set of tags..
>
> please illustrate a little your thought by taking my tag combination table
> design..
>
>
> On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:
>
>> No-sql solution with real-time counters would work, e.g. Cassandra or
>> hbase. But I think elastic search or Solr would be simpler and can do the
>> counting on access. There are solutions that are the combination of both
>> these approaches.
>>
>>
>> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> please give suggestions on this...
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am new to big data technologies and design so looking for help from
>>>> java world.
>>>>
>>>> I have concept of tags and tagcombinations.
>>>> For example U.S.A and Pen are two tags AND if they come together in
>>>> some definition then register a tagcombination(U.S.A-Pen) for that..
>>>>
>>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>>> India-Pen-Shampoo)
>>>>
>>>> - millions of tags
>>>> - billions of tagcombinations
>>>> - one tagcombination generally have 2-8 tags....
>>>> - Every day we get lakhs of new tagcombinations to write
>>>> - daily crores of queries to find matching combination by set of tags
>>>>
>>>> Query need to support :
>>>> one tag or set of tags appears in how many tagcombinationids ????
>>>> If i query for Pen,India then it should return two tagcombinaions
>>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>>> realtime.
>>>>
>>>> Please suggest a solution which is distributed with java client and can
>>>> handle scale of data  i am looking for..
>>>>
>>>> Thanks
>>>> Naresh
>>>>
>>>
>>>
>>>
>>>
>>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

This is what a friend of mine that knows elastic search had to say about
this:

o   Their tagcombinations are no different than say a category or similar
grouping for data

o   A search can then be executed on the index using a mixture of search
functions

§  Search on index for the tags category

§  Then refine the search based on filters such as "India" + "Pen"

§  This would return the IMP002 result





On Tue, Jan 28, 2014 at 9:52 AM, Naresh Yadav <ny...@gmail.com> wrote:

> i had tried on cassandra, that attempt was not convincing, but not used
> distributed counters....i actually needed tagcombination ids in output, not
> the no of matches, for the given set of tags..
>
> please illustrate a little your thought by taking my tag combination table
> design..
>
>
> On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:
>
>> No-sql solution with real-time counters would work, e.g. Cassandra or
>> hbase. But I think elastic search or Solr would be simpler and can do the
>> counting on access. There are solutions that are the combination of both
>> these approaches.
>>
>>
>> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> please give suggestions on this...
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am new to big data technologies and design so looking for help from
>>>> java world.
>>>>
>>>> I have concept of tags and tagcombinations.
>>>> For example U.S.A and Pen are two tags AND if they come together in
>>>> some definition then register a tagcombination(U.S.A-Pen) for that..
>>>>
>>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>>> India-Pen-Shampoo)
>>>>
>>>> - millions of tags
>>>> - billions of tagcombinations
>>>> - one tagcombination generally have 2-8 tags....
>>>> - Every day we get lakhs of new tagcombinations to write
>>>> - daily crores of queries to find matching combination by set of tags
>>>>
>>>> Query need to support :
>>>> one tag or set of tags appears in how many tagcombinationids ????
>>>> If i query for Pen,India then it should return two tagcombinaions
>>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>>> realtime.
>>>>
>>>> Please suggest a solution which is distributed with java client and can
>>>> handle scale of data  i am looking for..
>>>>
>>>> Thanks
>>>> Naresh
>>>>
>>>
>>>
>>>
>>>
>>
>
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

i had tried on cassandra, that attempt was not convincing, but not used
distributed counters....i actually needed tagcombination ids in output, not
the no of matches, for the given set of tags..

please illustrate a little your thought by taking my tag combination table
design..


On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:

> No-sql solution with real-time counters would work, e.g. Cassandra or
> hbase. But I think elastic search or Solr would be simpler and can do the
> counting on access. There are solutions that are the combination of both
> these approaches.
>
>
> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> please give suggestions on this...
>>
>>
>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> Hi all,
>>>
>>> I am new to big data technologies and design so looking for help from
>>> java world.
>>>
>>> I have concept of tags and tagcombinations.
>>> For example U.S.A and Pen are two tags AND if they come together in some
>>> definition then register a tagcombination(U.S.A-Pen) for that..
>>>
>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>> India-Pen-Shampoo)
>>>
>>> - millions of tags
>>> - billions of tagcombinations
>>> - one tagcombination generally have 2-8 tags....
>>> - Every day we get lakhs of new tagcombinations to write
>>> - daily crores of queries to find matching combination by set of tags
>>>
>>> Query need to support :
>>> one tag or set of tags appears in how many tagcombinationids ????
>>> If i query for Pen,India then it should return two tagcombinaions
>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>> realtime.
>>>
>>> Please suggest a solution which is distributed with java client and can
>>> handle scale of data  i am looking for..
>>>
>>> Thanks
>>> Naresh
>>>
>>
>>
>>
>>
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

i had tried on cassandra, that attempt was not convincing, but not used
distributed counters....i actually needed tagcombination ids in output, not
the no of matches, for the given set of tags..

please illustrate a little your thought by taking my tag combination table
design..


On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:

> No-sql solution with real-time counters would work, e.g. Cassandra or
> hbase. But I think elastic search or Solr would be simpler and can do the
> counting on access. There are solutions that are the combination of both
> these approaches.
>
>
> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> please give suggestions on this...
>>
>>
>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> Hi all,
>>>
>>> I am new to big data technologies and design so looking for help from
>>> java world.
>>>
>>> I have concept of tags and tagcombinations.
>>> For example U.S.A and Pen are two tags AND if they come together in some
>>> definition then register a tagcombination(U.S.A-Pen) for that..
>>>
>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>> India-Pen-Shampoo)
>>>
>>> - millions of tags
>>> - billions of tagcombinations
>>> - one tagcombination generally have 2-8 tags....
>>> - Every day we get lakhs of new tagcombinations to write
>>> - daily crores of queries to find matching combination by set of tags
>>>
>>> Query need to support :
>>> one tag or set of tags appears in how many tagcombinationids ????
>>> If i query for Pen,India then it should return two tagcombinaions
>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>> realtime.
>>>
>>> Please suggest a solution which is distributed with java client and can
>>> handle scale of data  i am looking for..
>>>
>>> Thanks
>>> Naresh
>>>
>>
>>
>>
>>
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

i had tried on cassandra, that attempt was not convincing, but not used
distributed counters....i actually needed tagcombination ids in output, not
the no of matches, for the given set of tags..

please illustrate a little your thought by taking my tag combination table
design..


On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:

> No-sql solution with real-time counters would work, e.g. Cassandra or
> hbase. But I think elastic search or Solr would be simpler and can do the
> counting on access. There are solutions that are the combination of both
> these approaches.
>
>
> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> please give suggestions on this...
>>
>>
>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> Hi all,
>>>
>>> I am new to big data technologies and design so looking for help from
>>> java world.
>>>
>>> I have concept of tags and tagcombinations.
>>> For example U.S.A and Pen are two tags AND if they come together in some
>>> definition then register a tagcombination(U.S.A-Pen) for that..
>>>
>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>> India-Pen-Shampoo)
>>>
>>> - millions of tags
>>> - billions of tagcombinations
>>> - one tagcombination generally have 2-8 tags....
>>> - Every day we get lakhs of new tagcombinations to write
>>> - daily crores of queries to find matching combination by set of tags
>>>
>>> Query need to support :
>>> one tag or set of tags appears in how many tagcombinationids ????
>>> If i query for Pen,India then it should return two tagcombinaions
>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>> realtime.
>>>
>>> Please suggest a solution which is distributed with java client and can
>>> handle scale of data  i am looking for..
>>>
>>> Thanks
>>> Naresh
>>>
>>
>>
>>
>>
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

i had tried on cassandra, that attempt was not convincing, but not used
distributed counters....i actually needed tagcombination ids in output, not
the no of matches, for the given set of tags..

please illustrate a little your thought by taking my tag combination table
design..


On Tue, Jan 28, 2014 at 10:57 PM, Peyman Mohajerian <mo...@gmail.com>wrote:

> No-sql solution with real-time counters would work, e.g. Cassandra or
> hbase. But I think elastic search or Solr would be simpler and can do the
> counting on access. There are solutions that are the combination of both
> these approaches.
>
>
> On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> please give suggestions on this...
>>
>>
>> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>>
>>> Hi all,
>>>
>>> I am new to big data technologies and design so looking for help from
>>> java world.
>>>
>>> I have concept of tags and tagcombinations.
>>> For example U.S.A and Pen are two tags AND if they come together in some
>>> definition then register a tagcombination(U.S.A-Pen) for that..
>>>
>>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>>> India-Pen-Shampoo)
>>>
>>> - millions of tags
>>> - billions of tagcombinations
>>> - one tagcombination generally have 2-8 tags....
>>> - Every day we get lakhs of new tagcombinations to write
>>> - daily crores of queries to find matching combination by set of tags
>>>
>>> Query need to support :
>>> one tag or set of tags appears in how many tagcombinationids ????
>>> If i query for Pen,India then it should return two tagcombinaions
>>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>>> realtime.
>>>
>>> Please suggest a solution which is distributed with java client and can
>>> handle scale of data  i am looking for..
>>>
>>> Thanks
>>> Naresh
>>>
>>
>>
>>
>>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

No-sql solution with real-time counters would work, e.g. Cassandra or
hbase. But I think elastic search or Solr would be simpler and can do the
counting on access. There are solutions that are the combination of both
these approaches.


On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com> wrote:

> please give suggestions on this...
>
>
> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> Hi all,
>>
>> I am new to big data technologies and design so looking for help from
>> java world.
>>
>> I have concept of tags and tagcombinations.
>> For example U.S.A and Pen are two tags AND if they come together in some
>> definition then register a tagcombination(U.S.A-Pen) for that..
>>
>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>> India-Pen-Shampoo)
>>
>> - millions of tags
>> - billions of tagcombinations
>> - one tagcombination generally have 2-8 tags....
>> - Every day we get lakhs of new tagcombinations to write
>> - daily crores of queries to find matching combination by set of tags
>>
>> Query need to support :
>> one tag or set of tags appears in how many tagcombinationids ????
>> If i query for Pen,India then it should return two tagcombinaions
>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>> realtime.
>>
>> Please suggest a solution which is distributed with java client and can
>> handle scale of data  i am looking for..
>>
>> Thanks
>> Naresh
>>
>
>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

No-sql solution with real-time counters would work, e.g. Cassandra or
hbase. But I think elastic search or Solr would be simpler and can do the
counting on access. There are solutions that are the combination of both
these approaches.


On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com> wrote:

> please give suggestions on this...
>
>
> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> Hi all,
>>
>> I am new to big data technologies and design so looking for help from
>> java world.
>>
>> I have concept of tags and tagcombinations.
>> For example U.S.A and Pen are two tags AND if they come together in some
>> definition then register a tagcombination(U.S.A-Pen) for that..
>>
>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>> India-Pen-Shampoo)
>>
>> - millions of tags
>> - billions of tagcombinations
>> - one tagcombination generally have 2-8 tags....
>> - Every day we get lakhs of new tagcombinations to write
>> - daily crores of queries to find matching combination by set of tags
>>
>> Query need to support :
>> one tag or set of tags appears in how many tagcombinationids ????
>> If i query for Pen,India then it should return two tagcombinaions
>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>> realtime.
>>
>> Please suggest a solution which is distributed with java client and can
>> handle scale of data  i am looking for..
>>
>> Thanks
>> Naresh
>>
>
>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

No-sql solution with real-time counters would work, e.g. Cassandra or
hbase. But I think elastic search or Solr would be simpler and can do the
counting on access. There are solutions that are the combination of both
these approaches.


On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com> wrote:

> please give suggestions on this...
>
>
> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> Hi all,
>>
>> I am new to big data technologies and design so looking for help from
>> java world.
>>
>> I have concept of tags and tagcombinations.
>> For example U.S.A and Pen are two tags AND if they come together in some
>> definition then register a tagcombination(U.S.A-Pen) for that..
>>
>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>> India-Pen-Shampoo)
>>
>> - millions of tags
>> - billions of tagcombinations
>> - one tagcombination generally have 2-8 tags....
>> - Every day we get lakhs of new tagcombinations to write
>> - daily crores of queries to find matching combination by set of tags
>>
>> Query need to support :
>> one tag or set of tags appears in how many tagcombinationids ????
>> If i query for Pen,India then it should return two tagcombinaions
>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>> realtime.
>>
>> Please suggest a solution which is distributed with java client and can
>> handle scale of data  i am looking for..
>>
>> Thanks
>> Naresh
>>
>
>
>
>

Re: Suggestion technology/design on this usecase

[Peyman Mohajerian <mo...@gmail.com>]

No-sql solution with real-time counters would work, e.g. Cassandra or
hbase. But I think elastic search or Solr would be simpler and can do the
counting on access. There are solutions that are the combination of both
these approaches.


On Tue, Jan 28, 2014 at 8:51 AM, Naresh Yadav <ny...@gmail.com> wrote:

> please give suggestions on this...
>
>
> On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com>wrote:
>
>> Hi all,
>>
>> I am new to big data technologies and design so looking for help from
>> java world.
>>
>> I have concept of tags and tagcombinations.
>> For example U.S.A and Pen are two tags AND if they come together in some
>> definition then register a tagcombination(U.S.A-Pen) for that..
>>
>> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
>> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
>> India-Pen-Shampoo)
>>
>> - millions of tags
>> - billions of tagcombinations
>> - one tagcombination generally have 2-8 tags....
>> - Every day we get lakhs of new tagcombinations to write
>> - daily crores of queries to find matching combination by set of tags
>>
>> Query need to support :
>> one tag or set of tags appears in how many tagcombinationids ????
>> If i query for Pen,India then it should return two tagcombinaions
>> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
>> realtime.
>>
>> Please suggest a solution which is distributed with java client and can
>> handle scale of data  i am looking for..
>>
>> Thanks
>> Naresh
>>
>
>
>
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

please give suggestions on this...


On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com> wrote:

> Hi all,
>
> I am new to big data technologies and design so looking for help from java
> world.
>
> I have concept of tags and tagcombinations.
> For example U.S.A and Pen are two tags AND if they come together in some
> definition then register a tagcombination(U.S.A-Pen) for that..
>
> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
> India-Pen-Shampoo)
>
> - millions of tags
> - billions of tagcombinations
> - one tagcombination generally have 2-8 tags....
> - Every day we get lakhs of new tagcombinations to write
> - daily crores of queries to find matching combination by set of tags
>
> Query need to support :
> one tag or set of tags appears in how many tagcombinationids ????
> If i query for Pen,India then it should return two tagcombinaions
> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
> realtime.
>
> Please suggest a solution which is distributed with java client and can
> handle scale of data  i am looking for..
>
> Thanks
> Naresh
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

please give suggestions on this...


On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com> wrote:

> Hi all,
>
> I am new to big data technologies and design so looking for help from java
> world.
>
> I have concept of tags and tagcombinations.
> For example U.S.A and Pen are two tags AND if they come together in some
> definition then register a tagcombination(U.S.A-Pen) for that..
>
> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
> India-Pen-Shampoo)
>
> - millions of tags
> - billions of tagcombinations
> - one tagcombination generally have 2-8 tags....
> - Every day we get lakhs of new tagcombinations to write
> - daily crores of queries to find matching combination by set of tags
>
> Query need to support :
> one tag or set of tags appears in how many tagcombinationids ????
> If i query for Pen,India then it should return two tagcombinaions
> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
> realtime.
>
> Please suggest a solution which is distributed with java client and can
> handle scale of data  i am looking for..
>
> Thanks
> Naresh
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

please give suggestions on this...


On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com> wrote:

> Hi all,
>
> I am new to big data technologies and design so looking for help from java
> world.
>
> I have concept of tags and tagcombinations.
> For example U.S.A and Pen are two tags AND if they come together in some
> definition then register a tagcombination(U.S.A-Pen) for that..
>
> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
> India-Pen-Shampoo)
>
> - millions of tags
> - billions of tagcombinations
> - one tagcombination generally have 2-8 tags....
> - Every day we get lakhs of new tagcombinations to write
> - daily crores of queries to find matching combination by set of tags
>
> Query need to support :
> one tag or set of tags appears in how many tagcombinationids ????
> If i query for Pen,India then it should return two tagcombinaions
> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
> realtime.
>
> Please suggest a solution which is distributed with java client and can
> handle scale of data  i am looking for..
>
> Thanks
> Naresh
>

Re: Suggestion technology/design on this usecase

[Naresh Yadav <ny...@gmail.com>]

please give suggestions on this...


On Tue, Jan 28, 2014 at 3:18 PM, Naresh Yadav <ny...@gmail.com> wrote:

> Hi all,
>
> I am new to big data technologies and design so looking for help from java
> world.
>
> I have concept of tags and tagcombinations.
> For example U.S.A and Pen are two tags AND if they come together in some
> definition then register a tagcombination(U.S.A-Pen) for that..
>
> *tags *(U.S.A, Pen, Pencil, India, Shampoo)
> *tagcombinations*(U.S.A-Pen, India-pencil, U.S.A-Pencil, India-Pen,
> India-Pen-Shampoo)
>
> - millions of tags
> - billions of tagcombinations
> - one tagcombination generally have 2-8 tags....
> - Every day we get lakhs of new tagcombinations to write
> - daily crores of queries to find matching combination by set of tags
>
> Query need to support :
> one tag or set of tags appears in how many tagcombinationids ????
> If i query for Pen,India then it should return two tagcombinaions
> (India-Pen, India-Pen-Shampoo))..Query will be fired by application in
> realtime.
>
> Please suggest a solution which is distributed with java client and can
> handle scale of data  i am looking for..
>
> Thanks
> Naresh
>