You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Boaz Citrin <bc...@gmail.com> on 2014/10/05 01:24:23 UTC
View question
Hello,
My documents contain two fields to maintain group associations, say "group"
holds the group document id, and "associated" holds the date this document
was added to the group.
Now I want to be able to know how many documents were added to a given
group[s] between two given dates.
The challenge is that to be able to filter by dates, I need to have the
date as the key first part.
But I also need the group as the first key part in order to aggregate the
number of group associations.
So I see two options here:
1.
Map: associated, {"group": group}
Reduce: a function that aggregates all values by group, which I assume is
fine as I know the number of groups is relatively small.
(plus configuring reduce_limit=false ...)
2.
Map: [group,associated], 1
Reduce: sum(values)
Here I cannot retrieve multiple groups at once, so I use a request per
desired group.
Tried the two approaches, with the first one gives faster response. Which
leads me to two questions:
1. Is there any risk in a reduce function that produces a potentially long
string?
2. Is there a better way to achieve what I do here?
Thanks!
Boaz
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
Right... In your query it seems down to the milliseconds... Is that what
you want?... So you want to be able to query between to totally arbitrary
fates down to the millisecond. Is that right?
On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> I need to filter by date and group by group...
>
> For example having these docs:
> {
> "group" : "a",
> "associated" : "2014-10-04T21:58:59.377Z",
> ...
> }
> {
> "group" : "a",
> "associated" : "2014-10-03T21:58:59.377Z",
> ...
> }
> {
> "group" : "b",
> "associated" : "2014-10-04T21:58:59.377Z",
> ...
> }
> {
> "group" : "b",
> "associated" : "2014-10-01T21:58:59.377Z",
> ...
> }
>
> I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
> to-date= "2014-10-04T22:58:59.377Z"
>
> And the result will be -
>
> a, 2
> b, 1
>
>
>
>
> On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com>
> wrote:
>
> > Right ...are you saying... That you need grouping down to the day?... In
> > which case you can just emit[ date, group]... The fate being the
> > milliseconds truncated to the day and just have a reduce that's
> _count....
> > By the way the date can also be broken down to [year, month, day, group]
> >
> > You would need to change the group_level to 4 instead of 2 bit the view
> > would be human readable.
> >
> > The truck here is that you query with start key ,end_key to get the the
> > days and re reduce those values again for that arbitrary date range in a
> > reduce function and you should be good
> > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >
> > > I need to get the count of document that were associated to a group
> > between
> > > two given dates. Thanks!
> > >
> > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <si...@gmail.com>
> > > wrote:
> > >
> > > > Hey what date are you looking to filter to? Day/ month..year?
> > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > > >
> > > > > Thanks Giovanni,
> > > > > You say I can get all the groups at the same time,
> > > > > but how can I achieve this and also filter by date?
> > > > >
> > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com>
> > wrote:
> > > > >
> > > > > > You can use the second with group_level=1 and get all the groups
> at
> > > the
> > > > > > same time.
> > > > > > And you can use _count instead of _sum, so you don't even need to
> > > emit
> > > > > any
> > > > > > value, just the key.
> > > > > >
> > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > My documents contain two fields to maintain group associations,
> > say
> > > > > > "group"
> > > > > > > holds the group document id, and "associated" holds the date
> this
> > > > > > document
> > > > > > > was added to the group.
> > > > > > > Now I want to be able to know how many documents were added to
> a
> > > > given
> > > > > > > group[s] between two given dates.
> > > > > > > The challenge is that to be able to filter by dates, I need to
> > have
> > > > the
> > > > > > > date as the key first part.
> > > > > > > But I also need the group as the first key part in order to
> > > aggregate
> > > > > the
> > > > > > > number of group associations.
> > > > > > >
> > > > > > > So I see two options here:
> > > > > > >
> > > > > > > 1.
> > > > > > > Map: associated, {"group": group}
> > > > > > > Reduce: a function that aggregates all values by group, which I
> > > > assume
> > > > > is
> > > > > > > fine as I know the number of groups is relatively small.
> > > > > > > (plus configuring reduce_limit=false ...)
> > > > > > >
> > > > > > > 2.
> > > > > > > Map: [group,associated], 1
> > > > > > > Reduce: sum(values)
> > > > > > > Here I cannot retrieve multiple groups at once, so I use a
> > request
> > > > per
> > > > > > > desired group.
> > > > > > >
> > > > > > > Tried the two approaches, with the first one gives faster
> > response.
> > > > > Which
> > > > > > > leads me to two questions:
> > > > > > > 1. Is there any risk in a reduce function that produces a
> > > potentially
> > > > > > long
> > > > > > > string?
> > > > > > > 2. Is there a better way to achieve what I do here?
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Boaz
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
chaining map reduce sounds really exciting....but for your current
solution...list functions would be the way to go
On Sun, Oct 5, 2014 at 10:03 AM, Giovanni P <fi...@gmail.com> wrote:
> as I always have trouble imagining the results of a some reduce function
> with group_level and stuff alike, I made this app for quickly simulating
> these things: http://fiatjaf.github.io/couchdb-mr-simulator/
>
> here's the same simulation Aurélien did:
>
> http://fiatjaf.github.io/couchdb-mr-simulator/#{%22params%22:%22reduceFn:%20_count\nreduce:%20true\ngroup:%20true\ngroup_level:%202\nstartkey:%20[2013]\nendkey:%20[2014,%20{}]%22,%22emitted%22:%222012,%20\%22A\%22\n2012,%20\%22B\%22\n2012,%20\%22C\%22\n2013,%20\%22A\%22\n2013,%20\%22A\%22\n2013,%20\%22A\%22\n2013,%20\%22C\%22\n2014,%20\%22A\%22\n2014,%20\%22A\%22\n2014,%20\%22B\%22%22}
> <
> http://fiatjaf.github.io/couchdb-mr-simulator/#%7B%22params%22:%22reduceFn:%20_count%5Cnreduce:%20true%5Cngroup:%20true%5Cngroup_level:%202%5Cnstartkey:%20[2013]%5Cnendkey:%20[2014,%20%7B%7D]%22,%22emitted%22:%222012,%20%5C%22A%5C%22%5Cn2012,%20%5C%22B%5C%22%5Cn2012,%20%5C%22C%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22C%5C%22%5Cn2014,%20%5C%22A%5C%22%5Cn2014,%20%5C%22A%5C%22%5Cn2014,%20%5C%22B%5C%22%22%7D
> >
>
> On Sun, Oct 5, 2014 at 9:05 AM, Aurélien Bénel <au...@utt.fr>
> wrote:
>
> > > Thanks Aurélien for a better explanation of my problem :-)
> >
> > Good to hear :)
> >
> > For the additional computation step, you can:
> >
> > 1. switch to CloudAnt's BigCouch to get chained mapreduce
> > http://examples.cloudant.com/sales/_design/sales/index.html
> >
> > 2. or wait for BigCouch integration into Apache CouchDB (in version 2.0)
> > https://issues.apache.org/jira/browse/COUCHDB/fixforversion/12315572
> >
> > 3. or "cheat" and do it with a list.
> >
> >
> > Regards,
> >
> > Aurélien
> >
> >
> > >>>> I need to filter by date and group by group...
> > >>> You can use the second with group_level=1 and get all the groups at
> the
> > >> same time. And you can use _count
> > >>
> > >>
> > >> I'm afraid this is not so easy...
> > >> Using a filter by date will also *group* by date.
> > >>
> > >> Let's take a very simple example...
> > >>
> > >> Here are keys emitted by a map:
> > >> [2012, 'A']
> > >> [2012, 'B']
> > >> [2012, 'C']
> > >> [2013, 'A']
> > >> [2013, 'A']
> > >> [2013, 'A']
> > >> [2013, 'C']
> > >> [2014, 'A']
> > >> [2014, 'A']
> > >> [2014, 'B']
> > >>
> > >> If we query it with `_count` as the reduce function, `group=true`,
> > >> `startkey=[2013]` and `endkey=[2014,{}]`, we'll get:
> > >> [2013, 'A'] -> 3
> > >> [2013, 'C'] -> 1
> > >> [2014, 'A'] -> 2
> > >> [2014, 'B'] -> 1
> > >>
> > >> Boaz will need additional computation to get:
> > >> 'A' -> 5
> > >> 'B' -> 1
> > >> 'C' -> 1
> > >>
> > >>
> > >> Regards,
> > >>
> > >> Aurélien
> >
> >
>
Re: View question
Posted by Giovanni P <fi...@gmail.com>.
as I always have trouble imagining the results of a some reduce function
with group_level and stuff alike, I made this app for quickly simulating
these things: http://fiatjaf.github.io/couchdb-mr-simulator/
here's the same simulation Aurélien did:
http://fiatjaf.github.io/couchdb-mr-simulator/#{%22params%22:%22reduceFn:%20_count\nreduce:%20true\ngroup:%20true\ngroup_level:%202\nstartkey:%20[2013]\nendkey:%20[2014,%20{}]%22,%22emitted%22:%222012,%20\%22A\%22\n2012,%20\%22B\%22\n2012,%20\%22C\%22\n2013,%20\%22A\%22\n2013,%20\%22A\%22\n2013,%20\%22A\%22\n2013,%20\%22C\%22\n2014,%20\%22A\%22\n2014,%20\%22A\%22\n2014,%20\%22B\%22%22}
<http://fiatjaf.github.io/couchdb-mr-simulator/#%7B%22params%22:%22reduceFn:%20_count%5Cnreduce:%20true%5Cngroup:%20true%5Cngroup_level:%202%5Cnstartkey:%20[2013]%5Cnendkey:%20[2014,%20%7B%7D]%22,%22emitted%22:%222012,%20%5C%22A%5C%22%5Cn2012,%20%5C%22B%5C%22%5Cn2012,%20%5C%22C%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22A%5C%22%5Cn2013,%20%5C%22C%5C%22%5Cn2014,%20%5C%22A%5C%22%5Cn2014,%20%5C%22A%5C%22%5Cn2014,%20%5C%22B%5C%22%22%7D>
On Sun, Oct 5, 2014 at 9:05 AM, Aurélien Bénel <au...@utt.fr>
wrote:
> > Thanks Aurélien for a better explanation of my problem :-)
>
> Good to hear :)
>
> For the additional computation step, you can:
>
> 1. switch to CloudAnt's BigCouch to get chained mapreduce
> http://examples.cloudant.com/sales/_design/sales/index.html
>
> 2. or wait for BigCouch integration into Apache CouchDB (in version 2.0)
> https://issues.apache.org/jira/browse/COUCHDB/fixforversion/12315572
>
> 3. or "cheat" and do it with a list.
>
>
> Regards,
>
> Aurélien
>
>
> >>>> I need to filter by date and group by group...
> >>> You can use the second with group_level=1 and get all the groups at the
> >> same time. And you can use _count
> >>
> >>
> >> I'm afraid this is not so easy...
> >> Using a filter by date will also *group* by date.
> >>
> >> Let's take a very simple example...
> >>
> >> Here are keys emitted by a map:
> >> [2012, 'A']
> >> [2012, 'B']
> >> [2012, 'C']
> >> [2013, 'A']
> >> [2013, 'A']
> >> [2013, 'A']
> >> [2013, 'C']
> >> [2014, 'A']
> >> [2014, 'A']
> >> [2014, 'B']
> >>
> >> If we query it with `_count` as the reduce function, `group=true`,
> >> `startkey=[2013]` and `endkey=[2014,{}]`, we'll get:
> >> [2013, 'A'] -> 3
> >> [2013, 'C'] -> 1
> >> [2014, 'A'] -> 2
> >> [2014, 'B'] -> 1
> >>
> >> Boaz will need additional computation to get:
> >> 'A' -> 5
> >> 'B' -> 1
> >> 'C' -> 1
> >>
> >>
> >> Regards,
> >>
> >> Aurélien
>
>
Re: View question
Posted by Aurélien Bénel <au...@utt.fr>.
> Thanks Aurélien for a better explanation of my problem :-)
Good to hear :)
For the additional computation step, you can:
1. switch to CloudAnt's BigCouch to get chained mapreduce
http://examples.cloudant.com/sales/_design/sales/index.html
2. or wait for BigCouch integration into Apache CouchDB (in version 2.0)
https://issues.apache.org/jira/browse/COUCHDB/fixforversion/12315572
3. or "cheat" and do it with a list.
Regards,
Aurélien
>>>> I need to filter by date and group by group...
>>> You can use the second with group_level=1 and get all the groups at the
>> same time. And you can use _count
>>
>>
>> I'm afraid this is not so easy...
>> Using a filter by date will also *group* by date.
>>
>> Let's take a very simple example...
>>
>> Here are keys emitted by a map:
>> [2012, 'A']
>> [2012, 'B']
>> [2012, 'C']
>> [2013, 'A']
>> [2013, 'A']
>> [2013, 'A']
>> [2013, 'C']
>> [2014, 'A']
>> [2014, 'A']
>> [2014, 'B']
>>
>> If we query it with `_count` as the reduce function, `group=true`,
>> `startkey=[2013]` and `endkey=[2014,{}]`, we'll get:
>> [2013, 'A'] -> 3
>> [2013, 'C'] -> 1
>> [2014, 'A'] -> 2
>> [2014, 'B'] -> 1
>>
>> Boaz will need additional computation to get:
>> 'A' -> 5
>> 'B' -> 1
>> 'C' -> 1
>>
>>
>> Regards,
>>
>> Aurélien
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
Thanks Aurélien for a better explanation of my problem :-)
On Sun, Oct 5, 2014 at 1:01 PM, Aurélien Bénel <au...@utt.fr>
wrote:
> >> I need to filter by date and group by group...
> > You can use the second with group_level=1 and get all the groups at the
> same time. And you can use _count
>
>
> I'm afraid this is not so easy...
> Using a filter by date will also *group* by date.
>
> Let's take a very simple example...
>
> Here are keys emitted by a map:
> [2012, 'A']
> [2012, 'B']
> [2012, 'C']
> [2013, 'A']
> [2013, 'A']
> [2013, 'A']
> [2013, 'C']
> [2014, 'A']
> [2014, 'A']
> [2014, 'B']
>
> If we query it with `_count` as the reduce function, `group=true`,
> `startkey=[2013]` and `endkey=[2014,{}]`, we'll get:
> [2013, 'A'] -> 3
> [2013, 'C'] -> 1
> [2014, 'A'] -> 2
> [2014, 'B'] -> 1
>
> Boaz will need additional computation to get:
> 'A' -> 5
> 'B' -> 1
> 'C' -> 1
>
>
> Regards,
>
> Aurélien
Re: View question
Posted by Aurélien Bénel <au...@utt.fr>.
>> I need to filter by date and group by group...
> You can use the second with group_level=1 and get all the groups at the same time. And you can use _count
I'm afraid this is not so easy...
Using a filter by date will also *group* by date.
Let's take a very simple example...
Here are keys emitted by a map:
[2012, 'A']
[2012, 'B']
[2012, 'C']
[2013, 'A']
[2013, 'A']
[2013, 'A']
[2013, 'C']
[2014, 'A']
[2014, 'A']
[2014, 'B']
If we query it with `_count` as the reduce function, `group=true`, `startkey=[2013]` and `endkey=[2014,{}]`, we'll get:
[2013, 'A'] -> 3
[2013, 'C'] -> 1
[2014, 'A'] -> 2
[2014, 'B'] -> 1
Boaz will need additional computation to get:
'A' -> 5
'B' -> 1
'C' -> 1
Regards,
Aurélien
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
ok so you suggest using a list function to aggregate the totals per group...
do you think this will give a better performance then a reduce that does
the same?
On Sun, Oct 5, 2014 at 11:04 AM, Stanley Iriele <si...@gmail.com>
wrote:
> Comparing numbers is much faster and cheaper space wise than strings
> On Oct 5, 2014 1:04 AM, "Stanley Iriele" <si...@gmail.com> wrote:
>
> > You would need a view... As I described earlier.... But BE SURE TO store
> > the UTC milliseconds on the doc...what...or use the getTime() function to
> > emit [ seconds, group]... And then your reduce would be _count.... Lastly
> > you will need to write a list function that aggregates over your results
> on
> > last time to get your desired output
> > On Oct 5, 2014 12:33 AM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >
> >> how could I solve it if let's say it would be limited to the second?
> >>
> >> On Sun, Oct 5, 2014 at 10:07 AM, Stanley Iriele <si...@gmail.com>
> >> wrote:
> >>
> >> > Or can it be down to say....the hour... Or minute... Something like
> that
> >> > On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >> >
> >> > > I need to filter by date and group by group...
> >> > >
> >> > > For example having these docs:
> >> > > {
> >> > > "group" : "a",
> >> > > "associated" : "2014-10-04T21:58:59.377Z",
> >> > > ...
> >> > > }
> >> > > {
> >> > > "group" : "a",
> >> > > "associated" : "2014-10-03T21:58:59.377Z",
> >> > > ...
> >> > > }
> >> > > {
> >> > > "group" : "b",
> >> > > "associated" : "2014-10-04T21:58:59.377Z",
> >> > > ...
> >> > > }
> >> > > {
> >> > > "group" : "b",
> >> > > "associated" : "2014-10-01T21:58:59.377Z",
> >> > > ...
> >> > > }
> >> > >
> >> > > I want to support a query with from-date =
> "2014-10-02T21:58:59.377Z"
> >> > > to-date= "2014-10-04T22:58:59.377Z"
> >> > >
> >> > > And the result will be -
> >> > >
> >> > > a, 2
> >> > > b, 1
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <
> siriele2x3@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Right ...are you saying... That you need grouping down to the
> >> day?...
> >> > In
> >> > > > which case you can just emit[ date, group]... The fate being the
> >> > > > milliseconds truncated to the day and just have a reduce that's
> >> > > _count....
> >> > > > By the way the date can also be broken down to [year, month, day,
> >> > group]
> >> > > >
> >> > > > You would need to change the group_level to 4 instead of 2 bit the
> >> view
> >> > > > would be human readable.
> >> > > >
> >> > > > The truck here is that you query with start key ,end_key to get
> the
> >> the
> >> > > > days and re reduce those values again for that arbitrary date
> range
> >> in
> >> > a
> >> > > > reduce function and you should be good
> >> > > > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >> > > >
> >> > > > > I need to get the count of document that were associated to a
> >> group
> >> > > > between
> >> > > > > two given dates. Thanks!
> >> > > > >
> >> > > > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <
> >> siriele2x3@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hey what date are you looking to filter to? Day/ month..year?
> >> > > > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com>
> >> wrote:
> >> > > > > >
> >> > > > > > > Thanks Giovanni,
> >> > > > > > > You say I can get all the groups at the same time,
> >> > > > > > > but how can I achieve this and also filter by date?
> >> > > > > > >
> >> > > > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <
> fiatjaf@gmail.com
> >> >
> >> > > > wrote:
> >> > > > > > >
> >> > > > > > > > You can use the second with group_level=1 and get all the
> >> > groups
> >> > > at
> >> > > > > the
> >> > > > > > > > same time.
> >> > > > > > > > And you can use _count instead of _sum, so you don't even
> >> need
> >> > to
> >> > > > > emit
> >> > > > > > > any
> >> > > > > > > > value, just the key.
> >> > > > > > > >
> >> > > > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <
> >> bcitrin@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hello,
> >> > > > > > > > >
> >> > > > > > > > > My documents contain two fields to maintain group
> >> > associations,
> >> > > > say
> >> > > > > > > > "group"
> >> > > > > > > > > holds the group document id, and "associated" holds the
> >> date
> >> > > this
> >> > > > > > > > document
> >> > > > > > > > > was added to the group.
> >> > > > > > > > > Now I want to be able to know how many documents were
> >> added
> >> > to
> >> > > a
> >> > > > > > given
> >> > > > > > > > > group[s] between two given dates.
> >> > > > > > > > > The challenge is that to be able to filter by dates, I
> >> need
> >> > to
> >> > > > have
> >> > > > > > the
> >> > > > > > > > > date as the key first part.
> >> > > > > > > > > But I also need the group as the first key part in order
> >> to
> >> > > > > aggregate
> >> > > > > > > the
> >> > > > > > > > > number of group associations.
> >> > > > > > > > >
> >> > > > > > > > > So I see two options here:
> >> > > > > > > > >
> >> > > > > > > > > 1.
> >> > > > > > > > > Map: associated, {"group": group}
> >> > > > > > > > > Reduce: a function that aggregates all values by group,
> >> > which I
> >> > > > > > assume
> >> > > > > > > is
> >> > > > > > > > > fine as I know the number of groups is relatively small.
> >> > > > > > > > > (plus configuring reduce_limit=false ...)
> >> > > > > > > > >
> >> > > > > > > > > 2.
> >> > > > > > > > > Map: [group,associated], 1
> >> > > > > > > > > Reduce: sum(values)
> >> > > > > > > > > Here I cannot retrieve multiple groups at once, so I
> use a
> >> > > > request
> >> > > > > > per
> >> > > > > > > > > desired group.
> >> > > > > > > > >
> >> > > > > > > > > Tried the two approaches, with the first one gives
> faster
> >> > > > response.
> >> > > > > > > Which
> >> > > > > > > > > leads me to two questions:
> >> > > > > > > > > 1. Is there any risk in a reduce function that produces
> a
> >> > > > > potentially
> >> > > > > > > > long
> >> > > > > > > > > string?
> >> > > > > > > > > 2. Is there a better way to achieve what I do here?
> >> > > > > > > > >
> >> > > > > > > > > Thanks!
> >> > > > > > > > >
> >> > > > > > > > > Boaz
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
Comparing numbers is much faster and cheaper space wise than strings
On Oct 5, 2014 1:04 AM, "Stanley Iriele" <si...@gmail.com> wrote:
> You would need a view... As I described earlier.... But BE SURE TO store
> the UTC milliseconds on the doc...what...or use the getTime() function to
> emit [ seconds, group]... And then your reduce would be _count.... Lastly
> you will need to write a list function that aggregates over your results on
> last time to get your desired output
> On Oct 5, 2014 12:33 AM, "Boaz Citrin" <bc...@gmail.com> wrote:
>
>> how could I solve it if let's say it would be limited to the second?
>>
>> On Sun, Oct 5, 2014 at 10:07 AM, Stanley Iriele <si...@gmail.com>
>> wrote:
>>
>> > Or can it be down to say....the hour... Or minute... Something like that
>> > On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
>> >
>> > > I need to filter by date and group by group...
>> > >
>> > > For example having these docs:
>> > > {
>> > > "group" : "a",
>> > > "associated" : "2014-10-04T21:58:59.377Z",
>> > > ...
>> > > }
>> > > {
>> > > "group" : "a",
>> > > "associated" : "2014-10-03T21:58:59.377Z",
>> > > ...
>> > > }
>> > > {
>> > > "group" : "b",
>> > > "associated" : "2014-10-04T21:58:59.377Z",
>> > > ...
>> > > }
>> > > {
>> > > "group" : "b",
>> > > "associated" : "2014-10-01T21:58:59.377Z",
>> > > ...
>> > > }
>> > >
>> > > I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
>> > > to-date= "2014-10-04T22:58:59.377Z"
>> > >
>> > > And the result will be -
>> > >
>> > > a, 2
>> > > b, 1
>> > >
>> > >
>> > >
>> > >
>> > > On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com>
>> > > wrote:
>> > >
>> > > > Right ...are you saying... That you need grouping down to the
>> day?...
>> > In
>> > > > which case you can just emit[ date, group]... The fate being the
>> > > > milliseconds truncated to the day and just have a reduce that's
>> > > _count....
>> > > > By the way the date can also be broken down to [year, month, day,
>> > group]
>> > > >
>> > > > You would need to change the group_level to 4 instead of 2 bit the
>> view
>> > > > would be human readable.
>> > > >
>> > > > The truck here is that you query with start key ,end_key to get the
>> the
>> > > > days and re reduce those values again for that arbitrary date range
>> in
>> > a
>> > > > reduce function and you should be good
>> > > > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
>> > > >
>> > > > > I need to get the count of document that were associated to a
>> group
>> > > > between
>> > > > > two given dates. Thanks!
>> > > > >
>> > > > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <
>> siriele2x3@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hey what date are you looking to filter to? Day/ month..year?
>> > > > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com>
>> wrote:
>> > > > > >
>> > > > > > > Thanks Giovanni,
>> > > > > > > You say I can get all the groups at the same time,
>> > > > > > > but how can I achieve this and also filter by date?
>> > > > > > >
>> > > > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fiatjaf@gmail.com
>> >
>> > > > wrote:
>> > > > > > >
>> > > > > > > > You can use the second with group_level=1 and get all the
>> > groups
>> > > at
>> > > > > the
>> > > > > > > > same time.
>> > > > > > > > And you can use _count instead of _sum, so you don't even
>> need
>> > to
>> > > > > emit
>> > > > > > > any
>> > > > > > > > value, just the key.
>> > > > > > > >
>> > > > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <
>> bcitrin@gmail.com
>> > >
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hello,
>> > > > > > > > >
>> > > > > > > > > My documents contain two fields to maintain group
>> > associations,
>> > > > say
>> > > > > > > > "group"
>> > > > > > > > > holds the group document id, and "associated" holds the
>> date
>> > > this
>> > > > > > > > document
>> > > > > > > > > was added to the group.
>> > > > > > > > > Now I want to be able to know how many documents were
>> added
>> > to
>> > > a
>> > > > > > given
>> > > > > > > > > group[s] between two given dates.
>> > > > > > > > > The challenge is that to be able to filter by dates, I
>> need
>> > to
>> > > > have
>> > > > > > the
>> > > > > > > > > date as the key first part.
>> > > > > > > > > But I also need the group as the first key part in order
>> to
>> > > > > aggregate
>> > > > > > > the
>> > > > > > > > > number of group associations.
>> > > > > > > > >
>> > > > > > > > > So I see two options here:
>> > > > > > > > >
>> > > > > > > > > 1.
>> > > > > > > > > Map: associated, {"group": group}
>> > > > > > > > > Reduce: a function that aggregates all values by group,
>> > which I
>> > > > > > assume
>> > > > > > > is
>> > > > > > > > > fine as I know the number of groups is relatively small.
>> > > > > > > > > (plus configuring reduce_limit=false ...)
>> > > > > > > > >
>> > > > > > > > > 2.
>> > > > > > > > > Map: [group,associated], 1
>> > > > > > > > > Reduce: sum(values)
>> > > > > > > > > Here I cannot retrieve multiple groups at once, so I use a
>> > > > request
>> > > > > > per
>> > > > > > > > > desired group.
>> > > > > > > > >
>> > > > > > > > > Tried the two approaches, with the first one gives faster
>> > > > response.
>> > > > > > > Which
>> > > > > > > > > leads me to two questions:
>> > > > > > > > > 1. Is there any risk in a reduce function that produces a
>> > > > > potentially
>> > > > > > > > long
>> > > > > > > > > string?
>> > > > > > > > > 2. Is there a better way to achieve what I do here?
>> > > > > > > > >
>> > > > > > > > > Thanks!
>> > > > > > > > >
>> > > > > > > > > Boaz
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
You would need a view... As I described earlier.... But BE SURE TO store
the UTC milliseconds on the doc...what...or use the getTime() function to
emit [ seconds, group]... And then your reduce would be _count.... Lastly
you will need to write a list function that aggregates over your results on
last time to get your desired output
On Oct 5, 2014 12:33 AM, "Boaz Citrin" <bc...@gmail.com> wrote:
> how could I solve it if let's say it would be limited to the second?
>
> On Sun, Oct 5, 2014 at 10:07 AM, Stanley Iriele <si...@gmail.com>
> wrote:
>
> > Or can it be down to say....the hour... Or minute... Something like that
> > On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >
> > > I need to filter by date and group by group...
> > >
> > > For example having these docs:
> > > {
> > > "group" : "a",
> > > "associated" : "2014-10-04T21:58:59.377Z",
> > > ...
> > > }
> > > {
> > > "group" : "a",
> > > "associated" : "2014-10-03T21:58:59.377Z",
> > > ...
> > > }
> > > {
> > > "group" : "b",
> > > "associated" : "2014-10-04T21:58:59.377Z",
> > > ...
> > > }
> > > {
> > > "group" : "b",
> > > "associated" : "2014-10-01T21:58:59.377Z",
> > > ...
> > > }
> > >
> > > I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
> > > to-date= "2014-10-04T22:58:59.377Z"
> > >
> > > And the result will be -
> > >
> > > a, 2
> > > b, 1
> > >
> > >
> > >
> > >
> > > On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com>
> > > wrote:
> > >
> > > > Right ...are you saying... That you need grouping down to the day?...
> > In
> > > > which case you can just emit[ date, group]... The fate being the
> > > > milliseconds truncated to the day and just have a reduce that's
> > > _count....
> > > > By the way the date can also be broken down to [year, month, day,
> > group]
> > > >
> > > > You would need to change the group_level to 4 instead of 2 bit the
> view
> > > > would be human readable.
> > > >
> > > > The truck here is that you query with start key ,end_key to get the
> the
> > > > days and re reduce those values again for that arbitrary date range
> in
> > a
> > > > reduce function and you should be good
> > > > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > > >
> > > > > I need to get the count of document that were associated to a group
> > > > between
> > > > > two given dates. Thanks!
> > > > >
> > > > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <
> siriele2x3@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hey what date are you looking to filter to? Day/ month..year?
> > > > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com>
> wrote:
> > > > > >
> > > > > > > Thanks Giovanni,
> > > > > > > You say I can get all the groups at the same time,
> > > > > > > but how can I achieve this and also filter by date?
> > > > > > >
> > > > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > You can use the second with group_level=1 and get all the
> > groups
> > > at
> > > > > the
> > > > > > > > same time.
> > > > > > > > And you can use _count instead of _sum, so you don't even
> need
> > to
> > > > > emit
> > > > > > > any
> > > > > > > > value, just the key.
> > > > > > > >
> > > > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <
> bcitrin@gmail.com
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > My documents contain two fields to maintain group
> > associations,
> > > > say
> > > > > > > > "group"
> > > > > > > > > holds the group document id, and "associated" holds the
> date
> > > this
> > > > > > > > document
> > > > > > > > > was added to the group.
> > > > > > > > > Now I want to be able to know how many documents were added
> > to
> > > a
> > > > > > given
> > > > > > > > > group[s] between two given dates.
> > > > > > > > > The challenge is that to be able to filter by dates, I need
> > to
> > > > have
> > > > > > the
> > > > > > > > > date as the key first part.
> > > > > > > > > But I also need the group as the first key part in order to
> > > > > aggregate
> > > > > > > the
> > > > > > > > > number of group associations.
> > > > > > > > >
> > > > > > > > > So I see two options here:
> > > > > > > > >
> > > > > > > > > 1.
> > > > > > > > > Map: associated, {"group": group}
> > > > > > > > > Reduce: a function that aggregates all values by group,
> > which I
> > > > > > assume
> > > > > > > is
> > > > > > > > > fine as I know the number of groups is relatively small.
> > > > > > > > > (plus configuring reduce_limit=false ...)
> > > > > > > > >
> > > > > > > > > 2.
> > > > > > > > > Map: [group,associated], 1
> > > > > > > > > Reduce: sum(values)
> > > > > > > > > Here I cannot retrieve multiple groups at once, so I use a
> > > > request
> > > > > > per
> > > > > > > > > desired group.
> > > > > > > > >
> > > > > > > > > Tried the two approaches, with the first one gives faster
> > > > response.
> > > > > > > Which
> > > > > > > > > leads me to two questions:
> > > > > > > > > 1. Is there any risk in a reduce function that produces a
> > > > > potentially
> > > > > > > > long
> > > > > > > > > string?
> > > > > > > > > 2. Is there a better way to achieve what I do here?
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Boaz
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
how could I solve it if let's say it would be limited to the second?
On Sun, Oct 5, 2014 at 10:07 AM, Stanley Iriele <si...@gmail.com>
wrote:
> Or can it be down to say....the hour... Or minute... Something like that
> On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
>
> > I need to filter by date and group by group...
> >
> > For example having these docs:
> > {
> > "group" : "a",
> > "associated" : "2014-10-04T21:58:59.377Z",
> > ...
> > }
> > {
> > "group" : "a",
> > "associated" : "2014-10-03T21:58:59.377Z",
> > ...
> > }
> > {
> > "group" : "b",
> > "associated" : "2014-10-04T21:58:59.377Z",
> > ...
> > }
> > {
> > "group" : "b",
> > "associated" : "2014-10-01T21:58:59.377Z",
> > ...
> > }
> >
> > I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
> > to-date= "2014-10-04T22:58:59.377Z"
> >
> > And the result will be -
> >
> > a, 2
> > b, 1
> >
> >
> >
> >
> > On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com>
> > wrote:
> >
> > > Right ...are you saying... That you need grouping down to the day?...
> In
> > > which case you can just emit[ date, group]... The fate being the
> > > milliseconds truncated to the day and just have a reduce that's
> > _count....
> > > By the way the date can also be broken down to [year, month, day,
> group]
> > >
> > > You would need to change the group_level to 4 instead of 2 bit the view
> > > would be human readable.
> > >
> > > The truck here is that you query with start key ,end_key to get the the
> > > days and re reduce those values again for that arbitrary date range in
> a
> > > reduce function and you should be good
> > > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > >
> > > > I need to get the count of document that were associated to a group
> > > between
> > > > two given dates. Thanks!
> > > >
> > > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <siriele2x3@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hey what date are you looking to filter to? Day/ month..year?
> > > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > > > >
> > > > > > Thanks Giovanni,
> > > > > > You say I can get all the groups at the same time,
> > > > > > but how can I achieve this and also filter by date?
> > > > > >
> > > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > You can use the second with group_level=1 and get all the
> groups
> > at
> > > > the
> > > > > > > same time.
> > > > > > > And you can use _count instead of _sum, so you don't even need
> to
> > > > emit
> > > > > > any
> > > > > > > value, just the key.
> > > > > > >
> > > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bcitrin@gmail.com
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > My documents contain two fields to maintain group
> associations,
> > > say
> > > > > > > "group"
> > > > > > > > holds the group document id, and "associated" holds the date
> > this
> > > > > > > document
> > > > > > > > was added to the group.
> > > > > > > > Now I want to be able to know how many documents were added
> to
> > a
> > > > > given
> > > > > > > > group[s] between two given dates.
> > > > > > > > The challenge is that to be able to filter by dates, I need
> to
> > > have
> > > > > the
> > > > > > > > date as the key first part.
> > > > > > > > But I also need the group as the first key part in order to
> > > > aggregate
> > > > > > the
> > > > > > > > number of group associations.
> > > > > > > >
> > > > > > > > So I see two options here:
> > > > > > > >
> > > > > > > > 1.
> > > > > > > > Map: associated, {"group": group}
> > > > > > > > Reduce: a function that aggregates all values by group,
> which I
> > > > > assume
> > > > > > is
> > > > > > > > fine as I know the number of groups is relatively small.
> > > > > > > > (plus configuring reduce_limit=false ...)
> > > > > > > >
> > > > > > > > 2.
> > > > > > > > Map: [group,associated], 1
> > > > > > > > Reduce: sum(values)
> > > > > > > > Here I cannot retrieve multiple groups at once, so I use a
> > > request
> > > > > per
> > > > > > > > desired group.
> > > > > > > >
> > > > > > > > Tried the two approaches, with the first one gives faster
> > > response.
> > > > > > Which
> > > > > > > > leads me to two questions:
> > > > > > > > 1. Is there any risk in a reduce function that produces a
> > > > potentially
> > > > > > > long
> > > > > > > > string?
> > > > > > > > 2. Is there a better way to achieve what I do here?
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Boaz
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
Or can it be down to say....the hour... Or minute... Something like that
On Oct 4, 2014 11:53 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> I need to filter by date and group by group...
>
> For example having these docs:
> {
> "group" : "a",
> "associated" : "2014-10-04T21:58:59.377Z",
> ...
> }
> {
> "group" : "a",
> "associated" : "2014-10-03T21:58:59.377Z",
> ...
> }
> {
> "group" : "b",
> "associated" : "2014-10-04T21:58:59.377Z",
> ...
> }
> {
> "group" : "b",
> "associated" : "2014-10-01T21:58:59.377Z",
> ...
> }
>
> I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
> to-date= "2014-10-04T22:58:59.377Z"
>
> And the result will be -
>
> a, 2
> b, 1
>
>
>
>
> On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com>
> wrote:
>
> > Right ...are you saying... That you need grouping down to the day?... In
> > which case you can just emit[ date, group]... The fate being the
> > milliseconds truncated to the day and just have a reduce that's
> _count....
> > By the way the date can also be broken down to [year, month, day, group]
> >
> > You would need to change the group_level to 4 instead of 2 bit the view
> > would be human readable.
> >
> > The truck here is that you query with start key ,end_key to get the the
> > days and re reduce those values again for that arbitrary date range in a
> > reduce function and you should be good
> > On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >
> > > I need to get the count of document that were associated to a group
> > between
> > > two given dates. Thanks!
> > >
> > > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <si...@gmail.com>
> > > wrote:
> > >
> > > > Hey what date are you looking to filter to? Day/ month..year?
> > > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > > >
> > > > > Thanks Giovanni,
> > > > > You say I can get all the groups at the same time,
> > > > > but how can I achieve this and also filter by date?
> > > > >
> > > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com>
> > wrote:
> > > > >
> > > > > > You can use the second with group_level=1 and get all the groups
> at
> > > the
> > > > > > same time.
> > > > > > And you can use _count instead of _sum, so you don't even need to
> > > emit
> > > > > any
> > > > > > value, just the key.
> > > > > >
> > > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > My documents contain two fields to maintain group associations,
> > say
> > > > > > "group"
> > > > > > > holds the group document id, and "associated" holds the date
> this
> > > > > > document
> > > > > > > was added to the group.
> > > > > > > Now I want to be able to know how many documents were added to
> a
> > > > given
> > > > > > > group[s] between two given dates.
> > > > > > > The challenge is that to be able to filter by dates, I need to
> > have
> > > > the
> > > > > > > date as the key first part.
> > > > > > > But I also need the group as the first key part in order to
> > > aggregate
> > > > > the
> > > > > > > number of group associations.
> > > > > > >
> > > > > > > So I see two options here:
> > > > > > >
> > > > > > > 1.
> > > > > > > Map: associated, {"group": group}
> > > > > > > Reduce: a function that aggregates all values by group, which I
> > > > assume
> > > > > is
> > > > > > > fine as I know the number of groups is relatively small.
> > > > > > > (plus configuring reduce_limit=false ...)
> > > > > > >
> > > > > > > 2.
> > > > > > > Map: [group,associated], 1
> > > > > > > Reduce: sum(values)
> > > > > > > Here I cannot retrieve multiple groups at once, so I use a
> > request
> > > > per
> > > > > > > desired group.
> > > > > > >
> > > > > > > Tried the two approaches, with the first one gives faster
> > response.
> > > > > Which
> > > > > > > leads me to two questions:
> > > > > > > 1. Is there any risk in a reduce function that produces a
> > > potentially
> > > > > > long
> > > > > > > string?
> > > > > > > 2. Is there a better way to achieve what I do here?
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Boaz
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
I need to filter by date and group by group...
For example having these docs:
{
"group" : "a",
"associated" : "2014-10-04T21:58:59.377Z",
...
}
{
"group" : "a",
"associated" : "2014-10-03T21:58:59.377Z",
...
}
{
"group" : "b",
"associated" : "2014-10-04T21:58:59.377Z",
...
}
{
"group" : "b",
"associated" : "2014-10-01T21:58:59.377Z",
...
}
I want to support a query with from-date = "2014-10-02T21:58:59.377Z"
to-date= "2014-10-04T22:58:59.377Z"
And the result will be -
a, 2
b, 1
On Sun, Oct 5, 2014 at 9:36 AM, Stanley Iriele <si...@gmail.com> wrote:
> Right ...are you saying... That you need grouping down to the day?... In
> which case you can just emit[ date, group]... The fate being the
> milliseconds truncated to the day and just have a reduce that's _count....
> By the way the date can also be broken down to [year, month, day, group]
>
> You would need to change the group_level to 4 instead of 2 bit the view
> would be human readable.
>
> The truck here is that you query with start key ,end_key to get the the
> days and re reduce those values again for that arbitrary date range in a
> reduce function and you should be good
> On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
>
> > I need to get the count of document that were associated to a group
> between
> > two given dates. Thanks!
> >
> > On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <si...@gmail.com>
> > wrote:
> >
> > > Hey what date are you looking to filter to? Day/ month..year?
> > > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> > >
> > > > Thanks Giovanni,
> > > > You say I can get all the groups at the same time,
> > > > but how can I achieve this and also filter by date?
> > > >
> > > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com>
> wrote:
> > > >
> > > > > You can use the second with group_level=1 and get all the groups at
> > the
> > > > > same time.
> > > > > And you can use _count instead of _sum, so you don't even need to
> > emit
> > > > any
> > > > > value, just the key.
> > > > >
> > > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > My documents contain two fields to maintain group associations,
> say
> > > > > "group"
> > > > > > holds the group document id, and "associated" holds the date this
> > > > > document
> > > > > > was added to the group.
> > > > > > Now I want to be able to know how many documents were added to a
> > > given
> > > > > > group[s] between two given dates.
> > > > > > The challenge is that to be able to filter by dates, I need to
> have
> > > the
> > > > > > date as the key first part.
> > > > > > But I also need the group as the first key part in order to
> > aggregate
> > > > the
> > > > > > number of group associations.
> > > > > >
> > > > > > So I see two options here:
> > > > > >
> > > > > > 1.
> > > > > > Map: associated, {"group": group}
> > > > > > Reduce: a function that aggregates all values by group, which I
> > > assume
> > > > is
> > > > > > fine as I know the number of groups is relatively small.
> > > > > > (plus configuring reduce_limit=false ...)
> > > > > >
> > > > > > 2.
> > > > > > Map: [group,associated], 1
> > > > > > Reduce: sum(values)
> > > > > > Here I cannot retrieve multiple groups at once, so I use a
> request
> > > per
> > > > > > desired group.
> > > > > >
> > > > > > Tried the two approaches, with the first one gives faster
> response.
> > > > Which
> > > > > > leads me to two questions:
> > > > > > 1. Is there any risk in a reduce function that produces a
> > potentially
> > > > > long
> > > > > > string?
> > > > > > 2. Is there a better way to achieve what I do here?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Boaz
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
Right ...are you saying... That you need grouping down to the day?... In
which case you can just emit[ date, group]... The fate being the
milliseconds truncated to the day and just have a reduce that's _count....
By the way the date can also be broken down to [year, month, day, group]
You would need to change the group_level to 4 instead of 2 bit the view
would be human readable.
The truck here is that you query with start key ,end_key to get the the
days and re reduce those values again for that arbitrary date range in a
reduce function and you should be good
On Oct 4, 2014 11:23 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> I need to get the count of document that were associated to a group between
> two given dates. Thanks!
>
> On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <si...@gmail.com>
> wrote:
>
> > Hey what date are you looking to filter to? Day/ month..year?
> > On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> >
> > > Thanks Giovanni,
> > > You say I can get all the groups at the same time,
> > > but how can I achieve this and also filter by date?
> > >
> > > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com> wrote:
> > >
> > > > You can use the second with group_level=1 and get all the groups at
> the
> > > > same time.
> > > > And you can use _count instead of _sum, so you don't even need to
> emit
> > > any
> > > > value, just the key.
> > > >
> > > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com>
> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > My documents contain two fields to maintain group associations, say
> > > > "group"
> > > > > holds the group document id, and "associated" holds the date this
> > > > document
> > > > > was added to the group.
> > > > > Now I want to be able to know how many documents were added to a
> > given
> > > > > group[s] between two given dates.
> > > > > The challenge is that to be able to filter by dates, I need to have
> > the
> > > > > date as the key first part.
> > > > > But I also need the group as the first key part in order to
> aggregate
> > > the
> > > > > number of group associations.
> > > > >
> > > > > So I see two options here:
> > > > >
> > > > > 1.
> > > > > Map: associated, {"group": group}
> > > > > Reduce: a function that aggregates all values by group, which I
> > assume
> > > is
> > > > > fine as I know the number of groups is relatively small.
> > > > > (plus configuring reduce_limit=false ...)
> > > > >
> > > > > 2.
> > > > > Map: [group,associated], 1
> > > > > Reduce: sum(values)
> > > > > Here I cannot retrieve multiple groups at once, so I use a request
> > per
> > > > > desired group.
> > > > >
> > > > > Tried the two approaches, with the first one gives faster response.
> > > Which
> > > > > leads me to two questions:
> > > > > 1. Is there any risk in a reduce function that produces a
> potentially
> > > > long
> > > > > string?
> > > > > 2. Is there a better way to achieve what I do here?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Boaz
> > > > >
> > > >
> > >
> >
>
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
I need to get the count of document that were associated to a group between
two given dates. Thanks!
On Sun, Oct 5, 2014 at 8:13 AM, Stanley Iriele <si...@gmail.com> wrote:
> Hey what date are you looking to filter to? Day/ month..year?
> On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
>
> > Thanks Giovanni,
> > You say I can get all the groups at the same time,
> > but how can I achieve this and also filter by date?
> >
> > On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com> wrote:
> >
> > > You can use the second with group_level=1 and get all the groups at the
> > > same time.
> > > And you can use _count instead of _sum, so you don't even need to emit
> > any
> > > value, just the key.
> > >
> > > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > My documents contain two fields to maintain group associations, say
> > > "group"
> > > > holds the group document id, and "associated" holds the date this
> > > document
> > > > was added to the group.
> > > > Now I want to be able to know how many documents were added to a
> given
> > > > group[s] between two given dates.
> > > > The challenge is that to be able to filter by dates, I need to have
> the
> > > > date as the key first part.
> > > > But I also need the group as the first key part in order to aggregate
> > the
> > > > number of group associations.
> > > >
> > > > So I see two options here:
> > > >
> > > > 1.
> > > > Map: associated, {"group": group}
> > > > Reduce: a function that aggregates all values by group, which I
> assume
> > is
> > > > fine as I know the number of groups is relatively small.
> > > > (plus configuring reduce_limit=false ...)
> > > >
> > > > 2.
> > > > Map: [group,associated], 1
> > > > Reduce: sum(values)
> > > > Here I cannot retrieve multiple groups at once, so I use a request
> per
> > > > desired group.
> > > >
> > > > Tried the two approaches, with the first one gives faster response.
> > Which
> > > > leads me to two questions:
> > > > 1. Is there any risk in a reduce function that produces a potentially
> > > long
> > > > string?
> > > > 2. Is there a better way to achieve what I do here?
> > > >
> > > > Thanks!
> > > >
> > > > Boaz
> > > >
> > >
> >
>
Re: View question
Posted by Stanley Iriele <si...@gmail.com>.
Hey what date are you looking to filter to? Day/ month..year?
On Oct 4, 2014 10:11 PM, "Boaz Citrin" <bc...@gmail.com> wrote:
> Thanks Giovanni,
> You say I can get all the groups at the same time,
> but how can I achieve this and also filter by date?
>
> On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com> wrote:
>
> > You can use the second with group_level=1 and get all the groups at the
> > same time.
> > And you can use _count instead of _sum, so you don't even need to emit
> any
> > value, just the key.
> >
> > On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > My documents contain two fields to maintain group associations, say
> > "group"
> > > holds the group document id, and "associated" holds the date this
> > document
> > > was added to the group.
> > > Now I want to be able to know how many documents were added to a given
> > > group[s] between two given dates.
> > > The challenge is that to be able to filter by dates, I need to have the
> > > date as the key first part.
> > > But I also need the group as the first key part in order to aggregate
> the
> > > number of group associations.
> > >
> > > So I see two options here:
> > >
> > > 1.
> > > Map: associated, {"group": group}
> > > Reduce: a function that aggregates all values by group, which I assume
> is
> > > fine as I know the number of groups is relatively small.
> > > (plus configuring reduce_limit=false ...)
> > >
> > > 2.
> > > Map: [group,associated], 1
> > > Reduce: sum(values)
> > > Here I cannot retrieve multiple groups at once, so I use a request per
> > > desired group.
> > >
> > > Tried the two approaches, with the first one gives faster response.
> Which
> > > leads me to two questions:
> > > 1. Is there any risk in a reduce function that produces a potentially
> > long
> > > string?
> > > 2. Is there a better way to achieve what I do here?
> > >
> > > Thanks!
> > >
> > > Boaz
> > >
> >
>
Re: View question
Posted by Boaz Citrin <bc...@gmail.com>.
Thanks Giovanni,
You say I can get all the groups at the same time,
but how can I achieve this and also filter by date?
On Sun, Oct 5, 2014 at 4:04 AM, Giovanni P <fi...@gmail.com> wrote:
> You can use the second with group_level=1 and get all the groups at the
> same time.
> And you can use _count instead of _sum, so you don't even need to emit any
> value, just the key.
>
> On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com> wrote:
>
> > Hello,
> >
> > My documents contain two fields to maintain group associations, say
> "group"
> > holds the group document id, and "associated" holds the date this
> document
> > was added to the group.
> > Now I want to be able to know how many documents were added to a given
> > group[s] between two given dates.
> > The challenge is that to be able to filter by dates, I need to have the
> > date as the key first part.
> > But I also need the group as the first key part in order to aggregate the
> > number of group associations.
> >
> > So I see two options here:
> >
> > 1.
> > Map: associated, {"group": group}
> > Reduce: a function that aggregates all values by group, which I assume is
> > fine as I know the number of groups is relatively small.
> > (plus configuring reduce_limit=false ...)
> >
> > 2.
> > Map: [group,associated], 1
> > Reduce: sum(values)
> > Here I cannot retrieve multiple groups at once, so I use a request per
> > desired group.
> >
> > Tried the two approaches, with the first one gives faster response. Which
> > leads me to two questions:
> > 1. Is there any risk in a reduce function that produces a potentially
> long
> > string?
> > 2. Is there a better way to achieve what I do here?
> >
> > Thanks!
> >
> > Boaz
> >
>
Re: View question
Posted by Giovanni P <fi...@gmail.com>.
You can use the second with group_level=1 and get all the groups at the
same time.
And you can use _count instead of _sum, so you don't even need to emit any
value, just the key.
On Sat, Oct 4, 2014 at 8:24 PM, Boaz Citrin <bc...@gmail.com> wrote:
> Hello,
>
> My documents contain two fields to maintain group associations, say "group"
> holds the group document id, and "associated" holds the date this document
> was added to the group.
> Now I want to be able to know how many documents were added to a given
> group[s] between two given dates.
> The challenge is that to be able to filter by dates, I need to have the
> date as the key first part.
> But I also need the group as the first key part in order to aggregate the
> number of group associations.
>
> So I see two options here:
>
> 1.
> Map: associated, {"group": group}
> Reduce: a function that aggregates all values by group, which I assume is
> fine as I know the number of groups is relatively small.
> (plus configuring reduce_limit=false ...)
>
> 2.
> Map: [group,associated], 1
> Reduce: sum(values)
> Here I cannot retrieve multiple groups at once, so I use a request per
> desired group.
>
> Tried the two approaches, with the first one gives faster response. Which
> leads me to two questions:
> 1. Is there any risk in a reduce function that produces a potentially long
> string?
> 2. Is there a better way to achieve what I do here?
>
> Thanks!
>
> Boaz
>