[MongoDB] How does drill aggregate data

[Andy <ta...@gmail.com>]

Hi, Drill Team
My name is Andy.
Currently, Im considering to use Apache Drill to query and aggregate data
from MongoDB.

But I really confused how does it work about aggregation.
For example i have this query:
SELECT user,SUM(amount)
FROM sales WHERE type=1
GROUP BY user

Then, i have 2 thinking on this:
1. Drill will use aggregate from MongoDB's api to do GROUP BY
2. Drill will filter data ( find from Mongo API) then it will do
aggregation by its own way ( such as: collect all matching documents onto
memory & do group by )

So, can you help me to understand how it works
Thanks in advanced

Re: [MongoDB] How does drill aggregate data

[AnilKumar B <ak...@gmail.com>]

Hi Andy,

Please find the answer inline.

Then, i have 2 thinking on this:
1. Drill will use aggregate from MongoDB's api to do GROUP BY
>> Currently Drill's mongo plugin will not push down group by operator to
mongo DB.
2. Drill will filter data ( find from Mongo API) then it will do
aggregation by its own way ( such as: collect all matching documents onto
memory & do group by )
>> Yes, it works in this way.

Thanks,
B Anil Kumar.



Thanks & Regards,
B Anil Kumar.

On Mon, Oct 2, 2017 at 8:42 AM, Andy <ta...@gmail.com> wrote:

> Hi, Drill Team
> My name is Andy.
> Currently, Im considering to use Apache Drill to query and aggregate data
> from MongoDB.
>
> But I really confused how does it work about aggregation.
> For example i have this query:
> SELECT user,SUM(amount)
> FROM sales WHERE type=1
> GROUP BY user
>
> Then, i have 2 thinking on this:
> 1. Drill will use aggregate from MongoDB's api to do GROUP BY
> 2. Drill will filter data ( find from Mongo API) then it will do
> aggregation by its own way ( such as: collect all matching documents onto
> memory & do group by )
>
> So, can you help me to understand how it works
> Thanks in advanced
>

Re: [MongoDB] How does drill aggregate data

[rahul challapalli <ch...@gmail.com>]

This will largely depend on the implementation of the Mongo DB storage
plugin. Based on my glimpse at the plugin code [1], it looks like we read
all the data from Mongo DB and then aggregation in drill

[1]
https://github.com/apache/drill/blob/master/contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoGroupScan.java

On Mon, Oct 2, 2017 at 8:42 AM, Andy <ta...@gmail.com> wrote:

> Hi, Drill Team
> My name is Andy.
> Currently, Im considering to use Apache Drill to query and aggregate data
> from MongoDB.
>
> But I really confused how does it work about aggregation.
> For example i have this query:
> SELECT user,SUM(amount)
> FROM sales WHERE type=1
> GROUP BY user
>
> Then, i have 2 thinking on this:
> 1. Drill will use aggregate from MongoDB's api to do GROUP BY
> 2. Drill will filter data ( find from Mongo API) then it will do
> aggregation by its own way ( such as: collect all matching documents onto
> memory & do group by )
>
> So, can you help me to understand how it works
> Thanks in advanced
>