You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Dor Ben Dov <do...@amdocs.com> on 2017/12/12 11:28:06 UTC
drill vs hive vs native hbase count
Hi Everyone,
I tried the same query of 'select count(*) from table x' from hive, drill gui (no hive) and natively using hbase count.
The hive and hbase native got the same result, but in Drill - I got a different (higher) result.
What might be or causing this gap ?
Regards,
Dor
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
Re: drill vs hive vs native hbase count
Posted by Aman Sinha <am...@apache.org>.
In addition, it would help if you can provide more details such as :
- what version of Drill and HBase are you running ?
- the EXPLAIN plan of the query you ran
- what was the expected and actual result ?
On Wed, Dec 13, 2017 at 9:21 AM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:
> Hi Dor,
>
> I guess you need to provide more information. Environment are using (how
> many drillbits, where data is located (fs, dfs))?
> Data format you are querying? How many columns in file?
>
> Kind regards
> Arina
>
> On Wed, Dec 13, 2017 at 2:06 PM, Dor Ben Dov <do...@amdocs.com>
> wrote:
>
> > Anyone ?
> > This is major integrity issue.
> >
> >
> > Regards,
> > Dor
> >
> > -----Original Message-----
> > From: Dor Ben Dov
> > Sent: יום ג 12 דצמבר 2017 13:28
> > To: dev@drill.apache.org
> > Subject: drill vs hive vs native hbase count
> >
> > Hi Everyone,
> >
> > I tried the same query of 'select count(*) from table x' from hive, drill
> > gui (no hive) and natively using hbase count.
> >
> > The hive and hbase native got the same result, but in Drill - I got a
> > different (higher) result.
> >
> > What might be or causing this gap ?
> >
> > Regards,
> > Dor
> >
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> >
>
Re: drill vs hive vs native hbase count
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Hi Dor,
I guess we'll need more information.
If data is in Hbase and you query data from Drill which plugin are you
using (Hive / Hbase / MaprDB)?
What count result is expected? What result Drill returns?
Also as as Aman noted explain plan of query also would help.
Kind regards
Arina
On Thu, Dec 14, 2017 at 9:22 AM, Dor Ben Dov <do...@amdocs.com> wrote:
> Hi Arina,
>
> I am using 3 driibits on my 3 instances of cloudera cluster.
> The data is located in Hbase and of course at the end as a file. How many
> columns - 10.
> The data is regular data but also jsons.
> Does this helps ?
>
> Regards,
> Dor
>
> -----Original Message-----
> From: Arina Yelchiyeva [mailto:arina.yelchiyeva@gmail.com]
> Sent: יום ד 13 דצמבר 2017 19:21
> To: dev@drill.apache.org
> Subject: Re: drill vs hive vs native hbase count
>
> Hi Dor,
>
> I guess you need to provide more information. Environment are using (how
> many drillbits, where data is located (fs, dfs))?
> Data format you are querying? How many columns in file?
>
> Kind regards
> Arina
>
> On Wed, Dec 13, 2017 at 2:06 PM, Dor Ben Dov <do...@amdocs.com>
> wrote:
>
> > Anyone ?
> > This is major integrity issue.
> >
> >
> > Regards,
> > Dor
> >
> > -----Original Message-----
> > From: Dor Ben Dov
> > Sent: יום ג 12 דצמבר 2017 13:28
> > To: dev@drill.apache.org
> > Subject: drill vs hive vs native hbase count
> >
> > Hi Everyone,
> >
> > I tried the same query of 'select count(*) from table x' from hive,
> > drill gui (no hive) and natively using hbase count.
> >
> > The hive and hbase native got the same result, but in Drill - I got a
> > different (higher) result.
> >
> > What might be or causing this gap ?
> >
> > Regards,
> > Dor
> >
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> >
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>
RE: drill vs hive vs native hbase count
Posted by Dor Ben Dov <do...@amdocs.com>.
Hi Arina,
I am using 3 driibits on my 3 instances of cloudera cluster.
The data is located in Hbase and of course at the end as a file. How many columns - 10.
The data is regular data but also jsons.
Does this helps ?
Regards,
Dor
-----Original Message-----
From: Arina Yelchiyeva [mailto:arina.yelchiyeva@gmail.com]
Sent: יום ד 13 דצמבר 2017 19:21
To: dev@drill.apache.org
Subject: Re: drill vs hive vs native hbase count
Hi Dor,
I guess you need to provide more information. Environment are using (how many drillbits, where data is located (fs, dfs))?
Data format you are querying? How many columns in file?
Kind regards
Arina
On Wed, Dec 13, 2017 at 2:06 PM, Dor Ben Dov <do...@amdocs.com> wrote:
> Anyone ?
> This is major integrity issue.
>
>
> Regards,
> Dor
>
> -----Original Message-----
> From: Dor Ben Dov
> Sent: יום ג 12 דצמבר 2017 13:28
> To: dev@drill.apache.org
> Subject: drill vs hive vs native hbase count
>
> Hi Everyone,
>
> I tried the same query of 'select count(*) from table x' from hive,
> drill gui (no hive) and natively using hbase count.
>
> The hive and hbase native got the same result, but in Drill - I got a
> different (higher) result.
>
> What might be or causing this gap ?
>
> Regards,
> Dor
>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
Re: drill vs hive vs native hbase count
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Hi Dor,
I guess you need to provide more information. Environment are using (how
many drillbits, where data is located (fs, dfs))?
Data format you are querying? How many columns in file?
Kind regards
Arina
On Wed, Dec 13, 2017 at 2:06 PM, Dor Ben Dov <do...@amdocs.com> wrote:
> Anyone ?
> This is major integrity issue.
>
>
> Regards,
> Dor
>
> -----Original Message-----
> From: Dor Ben Dov
> Sent: יום ג 12 דצמבר 2017 13:28
> To: dev@drill.apache.org
> Subject: drill vs hive vs native hbase count
>
> Hi Everyone,
>
> I tried the same query of 'select count(*) from table x' from hive, drill
> gui (no hive) and natively using hbase count.
>
> The hive and hbase native got the same result, but in Drill - I got a
> different (higher) result.
>
> What might be or causing this gap ?
>
> Regards,
> Dor
>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>
RE: drill vs hive vs native hbase count
Posted by Dor Ben Dov <do...@amdocs.com>.
Anyone ?
This is major integrity issue.
Regards,
Dor
-----Original Message-----
From: Dor Ben Dov
Sent: יום ג 12 דצמבר 2017 13:28
To: dev@drill.apache.org
Subject: drill vs hive vs native hbase count
Hi Everyone,
I tried the same query of 'select count(*) from table x' from hive, drill gui (no hive) and natively using hbase count.
The hive and hbase native got the same result, but in Drill - I got a different (higher) result.
What might be or causing this gap ?
Regards,
Dor
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>