You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Udbhav Agarwal <ud...@syncoms.com> on 2015/03/12 15:02:31 UTC
spark sql performance
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Okay Akhil! Thanks for the information.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:34 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org
Subject: Re: spark sql performance
Can't say that unless you try it.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Sounds great!
So can I expect response time in milliseconds from the join query over this much data ( 0.5 million in each table) ?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:27 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
So you can cache upto 8GB of data in memory (hope your data size of one table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm assuming you have around 12-16 cores total.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Lets say am using 4 machines with 3gb ram. My data is customers records with 5 columns each in two tables with 0.5 million records. I want to perform join query on these two tables.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:16 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
The size/type of your data, and your cluster configuration would be fine i think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Okay Akhil.
I am having 4 cores cpu.(2.4 ghz)
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 1:07 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org
Subject: Re: spark sql performance
You can see where it is spending time, whether there is any GC Time etc from the webUI (running on 4040),Also how many cores are you having?
Thanks
Best Regards
On Fri, Mar 13, 2015 at 1:05 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Additionally I wanted to tell that presently I was running the query on one machine with 3gm ram and the join query was taking around 6 seconds.
Thanks,
Udbhav Agarwal
From: Udbhav Agarwal
Sent: 13 March, 2015 12:45 PM
To: 'Akhil Das'
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: RE: spark sql performance
Okay Akhil! Thanks for the information.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:34 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
Can't say that unless you try it.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Sounds great!
So can I expect response time in milliseconds from the join query over this much data ( 0.5 million in each table) ?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:27 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
So you can cache upto 8GB of data in memory (hope your data size of one table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm assuming you have around 12-16 cores total.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Lets say am using 4 machines with 3gb ram. My data is customers records with 5 columns each in two tables with 0.5 million records. I want to perform join query on these two tables.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:16 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
The size/type of your data, and your cluster configuration would be fine i think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
Re: spark sql performance
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
You can see where it is spending time, whether there is any GC Time etc
from the webUI (running on 4040),Also how many cores are you having?
Thanks
Best Regards
On Fri, Mar 13, 2015 at 1:05 PM, Udbhav Agarwal <ud...@syncoms.com>
wrote:
> Additionally I wanted to tell that presently I was running the query on
> one machine with 3gm ram and the join query was taking around 6 seconds.
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Udbhav Agarwal
> *Sent:* 13 March, 2015 12:45 PM
> *To:* 'Akhil Das'
> *Cc:* user@spark.apache.org
> *Subject:* RE: spark sql performance
>
>
>
> Okay Akhil! Thanks for the information.
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com
> <ak...@sigmoidanalytics.com>]
> *Sent:* 13 March, 2015 12:34 PM
>
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> Can't say that unless you try it.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:32 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Sounds great!
>
> So can I expect response time in milliseconds from the join query over
> this much data ( 0.5 million in each table) ?
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:27 PM
>
>
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> So you can cache upto 8GB of data in memory (hope your data size of one
> table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm
> assuming you have around 12-16 cores total.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Lets say am using 4 machines with 3gb ram. My data is customers records
> with 5 columns each in two tables with 0.5 million records. I want to
> perform join query on these two tables.
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:16 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> The size/type of your data, and your cluster configuration would be fine i
> think.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Thanks Akhil,
>
> What more info should I give so we can estimate query time in my scenario?
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:01 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> That totally depends on your data size and your cluster setup.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Hi,
>
> What is query time for join query on hbase with spark sql. Say tables in
> hbase have 0.5 million records each. I am expecting a query time (latency)
> in milliseconds with spark sql. Can this be possible ?
>
>
>
>
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
>
>
>
>
>
>
>
>
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Additionally I wanted to tell that presently I was running the query on one machine with 3gm ram and the join query was taking around 6 seconds.
Thanks,
Udbhav Agarwal
From: Udbhav Agarwal
Sent: 13 March, 2015 12:45 PM
To: 'Akhil Das'
Cc: user@spark.apache.org
Subject: RE: spark sql performance
Okay Akhil! Thanks for the information.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:34 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
Can't say that unless you try it.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Sounds great!
So can I expect response time in milliseconds from the join query over this much data ( 0.5 million in each table) ?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:27 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
So you can cache upto 8GB of data in memory (hope your data size of one table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm assuming you have around 12-16 cores total.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Lets say am using 4 machines with 3gb ram. My data is customers records with 5 columns each in two tables with 0.5 million records. I want to perform join query on these two tables.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:16 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
The size/type of your data, and your cluster configuration would be fine i think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
Re: spark sql performance
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Can't say that unless you try it.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:32 PM, Udbhav Agarwal <udbhav.agarwal@syncoms.com
> wrote:
> Sounds great!
>
> So can I expect response time in milliseconds from the join query over
> this much data ( 0.5 million in each table) ?
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:27 PM
>
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> So you can cache upto 8GB of data in memory (hope your data size of one
> table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm
> assuming you have around 12-16 cores total.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Lets say am using 4 machines with 3gb ram. My data is customers records
> with 5 columns each in two tables with 0.5 million records. I want to
> perform join query on these two tables.
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:16 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> The size/type of your data, and your cluster configuration would be fine i
> think.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Thanks Akhil,
>
> What more info should I give so we can estimate query time in my scenario?
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:01 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> That totally depends on your data size and your cluster setup.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Hi,
>
> What is query time for join query on hbase with spark sql. Say tables in
> hbase have 0.5 million records each. I am expecting a query time (latency)
> in milliseconds with spark sql. Can this be possible ?
>
>
>
>
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
>
>
>
>
>
>
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Sounds great!
So can I expect response time in milliseconds from the join query over this much data ( 0.5 million in each table) ?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:27 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org
Subject: Re: spark sql performance
So you can cache upto 8GB of data in memory (hope your data size of one table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm assuming you have around 12-16 cores total.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Lets say am using 4 machines with 3gb ram. My data is customers records with 5 columns each in two tables with 0.5 million records. I want to perform join query on these two tables.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:16 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
The size/type of your data, and your cluster configuration would be fine i think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
Re: spark sql performance
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
So you can cache upto 8GB of data in memory (hope your data size of one
table is < 2GB), then it should be pretty fast with SparkSQL. Also i'm
assuming you have around 12-16 cores total.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:22 PM, Udbhav Agarwal <udbhav.agarwal@syncoms.com
> wrote:
> Lets say am using 4 machines with 3gb ram. My data is customers records
> with 5 columns each in two tables with 0.5 million records. I want to
> perform join query on these two tables.
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:16 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> The size/type of your data, and your cluster configuration would be fine i
> think.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Thanks Akhil,
>
> What more info should I give so we can estimate query time in my scenario?
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:01 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> That totally depends on your data size and your cluster setup.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Hi,
>
> What is query time for join query on hbase with spark sql. Say tables in
> hbase have 0.5 million records each. I am expecting a query time (latency)
> in milliseconds with spark sql. Can this be possible ?
>
>
>
>
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
>
>
>
>
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Lets say am using 4 machines with 3gb ram. My data is customers records with 5 columns each in two tables with 0.5 million records. I want to perform join query on these two tables.
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:16 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org
Subject: Re: spark sql performance
The size/type of your data, and your cluster configuration would be fine i think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com<ma...@sigmoidanalytics.com>]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org<ma...@spark.apache.org>
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
Re: spark sql performance
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
The size/type of your data, and your cluster configuration would be fine i
think.
Thanks
Best Regards
On Fri, Mar 13, 2015 at 12:07 PM, Udbhav Agarwal <udbhav.agarwal@syncoms.com
> wrote:
> Thanks Akhil,
>
> What more info should I give so we can estimate query time in my scenario?
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com]
> *Sent:* 13 March, 2015 12:01 PM
> *To:* Udbhav Agarwal
> *Cc:* user@spark.apache.org
> *Subject:* Re: spark sql performance
>
>
>
> That totally depends on your data size and your cluster setup.
>
>
> Thanks
>
> Best Regards
>
>
>
> On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <
> udbhav.agarwal@syncoms.com> wrote:
>
> Hi,
>
> What is query time for join query on hbase with spark sql. Say tables in
> hbase have 0.5 million records each. I am expecting a query time (latency)
> in milliseconds with spark sql. Can this be possible ?
>
>
>
>
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>
>
>
RE: spark sql performance
Posted by Udbhav Agarwal <ud...@syncoms.com>.
Thanks Akhil,
What more info should I give so we can estimate query time in my scenario?
Thanks,
Udbhav Agarwal
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: 13 March, 2015 12:01 PM
To: Udbhav Agarwal
Cc: user@spark.apache.org
Subject: Re: spark sql performance
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>> wrote:
Hi,
What is query time for join query on hbase with spark sql. Say tables in hbase have 0.5 million records each. I am expecting a query time (latency) in milliseconds with spark sql. Can this be possible ?
Thanks,
Udbhav Agarwal
Re: spark sql performance
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
That totally depends on your data size and your cluster setup.
Thanks
Best Regards
On Thu, Mar 12, 2015 at 7:32 PM, Udbhav Agarwal <ud...@syncoms.com>
wrote:
> Hi,
>
> What is query time for join query on hbase with spark sql. Say tables in
> hbase have 0.5 million records each. I am expecting a query time (latency)
> in milliseconds with spark sql. Can this be possible ?
>
>
>
>
>
>
>
>
>
> *Thanks,*
>
> *Udbhav Agarwal*
>
>
>