You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Rohit Karlupia <ro...@qubole.com> on 2018/01/23 05:01:39 UTC
Spark Tuning Tool
Hi,
I have been working on making the performance tuning of spark applications
bit easier. We have just released the beta version of the tool on Qubole.
https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
This is not OSS yet but we would like to contribute it to OSS. Fishing for
some interest in the community if people find this work interesting and
would like to try to it out.
thanks,
Rohit Karlupia
Re: Spark Tuning Tool
Posted by Mich Talebzadeh <mi...@gmail.com>.
looking good. do we have a downloadable version of this product? I assume
it will be installed on one of the edge nodes?
regards,
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 23 January 2018 at 23:34, ☼ R Nair (रविशंकर नायर) <
ravishankar.nair@gmail.com> wrote:
> Very good job, intact a missing link has been addressed.
>
> Any plan to porting to GITHUB, I would like to contribute.
>
> Best,
> RS
>
> On Tue, Jan 23, 2018 at 12:01 AM, Rohit Karlupia <ro...@qubole.com>
> wrote:
>
>> Hi,
>>
>> I have been working on making the performance tuning of spark
>> applications bit easier. We have just released the beta version of the
>> tool on Qubole.
>>
>> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>>
>> This is not OSS yet but we would like to contribute it to OSS. Fishing
>> for some interest in the community if people find this work interesting and
>> would like to try to it out.
>>
>> thanks,
>> Rohit Karlupia
>>
>>
>>
>
>
> --
>
>
Re: Spark Tuning Tool
Posted by "☼ R Nair (रविशंकर नायर)" <ra...@gmail.com>.
Very good job, intact a missing link has been addressed.
Any plan to porting to GITHUB, I would like to contribute.
Best,
RS
On Tue, Jan 23, 2018 at 12:01 AM, Rohit Karlupia <ro...@qubole.com> wrote:
> Hi,
>
> I have been working on making the performance tuning of spark applications
> bit easier. We have just released the beta version of the tool on Qubole.
>
> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>
> This is not OSS yet but we would like to contribute it to OSS. Fishing
> for some interest in the community if people find this work interesting and
> would like to try to it out.
>
> thanks,
> Rohit Karlupia
>
>
>
--
Re: Spark Tuning Tool
Posted by Shmuel Blitz <sh...@similarweb.com>.
Hi,
Which versions of Spark does the tool support?
Does the tool have any reference to the number of executor cores?
From your blog post it seems that this is a new feature on your service.
Are you offering the tool for download?
Shmuel
On Wed, Jan 24, 2018 at 7:02 PM, Timothy Chen <tn...@gmail.com> wrote:
> Interested to try as well.
>
> Tim
>
> On Tue, Jan 23, 2018 at 5:54 PM, Raj Adyanthaya <ra...@gmail.com> wrote:
> > Its very interesting and I do agree that it will get a lot of traction
> once
> > made open source.
> >
> > On Mon, Jan 22, 2018 at 9:01 PM, Rohit Karlupia <ro...@qubole.com>
> wrote:
> >>
> >> Hi,
> >>
> >> I have been working on making the performance tuning of spark
> applications
> >> bit easier. We have just released the beta version of the tool on
> Qubole.
> >>
> >> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
> >>
> >> This is not OSS yet but we would like to contribute it to OSS. Fishing
> >> for some interest in the community if people find this work interesting
> and
> >> would like to try to it out.
> >>
> >> thanks,
> >> Rohit Karlupia
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
--
Shmuel Blitz
Big Data Developer
Email: shmuel.blitz@similarweb.com
www.similarweb.com
<https://www.facebook.com/SimilarWeb/>
<https://www.linkedin.com/company/429838/> <https://twitter.com/similarweb>
Re: Spark Tuning Tool
Posted by Timothy Chen <tn...@gmail.com>.
Interested to try as well.
Tim
On Tue, Jan 23, 2018 at 5:54 PM, Raj Adyanthaya <ra...@gmail.com> wrote:
> Its very interesting and I do agree that it will get a lot of traction once
> made open source.
>
> On Mon, Jan 22, 2018 at 9:01 PM, Rohit Karlupia <ro...@qubole.com> wrote:
>>
>> Hi,
>>
>> I have been working on making the performance tuning of spark applications
>> bit easier. We have just released the beta version of the tool on Qubole.
>>
>> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>>
>> This is not OSS yet but we would like to contribute it to OSS. Fishing
>> for some interest in the community if people find this work interesting and
>> would like to try to it out.
>>
>> thanks,
>> Rohit Karlupia
>>
>>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Spark Tuning Tool
Posted by Raj Adyanthaya <ra...@gmail.com>.
Its very interesting and I do agree that it will get a lot of traction once
made open source.
On Mon, Jan 22, 2018 at 9:01 PM, Rohit Karlupia <ro...@qubole.com> wrote:
> Hi,
>
> I have been working on making the performance tuning of spark applications
> bit easier. We have just released the beta version of the tool on Qubole.
>
> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>
> This is not OSS yet but we would like to contribute it to OSS. Fishing
> for some interest in the community if people find this work interesting and
> would like to try to it out.
>
> thanks,
> Rohit Karlupia
>
>
>
Re: Spark Tuning Tool
Posted by manish ranjan <cs...@gmail.com>.
This is awesome work Rohit. Not only as a user, but I will be also super
interested in contributing to solving this pain point of my daily work.
Manish
~Manish
On Mon, Jan 22, 2018 at 9:21 PM, lucas.gary@gmail.com <lu...@gmail.com>
wrote:
> I'd be very interested in anything I can send to my analysts to assist
> them with their troubleshooting / optimization... Of course our engineers
> would appreciate it as well.
>
> However we'd be way more interested if it was OSS.
>
> Thanks!
>
> Gary Lucas
>
> On 22 January 2018 at 21:16, Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> That's very interesting, and might also get some interest on the dev@
>> list if it was open source.
>>
>> On Tue, Jan 23, 2018 at 4:02 PM, Roger Marin <ro...@rogersmarin.com>
>> wrote:
>>
>>> I'd be very interested.
>>>
>>> On 23 Jan. 2018 4:01 pm, "Rohit Karlupia" <ro...@qubole.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have been working on making the performance tuning of spark
>>>> applications bit easier. We have just released the beta version of the
>>>> tool on Qubole.
>>>>
>>>> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>>>>
>>>> This is not OSS yet but we would like to contribute it to OSS. Fishing
>>>> for some interest in the community if people find this work interesting and
>>>> would like to try to it out.
>>>>
>>>> thanks,
>>>> Rohit Karlupia
>>>>
>>>>
>>>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
Re: Spark Tuning Tool
Posted by "lucas.gary@gmail.com" <lu...@gmail.com>.
I'd be very interested in anything I can send to my analysts to assist them
with their troubleshooting / optimization... Of course our engineers would
appreciate it as well.
However we'd be way more interested if it was OSS.
Thanks!
Gary Lucas
On 22 January 2018 at 21:16, Holden Karau <ho...@pigscanfly.ca> wrote:
> That's very interesting, and might also get some interest on the dev@
> list if it was open source.
>
> On Tue, Jan 23, 2018 at 4:02 PM, Roger Marin <ro...@rogersmarin.com>
> wrote:
>
>> I'd be very interested.
>>
>> On 23 Jan. 2018 4:01 pm, "Rohit Karlupia" <ro...@qubole.com> wrote:
>>
>>> Hi,
>>>
>>> I have been working on making the performance tuning of spark
>>> applications bit easier. We have just released the beta version of the
>>> tool on Qubole.
>>>
>>> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>>>
>>> This is not OSS yet but we would like to contribute it to OSS. Fishing
>>> for some interest in the community if people find this work interesting and
>>> would like to try to it out.
>>>
>>> thanks,
>>> Rohit Karlupia
>>>
>>>
>>>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>
Re: Spark Tuning Tool
Posted by Holden Karau <ho...@pigscanfly.ca>.
That's very interesting, and might also get some interest on the dev@ list
if it was open source.
On Tue, Jan 23, 2018 at 4:02 PM, Roger Marin <ro...@rogersmarin.com> wrote:
> I'd be very interested.
>
> On 23 Jan. 2018 4:01 pm, "Rohit Karlupia" <ro...@qubole.com> wrote:
>
>> Hi,
>>
>> I have been working on making the performance tuning of spark
>> applications bit easier. We have just released the beta version of the
>> tool on Qubole.
>>
>> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>>
>> This is not OSS yet but we would like to contribute it to OSS. Fishing
>> for some interest in the community if people find this work interesting and
>> would like to try to it out.
>>
>> thanks,
>> Rohit Karlupia
>>
>>
>>
--
Twitter: https://twitter.com/holdenkarau
Re: Spark Tuning Tool
Posted by Roger Marin <ro...@rogersmarin.com>.
I'd be very interested.
On 23 Jan. 2018 4:01 pm, "Rohit Karlupia" <ro...@qubole.com> wrote:
> Hi,
>
> I have been working on making the performance tuning of spark applications
> bit easier. We have just released the beta version of the tool on Qubole.
>
> https://www.qubole.com/blog/introducing-quboles-spark-tuning-tool/
>
> This is not OSS yet but we would like to contribute it to OSS. Fishing
> for some interest in the community if people find this work interesting and
> would like to try to it out.
>
> thanks,
> Rohit Karlupia
>
>
>