You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by UMESH CHAUDHARY <um...@gmail.com> on 2015/07/28 08:48:13 UTC
Heatmap with Spark Streaming
I have just started using Spark Streaming and done few POCs. It is fairly
easy to implement. I was thinking of presenting the data using some smart
graphing & dashboarding tools e.g. Graphite or Grafna, but they don't have
heat-maps. I also looked at Zeppelin <http://zeppelin-project.org/> , but
unable to found any heat-map functionality. Could you please suggest any
data visualization tools using Heat-map and Spark streaming.
Re: Heatmap with Spark Streaming
Posted by Tathagata Das <td...@databricks.com>.
I do suggest that the non-spark related discussions be taken to a different
this forum as it does not directly contribute to the contents of this user
list.
On Thu, Jul 30, 2015 at 8:52 PM, UMESH CHAUDHARY <um...@gmail.com>
wrote:
> Thanks for the valuable suggestion.
> I also started with JAX-RS restful service with Angular.
> Since play can help a lot in my scenario, I would prefer it along with
> Angular. Is this combination fine over d3?
>
RE: Heatmap with Spark Streaming
Posted by UMESH CHAUDHARY <um...@gmail.com>.
Thanks for the valuable suggestion.
I also started with JAX-RS restful service with Angular.
Since play can help a lot in my scenario, I would prefer it along with
Angular. Is this combination fine over d3?
RE: Heatmap with Spark Streaming
Posted by Mohammed Guller <mo...@glassbeam.com>.
Umesh,
You can create a web-service in any of the languages supported by Spark and stream the result from this web-service to your D3-based client using Websocket or Server-Sent Events.
For example, you can create a webservice using Play. This app will integrate with Spark streaming in the back-end. The front-end can be a D3-based or any Javascript app. Play makes it easy to stream data to a web client. So a client does not need to continuously poll data from the back-end server.
Mohammed
From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: Thursday, July 30, 2015 2:07 AM
To: UMESH CHAUDHARY
Cc: user@spark.apache.org
Subject: Re: Heatmap with Spark Streaming
You can integrate it with any language (like php) and use ajax calls to update the charts.
Thanks
Best Regards
On Thu, Jul 30, 2015 at 2:11 PM, UMESH CHAUDHARY <um...@gmail.com>> wrote:
Thanks For the suggestion Akhil!
I looked at https://github.com/mbostock/d3/wiki/Gallery to know more about d3, all examples described here are on static data, how we can update our heat map from updated data, if we store it in Hbase or Mysql. I mean, do we need to query back and fourth for it.
Is there any pluggable and more quick component for heatmap with spark streaming?
On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das <ak...@sigmoidanalytics.com>> wrote:
You can easily push data to an intermediate storage from spark streaming (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3 js.
Thanks
Best Regards
On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY <um...@gmail.com>> wrote:
I have just started using Spark Streaming and done few POCs. It is fairly easy to implement. I was thinking of presenting the data using some smart graphing & dashboarding tools e.g. Graphite or Grafna, but they don't have heat-maps. I also looked at Zeppelin<http://zeppelin-project.org/> , but unable to found any heat-map functionality. Could you please suggest any data visualization tools using Heat-map and Spark streaming.
Re: Heatmap with Spark Streaming
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
You can integrate it with any language (like php) and use ajax calls to
update the charts.
Thanks
Best Regards
On Thu, Jul 30, 2015 at 2:11 PM, UMESH CHAUDHARY <um...@gmail.com>
wrote:
> Thanks For the suggestion Akhil!
> I looked at https://github.com/mbostock/d3/wiki/Gallery to know more
> about d3, all examples described here are on static data, how we can update
> our heat map from updated data, if we store it in Hbase or Mysql. I mean,
> do we need to query back and fourth for it.
> Is there any pluggable and more quick component for heatmap with spark
> streaming?
>
> On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das <ak...@sigmoidanalytics.com>
> wrote:
>
>> You can easily push data to an intermediate storage from spark streaming
>> (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3
>> js.
>>
>> Thanks
>> Best Regards
>>
>> On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY <um...@gmail.com>
>> wrote:
>>
>>> I have just started using Spark Streaming and done few POCs. It is
>>> fairly easy to implement. I was thinking of presenting the data using some
>>> smart graphing & dashboarding tools e.g. Graphite or Grafna, but they don't
>>> have heat-maps. I also looked at Zeppelin <http://zeppelin-project.org/> ,
>>> but unable to found any heat-map functionality. Could you please suggest
>>> any data visualization tools using Heat-map and Spark streaming.
>>>
>>>
>>>
>>
>
Re: Heatmap with Spark Streaming
Posted by UMESH CHAUDHARY <um...@gmail.com>.
Thanks For the suggestion Akhil!
I looked at https://github.com/mbostock/d3/wiki/Gallery to know more about
d3, all examples described here are on static data, how we can update our
heat map from updated data, if we store it in Hbase or Mysql. I mean, do we
need to query back and fourth for it.
Is there any pluggable and more quick component for heatmap with spark
streaming?
On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das <ak...@sigmoidanalytics.com>
wrote:
> You can easily push data to an intermediate storage from spark streaming
> (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3
> js.
>
> Thanks
> Best Regards
>
> On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY <um...@gmail.com>
> wrote:
>
>> I have just started using Spark Streaming and done few POCs. It is fairly
>> easy to implement. I was thinking of presenting the data using some smart
>> graphing & dashboarding tools e.g. Graphite or Grafna, but they don't have
>> heat-maps. I also looked at Zeppelin <http://zeppelin-project.org/> ,
>> but unable to found any heat-map functionality. Could you please suggest
>> any data visualization tools using Heat-map and Spark streaming.
>>
>>
>>
>
Re: Heatmap with Spark Streaming
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
You can easily push data to an intermediate storage from spark streaming
(like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3
js.
Thanks
Best Regards
On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY <um...@gmail.com>
wrote:
> I have just started using Spark Streaming and done few POCs. It is fairly
> easy to implement. I was thinking of presenting the data using some smart
> graphing & dashboarding tools e.g. Graphite or Grafna, but they don't have
> heat-maps. I also looked at Zeppelin <http://zeppelin-project.org/> , but
> unable to found any heat-map functionality. Could you please suggest any
> data visualization tools using Heat-map and Spark streaming.
>
>
>