You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by KhajaAsmath Mohammed <md...@gmail.com> on 2016/06/13 22:52:52 UTC
Suggestions on Lambda Architecture in Spark
Hi,
In my current project, we are planning to implement POC for lambda
architecture using spark streaming. My use case would be
Kafka --> bacth layer --> Saprk SQL --> Cassandra
Kafka --> Speed layer --> Spark Streaming --> Cassandra
Serving later --> Contact both the layers but I am not sure how the data is
queried in this case.
Does anyone have any github links or tutorials on how to implement lambda
architecture especially the serving layer?
Thanks,
Asmath.
Re: Suggestions on Lambda Architecture in Spark
Posted by Jörn Franke <jo...@gmail.com>.
You do not describe use cases, but technologies. First be aware on your needs and then check technologies.
Otherwise nobody can help you properly and you will end up with an inefficient stack for your needs.
> On 14 Jun 2016, at 00:52, KhajaAsmath Mohammed <md...@gmail.com> wrote:
>
> Hi,
>
> In my current project, we are planning to implement POC for lambda architecture using spark streaming. My use case would be
>
> Kafka --> bacth layer --> Saprk SQL --> Cassandra
>
> Kafka --> Speed layer --> Spark Streaming --> Cassandra
>
> Serving later --> Contact both the layers but I am not sure how the data is queried in this case.
>
> Does anyone have any github links or tutorials on how to implement lambda architecture especially the serving layer?
>
> Thanks,
> Asmath.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Suggestions on Lambda Architecture in Spark
Posted by Luciano Resende <lu...@gmail.com>.
This might be useful:
https://spark-summit.org/east-2016/events/lambda-at-weather-scale/
On Monday, June 13, 2016, KhajaAsmath Mohammed <md...@gmail.com>
wrote:
> Hi,
>
> In my current project, we are planning to implement POC for lambda
> architecture using spark streaming. My use case would be
>
> Kafka --> bacth layer --> Saprk SQL --> Cassandra
>
> Kafka --> Speed layer --> Spark Streaming --> Cassandra
>
> Serving later --> Contact both the layers but I am not sure how the data
> is queried in this case.
>
> Does anyone have any github links or tutorials on how to implement lambda
> architecture especially the serving layer?
>
> Thanks,
> Asmath.
>
--
Sent from my Mobile device
Re: Suggestions on Lambda Architecture in Spark
Posted by Sean Owen <so...@cloudera.com>.
Our labs project oryx is intended to be pretty much a POC of the
lambda architecture on Spark (for ML): http://oryx.io/ You might
consider reusing bits of that.
On Mon, Jun 13, 2016 at 11:52 PM, KhajaAsmath Mohammed
<md...@gmail.com> wrote:
> Hi,
>
> In my current project, we are planning to implement POC for lambda
> architecture using spark streaming. My use case would be
>
> Kafka --> bacth layer --> Saprk SQL --> Cassandra
>
> Kafka --> Speed layer --> Spark Streaming --> Cassandra
>
> Serving later --> Contact both the layers but I am not sure how the data is
> queried in this case.
>
> Does anyone have any github links or tutorials on how to implement lambda
> architecture especially the serving layer?
>
> Thanks,
> Asmath.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org