You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by Apache Wiki <wi...@apache.org> on 2015/02/17 18:23:44 UTC
[Samza Wiki] Update of "FAQ" by ChrisRiccomini
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Samza Wiki" for change notification.
The "FAQ" page has been changed by ChrisRiccomini:
https://wiki.apache.org/samza/FAQ?action=diff&rev1=4&rev2=5
Note that the first two configurations also have system-level settings(i.e. systems.<your system>.samza.reset.offset and systems.<your system>.samza.offset.default).
+ == How should Samza be run on AWS?
+
+ From Gian Merlino:
+
+
+ We've been using Samza in production on AWS for a little over a month.
+ We're just using the YARN runner on a mostly stock hadoop 2.4.0 cluster
+ (not EMR). Our experience is that c3s work well for the YARN instances and
+ i2s work well for the Kafka instances. Things have been pretty solid with
+ that setup.
+
+ For scaling up and scaling down YARN, we just terminate instances or add
+ instances, and this works pretty well. It can take a few minutes for the
+ cluster to realize a node has gone and respawn containers elsewhere.
+
+ We have a separate Kafka cluster just for Samza's use, different from our
+ main Kafka cluster. The main reason is that we wanted to isolate off the
+ disk and network load of state compactions and restores (we don't use
+ compacted topics in our main Kafka cluster, but we do use them with Samza,
+ and the extra load on Kafka can be substantial).
+