Spark meetup: Spark 0.8 update and Spark at Bizo on Mon Sep 30 in SF

[Reynold Xin <rx...@cs.berkeley.edu>]

Please visit the following page to RSVP:
http://www.meetup.com/spark-users/events/139804022/


This event will feature two talks: we will give an update on Spark 0.8
release, followed by a use case talk from Bizo.



Spark 0.8 update

Spark 0.8 is the biggest Spark release yet, as well as our first one under
Apache. With over 60 developers and 20 companies contributing, this release
adds a slew of new features. To make debugging and productionizing Spark
jobs easier, we have a new monitoring UI and metrics infrastructure. To
expand Spark's out-of-the-box capabilities, 0.8 adds MLlib, a standard
library of high-quality machine learning algorithms. For Python users,
PySpark has been greatly expanded to bring it near feature-parity with
Scala, and now supports IPython and Windows. And for deployability, Spark
0.8 includes much-improved support for YARN, new EC2 scripts, and simpler
packaging and linking. This talk will give a quick tour of these and other
new features.


Spark at Bizo

Bizo allows marketers target display campaigns to specific business
demographic audiences, eg. people in finance or medicine; basically we help
marketers get in front of the "right people" Part of the tooling we provide
to customers is a reporting platform that has all kinds of fun shiny funnel
charts stuffed with pretty metrics (seriously marketers love funnels even
more than developers love free pizza). Recently we had to build some new
reports that allow users to compare behavior of their website visitors
based on whether or not visitors have been exposed to one of our display
ads. This was a perfect opportunity for us to test out using Spark in
production as it involved processing a fairly large amount of log data from
multiple sources on a nightly basis.

This talk will walk through how we're using Spark in production today on
Amazon's EMR service. I'll cover how we've setup our installation &
deployment, how we structure our Spark jobs for easy unit testing, plus
talk about how we put together a successful Spark hackday to get other
engineering sub-teams at Bizo excited about using Spark. Finally I'll cover
some common-pitfalls & caveats we've encountered - especially with regards
of translating some of our older Hive jobs to Spark & how we go about
debugging failed Spark jobs.


We would like to thank Tagged for providing the space.


--
Reynold Xin, AMPLab, UC Berkeley
http://rxin.org