#mesoscon14 has ended
Back To Schedule
Thursday, August 21 • 1:00pm - 1:20pm
Spark on Mesos: Handling Big Data in a Distributed, Multi-Tenant Environment - Paco Nathan

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

This presentation will include an introduction to Apache Spark, a general purpose engine for handling a range of batch, interactive, and streaming applications with large-scale data. Spark is built for mixed workloads, so we'll review how it operates and how to think about building Spark apps — particularly how those aspects fit well with Mesos. We will also show how to build and run Apache Spark on Apache Mesos. A demo based on using the free-tier service atop AWS https://elastic.mesosphere.io/ from Mesosphere will cover build, config, packaging, deployment, and then run sample apps using the Scala REPL for Spark. Based on these Spark jobs running, we will explore the Mesos and Spark consoles together, drilling down into details about system performance and troubleshooting.

This is intended primarily for a developer audience, but not with a lot of expertise required. Some familiarity with running Linux shell commands is needed. The expectation is that audience members will learn how to run Spark atop Mesos, along with some basic introduction to building Spark apps in general.

Many people using Spark in the field have heard more about YARN, but do not realize that Spark on Mesos is much simpler to get started running. Also, there are performance benefits, and a closer match for the mixed workloads intention of Mesos. We will show how the two blend together well atop Linux for a full-stack solution.  

Survey this Session


Paco Nathan

Evil Mad Scientist, Liber 118
Paco Nathan, is a "player/coach" who has led innovative Data teams building large-scale apps for several years. Paco is an O'Reilly author, Apache Spark open source evangelist with Databricks, and an advisor for Amplify Partners... Read More →

Thursday August 21, 2014 1:00pm - 1:20pm CDT
Sheraton Ballroom IV

Attendees (0)