#mesoscon14: Full Schedule

7:30am CDT

Registration / Breakfast

Thursday August 21, 2014 7:30am - 9:00am CDT
Foyer

9:00am CDT

Welcome / Kickoff

Thursday August 21, 2014 9:00am - 9:05am CDT
Sheraton Ballroom IV

Keynote

9:05am CDT

Keynote: Ben Hindman, Co-creator of Mesos and PMC Chair, Twitter

Abstract to be added shortly.

Survey this Session

Thursday August 21, 2014 9:05am - 9:35am CDT
Sheraton Ballroom IV

Keynote

9:40am CDT

Mesos 101: Getting Started With Mesos - Ken Sipe, Mesosphere

If you are completely new to Mesos, this session is for you. This demonstration filled session will teach you everything you need to know about getting started with Mesos, including a high-level overview of how Mesos works, instructions for configuring and standing up a Mesos cluster, and best practices for running and scaling applications. By the end of this session, you will:

Have a basic understanding of Mesos fundamentals
Know how to stand up a Mesos cluster
Be able to launch applications on your Mesos cluster
Scale and operate your Mesos cluster

This session is targeted at all levels of IT, devops, and operations teams who are interested in learning how to stand up and operate a Mesos cluster.

This will help people gain practical, introductory knowledge of deploying and operating an Apache Mesos cluster as well as launching and managing applications on that cluster. It will help ops practitioners and developers of any experience level learn a new skill.

Survey this Session

Speakers

Ken Sipe

Distributed Applications Engineer, Mesosphere

Ken Sipe is a Cloud Solution Architect and Distributed Systems Engineer at Mesosphere, focused on helping companies simplify the development and operation of large scale infrastructure and distributed systems with Apache Mesos and DC/OS. Ken is a JavaOne Rockstar, an author and an... Read More →

Thursday August 21, 2014 9:40am - 11:05am CDT
Sheraton Ballroom IV

Tutorial

9:40am CDT

Mesos: An SDK for Distributed Systems Developers - Connor Doyle and Adam Bordelon, Mesosphere

In this tutorial, the experts from Mesosphere will explain the core concepts behind writing frameworks for Apache Mesos, including a detailed description of the low-level tools at your disposal as a framework author. Attendees are encouraged to follow along on their own computers as we demonstrate how to write elastic services using several popular programming languages (without a single line of networking code). Participants will leave with the skills to build the future of distributed systems!

Agenda:

Introduction to Apache Mesos
When to write a Framework
Environment Setup
Hands-On Implementation
Q & A

Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications. It can run Apache Hadoop, Apache Spark, MPI, Hypertable, Storm, Chronos, Marathon, and other applications on a dynamically shared pool of nodes. The biggest user of Mesos is Twitter, where it runs on tens of thousands of cores. Airbnb runs all of their data infrastructure on it, processing petabytes of data.

TUTORIAL REQUIREMENTS AND INSTRUCTIONS FOR ATTENDEES

No prior knowledge of Apache Mesos or distributed systems is required.
Familiarity with at least one of the target languages (Python, Scala, or Go) will be helpful.
All attendees should bring a laptop with Vagrant and VirtualBox pre-installed.

Survey this Session

Speakers

Adam Bordelon

Connor Doyle

Mesosphere

Connor Doyle is a software engineer at Mesosphere, focused on developing tools around Apache Mesos. Before joining Mesosphere, Connor worked at the design firm Gensler where he worked on process improvement, internal libraries and distributed architecture in Scala. He completed his... Read More →

Thursday August 21, 2014 9:40am - 11:05am CDT
Ontario Room

Tutorial

11:15am CDT

Run Any App on Mesos on Any Infrastructure Using Docker - Victor Vieux, Docker

Mesos is an cluster Operating System, you can run your application on an entire datacenter if you need, but you still have the dependencies management to handle; and it can sometimes be a pain. By combining Docker and Mesos, you can deploy any application through your cluster using Docker containers. Victor Vieux will briefly introduce of Docker, then dive into Mesos and Docker integration and at the end Victor Vieux will do a quick demo.

You will get something from that talk if you already know how to setup / run a Mesos cluster. You’ll learn why using Docker inside Mesos is useful and how to setup the Docker integration to automate the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.

This presentation will help sysadmins manage their Mesos cluster more efficiently by speeding up the deployment process through their entire cluster.

Survey this Session

Speakers

Victor Vieux

Docker Core Engineer, Docker, Inc

Victor Vieux works as a software engineer at Docker, Inc. He was one of the few guys working on the Docker internals, he's now working on Docker Swarm, a Docker-native clustering system. Originally from Paris, France, where he was famous for his work on robots (programming them in... Read More →

Thursday August 21, 2014 11:15am - 11:35am CDT
Sheraton Ballroom IV

Operations

11:40am CDT

Highly-Distributed DevOps - How an Elastic Cluster can solve your DevOps Challenges - Tobi Knaup, Mesosphere

Traditional DevOps tools are often not up for the job of running large scale infrastructure in a highly dynamic environment. How do you handle hockey-stick growth? How do you let dozens of engineers deploy dozens of apps, dozens of times per day? What do you do when something goes wrong? In this talk, Tobi Knaup will answer these questions and demonstrate tools that can help ops teams deliver on the promises of Mesos.

This talk is targeted at operations teams and IT organizations that are running (or looking to run) large, elastic clusters using Apache Mesos.

This talk will help more operations people become aware of how highly elastic clusters can solve the challenges of operating at scale, and the new tools that can help them do their jobs.

Speakers

Tobi Knaup

Co-Founder & CTO, Mesosphere

Tobi Knaup is the CTO and Co-Founder of Mesosphere, a startup that brings enterprises the power of cloud native technologies, with the control to run on any infrastructure - datacenter or cloud. He was one of the first engineers and tech lead at Airbnb. At Airbnb, he wrote large parts... Read More →

Thursday August 21, 2014 11:40am - 12:00pm CDT
Sheraton Ballroom IV

Operations

12:00pm CDT

Lunch Break

Thursday August 21, 2014 12:00pm - 1:00pm CDT
Chicago Ballroom

Break

1:00pm CDT

Spark on Mesos: Handling Big Data in a Distributed, Multi-Tenant Environment - Paco Nathan

This presentation will include an introduction to Apache Spark, a general purpose engine for handling a range of batch, interactive, and streaming applications with large-scale data. Spark is built for mixed workloads, so we'll review how it operates and how to think about building Spark apps — particularly how those aspects fit well with Mesos. We will also show how to build and run Apache Spark on Apache Mesos. A demo based on using the free-tier service atop AWS https://elastic.mesosphere.io/ from Mesosphere will cover build, config, packaging, deployment, and then run sample apps using the Scala REPL for Spark. Based on these Spark jobs running, we will explore the Mesos and Spark consoles together, drilling down into details about system performance and troubleshooting.

This is intended primarily for a developer audience, but not with a lot of expertise required. Some familiarity with running Linux shell commands is needed. The expectation is that audience members will learn how to run Spark atop Mesos, along with some basic introduction to building Spark apps in general.

Many people using Spark in the field have heard more about YARN, but do not realize that Spark on Mesos is much simpler to get started running. Also, there are performance benefits, and a closer match for the mixed workloads intention of Mesos. We will show how the two blend together well atop Linux for a full-stack solution.

Survey this Session

Speakers

Paco Nathan

Evil Mad Scientist, Liber 118

Paco Nathan, is a "player/coach" who has led innovative Data teams building large-scale apps for several years. Paco is an O'Reilly author, Apache Spark open source evangelist with Databricks, and an advisor for Amplify Partners... Read More →

Thursday August 21, 2014 1:00pm - 1:20pm CDT
Sheraton Ballroom IV

Developer

1:30pm CDT

Run Applications Like a Boss: Fault-Tolerant and at Scale with Marathon and Mesos - Connor Doyle, Mesosphere

Gone are the days where you could build an app that only needs to run on a few servers. Now every app needs to be like Google: highly-distributed, fast, scalable and fault-tolerant. Devops everywhere have been wasting time building their own custom scale-out architectures for web apps. Marathon is a framework built on Apache Mesos that simplifies and automates operations, and provides a simple self-serve interface for developers to launch their apps on a shared cluster in a scalable and fault-tolerant way.

This talk is for application developers, IT organizations, and Ops teams who are looking to deploy and run Linux apps on large clusters. We will discuss ways to deliver a fault-tolerant and elastic scale-out architecture for any long-lived application like Ruby on Rails/node.js-based web applications or traditional J2EE servers like Tomcat.

This presentation will help organizations and developers understand how to build and deploy Linux apps at large scale, in highly-distributed systems.

Survey this Session

Speakers

Connor Doyle

Mesosphere

Thursday August 21, 2014 1:30pm - 1:50pm CDT
Sheraton Ballroom IV

Developer

2:00pm CDT

Improving Operational Insights with Reactive Stream Processing using Apache Mesos - Justin Becker & Sharma Podila, Netflix

Netflix customers stream more than one billion hours of content each month, accounting for over a third of downstream Internet traffic. At this scale, a lot can go wrong. Justin Becker and Sharma Podila will present how Netflix is building the next generation insight capabilities to help continuously improve the experience for streaming customers. Specifically, the talk will focus on a reactive stream processing system, built using Apache Mesos. The talk will provide the justification for building a new framework. Provide some insight into the scheduling algorithm; a brief overview of optimizations for “stream locality” and auto-scaling. The presentation will also include an overview of the executor; approaches to horizontal and vertical scaling. Then, finish with future directions and collaboration opportunities within the Mesos community.

The audience is anyone interested in building a custom Framework using Mesos. Specifically, those interested in building and deploying Mesos within the Amazon Web Service (AWS) Cloud Environment. The talk will also include detailed information about approaches to scheduling in a distributed, auto-scaling, environment.

The presentation will provide new framework developers with a real world Mesos case study, operating at web scale. Justin and Sharma also plan to discuss future collaboration opportunities within the Mesos community. Topics include: auto-scaling, fault tolerance, core scheduling behavior.

Survey this Session

Speakers

Justin Becker

Justin Becker and Sharma Podila are Senior Software Engineers working for the Edge Engineering team at Netflix. Justin Becker has experience building distributed systems for machine learning applications. Sharma Podila has a strong background in scheduling algorithms for distributed... Read More →

Sharma Podila

Senior Software Engineer, Netflix

Sharma Podila is a Senior Software Enginner in the Edge Engineering team at Netflix, Inc. His current work includes Fenzo, an open source generic scheduler with plugin based optimizations, developing cloud native Mesos frameworks, and evolving the microservices platform at Netfli... Read More →

Thursday August 21, 2014 2:00pm - 2:20pm CDT
Sheraton Ballroom IV

Developer

2:30pm CDT

Past, Present, and Future of the Aurora Scheduler - Bill Farner, Twitter

Apache Aurora has been managing Twitter's production services on Mesos for three years, and now powers the majority of compute at the company. In that time, our team has adapted Aurora's scheduler in many ways to be robust, usable, and performant. In this talk, Bill will discuss robustness features critical to Aurora's success, and explain the features we're interested in building in the scheduler.

This talk will be useful for anyone involved in writing or maintaining services, as well as Mesos framework developers. Audience members will learn about how the Aurora scheduler works, and gain an understanding of the features that make it resilient enough to power twitter.com.

This talk aims to share design decisions, mistakes made, and features developed over several years of writing and maintaining a business-critical Mesos scheduler. This will help promote our open-source software as well as share our experience with other Mesos framework developers.

Survey this Session

Speakers

Bill Farner

Bill Farner has been a Software Engineer at Twitter for the last five years, serving as the Technical Lead on Aurora since its inception four years ago. He previously worked at Google on Production Engineering and storage infrastructure. Bill holds a MS in Computer Engineering from... Read More →

Thursday August 21, 2014 2:30pm - 2:50pm CDT
Sheraton Ballroom IV

Developer

2:50pm CDT

Break

Thursday August 21, 2014 2:50pm - 3:05pm CDT
Foyer

Break

3:05pm CDT

Keynote: John Wilkes, Google

Abstract to be added shortly.

Survey this Session

Thursday August 21, 2014 3:05pm - 3:45pm CDT
Sheraton Ballroom IV

Keynote

3:50pm CDT

Best of both worlds: Running YARN alongside Mesos - Mohit Soni, eBay & Renan DelValle, eBay

Almost anyone considering using Mesos in their datacenter, will come across a very interesting challenge, which is of running Mesos alongside YARN. Mesos and YARN are both resource managers, each with a unique architecture, protocol and ecosystem around them. Today, running these two alongside statically partitions the datacenter, something that both managers are designed to avoid, when they are the only resource manager in control.During this talk, we will present a novel solution that will allow Mesos and YARN to co-exist and share resources, without partitioning the datacenter.

Survey this Session

Speakers

Renan DelValle

Mohit Soni

Software Engineer, PayPal

Mohit works as an engineer in platform team at eBay Inc. His job is to focus on maximizing efficiency, increasing agility, and reducing cost. Mohit presented in past at DockerCon 2014, Hadoop India Summit 2011 and BarCamp 2010. You can follow Mohit as mohitsoni on GitHub and Twit... Read More →

Thursday August 21, 2014 3:50pm - 4:10pm CDT
Sheraton Ballroom IV

Lightning Talk

4:10pm CDT

Break

Thursday August 21, 2014 4:10pm - 4:25pm CDT
Foyer

Break

4:25pm CDT

How Airbnb uses Mesos to Simplify - Brenden Matthews, Airbnb

Airbnb, one of the largest Mesos users, has embraced this technology for the core of its data infrastructure. Mesos helps a small team of engineers stay focused on providing data tools & services to the company, without having to deal with most of the operational headaches. Mesos enables Airbnb to run and scale multiple distributed applications across the same infrastructure without compromising availability.

Anyone interested in gaining insight to how Airbnb manages its Mesos installations on AWS should attend. Topics will include dealing with resource contention, isolation, service discovery, and application deployment.

The presentation will be particularly useful for anyone who is interested in learning about some of the Mesos 'gotchas', as well as considerations when building large systems with Mesos at the core.

Survey this Session

Speakers

Brenden Matthews

Software Architect, Mesosphere

Brenden Matthews is a Software Architect at Mesosphere who works closely with customers and engineering. He ensures technical goals are aligned with customer needs. Previously, he was a software engineer at Airbnb, running Airbnb’s analytics stack. He is an Apache Mesos contributor... Read More →

Thursday August 21, 2014 4:25pm - 4:40pm CDT
Sheraton Ballroom IV

Lightning Talk

4:40pm CDT

Large Scale Mission-Critical Service and Job Deployment with Singularity - Gregory Chomatas, HubSpot

Singularity is an open source scheduler for Mesos Clusters, combining service, job and one-off task deployment in a single framework. Singularity moves beyond batch job scheduling by solving the difficult problem of scheduling and managing the lifecycle of Highly Available, Multi-Instance, Load Balanced services running in a Mesos cluster. In this practical presentation, Gregory will use Hubspot's new Deploy Architecture to demonstrate how Singularity can serve large-scale production deployments of mission-critical, resilient applications and how it solves some of the common pains like load balancing of service instances, managing deployments, graceful decommissioning of hardware, and dynamic log management. Integration with existing deploy architectures and load balancer managers through the Singularity REST API will be discussed and a short demonstration of the Singularity Web UI will be provided.

Survey this Session

Speakers

Gregory Chomatas

Tech Lead, HubSpot

Actively involved in linux and open source for the past 20 years. Tech Lead at HubSpot PaaS team and core developer of the deploy pipeline tools including the Orion Deployer, Deploy Metadata Service, HubSpot Deploy DSL and Singularity, an open source Mesos scheduler that automates... Read More →

Thursday August 21, 2014 4:40pm - 4:55pm CDT
Sheraton Ballroom IV

Lightning Talk

4:55pm CDT

Running Mesos at Atlassian - Ryan Thomas, Atlassian

Atlassian has been using Mesos to build an internal service deliver platform, along with Aurora and Docker. Our journey initially started with looking at the options we have, settling on Mesos and picking up Marathon and the mesos-docker-executor. Subsequent iterations led us to Aurora, and using Docker directly. This project is still in the development phase and we've managed to hit a lot of snags due to most of our components being on the bleeding edge.

Survey this Session

Speakers

Ryan Thomas

Thursday August 21, 2014 4:55pm - 5:10pm CDT
Sheraton Ballroom IV

Lightning Talk

5:10pm CDT

Panel Discussion: Using Mesos at Scale in Production - Moderated By Florian Leibert, Mesosphere

The list of companies that are building (or rebuilding) their infrastructure on Mesos is impressive and growing. In this panel Florian Leibert, CEO of Mesosphere, will lead a lively discussion with three of these companies—Airbnb, HubSpot, and Atlassian—where you’ll get the inside scoop on why they adopted Mesos, how it has changed their operations, and where they plan to take it in the future.

This panel discussion is targeted at business decision-makers and IT team leaders who are looking to run Mesos in production.

Early adopters and champions of Apache Mesos crave hearing from real users and live use cases and success stories to help them decide if Mesos is right for their environment. This panel discussion help educate the audience on what's involved in adopting Mesos as well as understanding the benefits Mesos to different organizations.

Survey this Session

Moderators

Florian Leibert

Florian Leibert is the CEO of Mesosphere. He was an early engineer at Twitter where he helped build critical infrastructure for doing analytics and search. He was primarily responsible for Twitter’s user search product. After a few years at Twitter, Flo joined Airbnb and built out... Read More →

Thursday August 21, 2014 5:10pm - 5:25pm CDT
Sheraton Ballroom IV

Business Keynote

5:25pm CDT

Closing Remarks

Thursday August 21, 2014 5:25pm - 5:30pm CDT
Sheraton Ballroom IV

Keynote

5:30pm CDT

LinuxCon + CloudOpen Onsite Attendee Reception & Booth Crawl

After the LinuxCon + CloudOpen closing keynotes on Thursday, all attendees are invited to join us for drinks, hors d'oeuvres and fun in the Sheraton Chicago Ballroom at the hotel. This is a great time to check out the sponsor showcase and mingle with your fellow attendees. In addition, enjoy games and entertainment.

Thursday August 21, 2014 5:30pm - 7:30pm CDT
Chicago Ballroom

5:40pm CDT

Hackathon Announcements and Team Pitches

Survey this Session

Thursday August 21, 2014 5:40pm - 6:30pm CDT
Sheraton Ballroom IV

Keynote

9:00am CDT

Mesos Hackathon, sponsored by Atlassian

Friday August 22, 2014 9:00am - 6:30pm CDT
Ontario Room