Keep up to date with every new upload!

Join free & follow O'Reilly Design Podcast - O'Re
Share
  • 2 years ago
Patrick Wendell on Spark's roadmap, Spark R API, and deep learning on the horizon

Patrick Wendell on Spark's roadmap, Spark R API, and deep learning on the horizon

The O'Reilly Radar Podcast: A special holiday cross-over of the O'Reilly Data Show Podcast.O'Reilly's Ben Lorica chats with Apache Spark release manager and Databricks co-founder Patrick Wendell about Spark's roadmap and interesting applications he's seeing in the growing Spark ecosystem.Here are some highlights from their chat:

We were really trying to solve research problems, so we were trying to work with the early users of Spark, getting feedback on what issues it had and what types of problems they were trying to solve with Spark, and then use that to influence the roadmap. It was definitely a more informal process, but from the very beginning, we were expressly user driven in the way we thought about building Spark, which is quite different than a lot of other open source projects. … From the beginning, we were focused on empowering other people and building platforms for other developers.

One of the early users was Conviva, a company that does analytics for real-time video dis

Comments