Today is time to talk about Data Pipelines and Data Engineering. I’m really excited to have on stage Sandy Ryza, Lead Engineer of Dagster.
If you’re a software engineer and you’re afraid of dealing with data pipelines, fear no more! Sandy is on a mission to make data pipelines easier to handle for software engineers. Join us in this episode to learn more about Dagster, and how it can make it easier for you to build and manage your data assets.
Enjoy the show 👨🍳
Show Notes
- 00.00 Intro
- 00.46 Episode Start
- 01.09 Sandy’s Introduction
- 02.29 What is Dagster?
- 05.14 How is Dagster affecting software engineers?
- 06.13 Data engineering as software engineering
- 11.34 Cloud vs Self-hosted
- 13.42 Dagster Plus vs Dagster Plus Pro
- 14.41 The history of Dagster
- 19.43 Who’s maintaining Dagster?
- 20.59 Contributing to Apache Spark
- 24.42 Being an Open Source Data Scientist
- 29.18 Speaking a different language than SWE
- 31.44 Moving from SWE to Data Scientist
- 34.38 Approaching the Data Scientist world
- 35.59 What’s next for Dagster
- 37.53 Further reading
- 39.46 Where people can find you online?
Resources
- dagster-io/dagster on GitHub
- Dagster Official Website
- Mentioned Resources
- @sryza on GitHub
- @s_ryz on Twitter