
Unlocking the Future of Data Pipelines - Apache Airflow 3 – PyCon Taiwan 2025
PyCon Taiwan 2025|Day 1, R1 13:20–13:50
🪄 說明 Description 🪄
Apache Airflow 3.0 is here, bringing an enhanced user experience and exciting features like dag versioning, newly designed data-centric syntax, event-driven scheduling, and a brand-new task execution interface that opens up the possibility of writing an Airflow task in a language other than Python!
In collaboration with the Ruff team, we have developed linting and auto-fixing rules that simplify the upgrade process, enabling seamless code upgrade with just one command.
At the end of this talk, I will share our experience of upgrading from Airflow 1 to Airflow 3 through a real-world open-source use case. I will highlight the challenges we faced and how we utilized the new features to optimize our data workflows. Whether you are a seasoned Airflow user or new to it, this session will provide you with valuable insights to enhance your data orchestration capabilities.
Slides: https://speakerdeck.com/leew/unlocking-the-future-of-data-pipeline-942ed56f-0083-4e25-9e30-04850095e824
https://tw.pycon.org/2025/en-us/conference/talk/366
🚀 講者介紹 About Speaker - 李唯 (Wei Lee) 🚀
Wei Lee is an open-source ethicist. He is currently a volunteer for PyCon Taiwan 🇹🇼 and serves as a maintainer for commitizen-tools. Additionally, he is an Apache Airflow committer and acts as both an Apache Airflow mentor and Meme Bot for OpenSource4You. In his free time, Wei enjoys exploring the world through travel, and attending PyCon events allows him to do just that. He has already participated in several PyCon events, including PyCon TW 🇹🇼, PyCon US 🇺🇸, PyCon JP 🇯🇵, PyCon CA 🇨🇦, PyCon APAC 🇵🇭, DurianPy 🇵🇭, and EuroPython 💻 🇨🇿, with more PyCons to come.
Personal Website: https://wei-lee.me
Twitter: @clleew
Follow “PyCon Taiwan”
⭐️ Official Website: https://tw.pycon.org
⭐️ Facebook: https://www.facebook.com/pycontw
⭐️ Instagram: https://www.instagram.com/pycontw
⭐️ Twitter: https://twitter.com/PyConTW
⭐️ LinkedIn: https://www.linkedin.com/company/pycontw
⭐️ Blogger: https://conf.python.tw/
觀看 128按讚 2留言 0
觀看影片 ↗