Apache Airflow
Apache Airflow19
Apache Airflow
Open-source platform for programmatically authoring, scheduling, and monitoring data pipelines.
About
What does Apache Airflow do?
Apache Airflow is an open-source workflow management platform for data engineering pipelines. It allows users to programmatically author, schedule, and monitor workflows, with tasks defined as Directed Acyclic Graphs (DAGs) in Python. Airflow is widely adopted by data engineers, data scientists, and MLOps teams for automating complex workflows, orchestrating machine learning tasks, and managing cloud-native operations.
When was Apache Airflow founded?
Apache Airflow was founded in 2014.
What industry does Apache Airflow operate in?
Apache Airflow operates in MLOps, DataOps, Workflow Automation, Scheduling System, Data Engineering, Open Source.
“Apache Airflow has become the backbone of modern data orchestration. It schedules, monitors, and manages complex workflows across data pipelines, machine learning models, and ETL processes. But running Airflow in production is not a "set it…”
View on Reddit“Is Apache Airflow still the go-to tool for managing data pipelines?. Apache Airflow has been around for a while, but it still comes up constantly whenever teams talk about data orchestration, ETL pipelines, workflow automation, and producti…”
View on Reddit“Apache Airflow 3.2.0 is live. I think, it's time to start ETLs in Apache Airflow 3.2.0 . No more money to pay legacy ETL systems.”
View on Reddit“Apache Airflow 3.0 is here – and it’s a big one!. After months of work from the community, Apache Airflow 3.0 has officially landed and it marks a major shift in how we think about orchestration! This release lays the foundation for a more…”
View on Reddit“Apache Airflow best practices - AMA. We used to run airflow on a vertically scaled single instance. It was a nightmare. Given we run 100s of DAGs, we invested considerable amount of time to revamp. Here are a few learnings: Infrastructure A…”
View on Reddit“Apache Airflow sucks change my mind. I'm a Data Scientist and really want to learn Data Engineering. I have tried several tools like : Docker, Google Big Query, Apache Spark, Pentaho, PostgreSQL. I found Apache Airflow somewhat interesting…”
View on RedditSome comments are pulled from public discussions around the web (look for the source icon). Quotes are excerpts; click through to read the full thread.