Apache Airflow

Apache Airflow

Verified

Workflow Orchestration

Open source workflow orchestration platform for data pipeline automation.

Open Source employees
Open Source Project
Founded 2014
Visit Website

Work at Apache Airflow?

Claim this profile to update your company information and connect with buyers

0

Product Overview

Apache Airflow

Apache Airflow is an open-source workflow orchestration platform that enables programmatic authoring, scheduling, and monitoring of data pipelines. It uses Python-based DAGs (Directed Acyclic Graphs) to define workflows and provides a rich UI for monitoring and managing pipeline execution.

Unique Value Proposition

Apache Airflow offers unparalleled flexibility through its code-as-configuration approach, allowing developers to define complex workflows in Python. With its extensive ecosystem of over 1,000 providers and operators, it can integrate with virtually any data tool or cloud service while remaining fully open-source and community-driven.

Categories

Workflow Automation
DataOps Platforms
ETL/ELT

Target Market

Industries

Technology
Financial Services
E-commerce
Healthcare
Media & Entertainment
Manufacturing

Company Size

10 - 50000 employees

Reviews (0)

No reviews yet. Be the first to review!

Pricing Information

Pricing Model

open_source

0

Key Features

Python-based DAG Definition
Dynamic Pipeline Generation
Extensible Plugin Architecture
Rich Web UI
Task Dependencies Management
Scheduling and Triggers
Backfilling Capabilities
Task Retries and SLAs
Branching and Conditional Logic
XCom for Task Communication
Connection Management
Variable Storage
Logging and Monitoring
Email Alerts
Parallelization
Resource Pools
Task Groups
Sensors and Hooks

Integrations

AWS (S3, EMR, Lambda, etc.)
Google Cloud Platform
Microsoft Azure
Snowflake
Databricks
Apache Spark
dbt
Kubernetes
Docker
PostgreSQL
MySQL
MongoDB
Redis
Slack
PagerDuty
GitHub
Jenkins
Tableau
Kafka
Elasticsearch
API Available
View Docs

Security Features

Role-Based Access Control (RBAC)
Authentication Backends (LDAP, OAuth)
SSL/TLS Support
Connection Encryption
Secrets Backend Integration
Audit Logging
Kerberos Support
API Authentication

Implementation & Support

Implementation Time

5 weeks (30 days)

Deployment Options

Cloud
On-Premise
Hybrid

Support Hours

Community Support