End-to-end Weather ETL built with Apache Airflow, MinIO, and PostgreSQL. Extract Open-Meteo hourly data → normalize to Parquet → load to L1/L2 tables with idempotent, backfillable DAGs, simple data checks, and SQL-only transformations.
python airflow sql docker-compose postgresql pandas orchestration minio parquet data-quality xcom apache-airflow etl-pipeline backfill airflow-dags dibimbing open-meteo dibimbing-id data-engineering-bootcamp s3-compatible-storage
-
Updated
Oct 20, 2025 - Python