Real-time AWS ecommerce data pipeline using MSK Serverless, MSK Connect, S3, AWS Glue, Delta Lake, SCD Type 2, and Redshift Serverless.
apache-spark pyspark data-engineering data-lake aws-ec2 redshift apache-kafka amazon-web-services amazon-s3 aws-glue amazon-msk scd-type-2 dimensional-modeling data-lakehouse elt-pipeline amazon-glue aws-msk-serverless medallion-architecture incremental-loading amazon-redshift-serverless
-
Updated
Aug 18, 2026 - Python