ETL pipeline using pyspark (Spark - Python)
-
Updated
Apr 4, 2020 - CSS
#
catalyst-optimizer
Here are 7 public repositories matching this topic...
Notes/blogs/tutorials/talks around basics & better optimzed usage of Apache Spark
-
Updated
Sep 25, 2018
An experimental investigation into Apache Spark's Catalyst optimizer, exploring when it optimizes effectively, what defeats it, and when optimization itself adds overhead. Through six controlled experiments, we analyzed predicate pushdown, join reordering, rule criticality, and algebraic equivalence, using PySpark in Google Colab.
-
Updated
Jul 23, 2026 - Jupyter Notebook
🐍 Simplify Python development with reusable code snippets, tools, and examples to enhance productivity and streamline your projects.
mysql python algorithm spark spider script python-library oop cffi python-programming ctypes economics python-bindings interview-practice mysql-replication-protocol myo-sdk hacktoberfest catalyst-optimizer
-
Updated
Oct 2, 2026 - Python
A comprehensive learning repository for Apache Spark and PySpark, covering core architecture, Catalyst optimizer, DataFrame operations, window functions, and performance optimization. Includes hands-on examples and real analytical projects.
big-data apache-spark etl distributed-computing pyspark data-engineering data-analysis performance-tuning parquet dataframe spark-sql catalyst-optimizer spark-architecture window-functions
-
Updated
Oct 1, 2026
Zero-touch semantic mutation testing engine for Apache Spark. Injects plan-level mutations into Catalyst LogicalPlans to measure test suite quality for PySpark and Spark pipelines.
testing spark mutation-testing pyspark data-engineering pytest-plugin quality-assurance catalyst-optimizer
-
Updated
Sep 25, 2026 - Java
🐍 Explore key Python concepts and enhance your coding skills with organized notes for easy reference and learning.
mysql spark anaconda notes cookie scheduler openstack pandas rpc xpath pattern-recognition matplotlib mlab python-tutorial tungsten python-resources python-patterns catalyst-optimizer
-
Updated
Oct 3, 2026
Add this topic to your repo
To associate your repository with the catalyst-optimizer topic, visit your repo's landing page and select "manage topics."