Skip to content
View SA01's full-sized avatar

Block or report SA01

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
SA01/README.md

Suffyan Asad

Data Architect & Engineering Lead · Halifax, Nova Scotia, Canada

13 years designing and building data platforms, ML and AI systems that scale to billions of rows and turn data into decisions. Founding team member to Architect and Lead.

🌐 Website · 📄 Full CV · 💼 LinkedIn · ✍️ Medium

Certifications: Databricks Certified Data Engineer Associate · Snowflake SnowPro Core


Impact

  • Built products trusted by Hulu/Disney, Roblox, Zoom, Klarna, Compass and Okta for infrastructure cost visibility and optimization, across $1B+ in annualized cloud spend.
  • 500% increase in the scale of data processing pipelines, to 2.5 billion rows per source, 30% faster and at 15% lower cost.
  • $600K saved for Zoom in one incident caught by outlier detection over billions of data points.
  • 79% reduction in latency from rearchitecting the data processing system, with an 80% processing speed-up.

📄 Click to view the full employment history and impact details in the extended CV

Projects

Led the architecture, benchmarks and proof of concept to replace MongoDB with a sharded ClickHouse warehouse for multi-tenant cost analytics, scaling processing capacity by 50% to over 1 billion rows per run. 79% lower median query latency, and six-month data loads cut from five days to one. Run with Claude Code, Skills and hooks for reproducible, evidence-backed decisions.

ClickHouse PySpark Ray Kubernetes DuckDB Claude Code

Architected the AWS cost pipelines that became the core of the platform, then extended them to Azure, Google Cloud, Kubernetes container costs and rule-based business hierarchies, at up to 2.5 billion rows per run. Fixed data skew, applied bucketing and partitioning, and used spot instances to cut runtime 30% and cost 15%.

Apache Spark Scala Amazon EMR Kubernetes Airflow Terraform

Anomaly Detection and Forecasting

Anomaly detection over 6–8 billion rows of cost data using statistical analysis, functional approximation and outlier detection, catching $600K of runaway spend for Zoom. Forecasting: an ensemble of three models producing 24-month forecasts at every level of the business hierarchy, tuned with cross-validation and back-testing on RMSE and MAPE, delivered in under 3 months.

Python Apache Spark facebook-prophet Kubeflow

Simulated AWS's Reserved Instance application algorithm on hourly EC2 inventory and optimized convertible RI exchanges with a mixed-integer linear program, plus near-real-time cost estimates that bridge the 2–3 day billing lag. A key capability in closing Intuit as a customer.

Python PuLP pandas Amazon EC2

📄 Click to view all project and employment details in the extended CV

Articles

✍️ All 25+ articles on Medium

Stack

Data processing Apache Spark, PySpark, Scala, Apache Airflow, Kubeflow
Platforms Databricks, Snowflake, AWS, Azure, Google Cloud
Warehouses and databases ClickHouse, PostgreSQL, Amazon Redshift, Google BigQuery, DuckDB
Data science and AI Forecasting, anomaly detection, optimization (PuLP), ReAct agents, MCP, Claude Code
Infrastructure Kubernetes, Docker, Terraform

Education

  • MS in Business Analytics, The George Washington University · Fulbright Scholar
  • BS in Computer Science, FAST-NU · Magna Cum Laude, Silver Medal

Pinned Loading

  1. spark-read-jdbc-tutorial spark-read-jdbc-tutorial Public

    This repository contains the code and examples for my article on Medium, which explains how to parallelize reading data from JDBC sources in Apache Spark.

    Python 7 5

  2. dbt-tutorial dbt-tutorial Public

    This repository contains the code and project files for my article on Medium, which serves as an introduction to DBT (Data Build Tool) and how to build data transformations with it.

    Dockerfile 7 3

  3. clickhouse-docker-compose-cluster clickhouse-docker-compose-cluster Public

    An example three node one replica Clickhouse cluster

    Dockerfile 3 2

  4. spark-custom-datasource-tutorial spark-custom-datasource-tutorial Public

    Contains the code and examples for my article on Medium, which explains how to create a custom JDBC read-only data source in Apache Spark 3

    Scala 4 1

  5. spark-window-functions spark-window-functions Public

    This repository contains the code and examples for my article on Medium, which provides an introduction to Window Functions in Apache Spark.

    Python 2

  6. docker-spark-cluster docker-spark-cluster Public

    Forked from mvillarrealb/docker-spark-cluster

    A simple spark standalone cluster for your testing environment purposses

    Dockerfile 7 7