SimuFed

What happens to a distributed aggregate when clients arrive late or drop out?

A small Python framework for exploring federated analytics. Client processes compute summary statistics, and a coordinator combines them without collecting their raw rows. Compare round-based and streaming aggregation under simulated delays and client dropouts.

Developed for COMPSCI 532 - Systems for Data Science. This is a local multiprocessing simulation of federated analytics, rather than a model-training framework.

At a glance

Capability	Implementation
Local computation	Count, sum, sum of squares, and histogram per client
Synchronous aggregation	Collect until all responses arrive or the deadline expires, then merge
Asynchronous aggregation	Merge on arrival; stop on completion, timeout, or an idle grace period
Fault simulation	Random client dropouts and configurable delays
Experiment workflow	Seeded partitions, CSV metrics, and comparison plots

CSV partitions -> client processes -> summary queue -> coordinator -> statistics
                         |
                  delay / dropout

The generator sorts a shared normal sample before partitioning it, giving clients different local distributions. Missing a client can therefore change the aggregate substantially.

Quick start

Use Python 3.10+. Run commands from the repository root.

git clone https://github.com/winepink/SimuFed.git
cd SimuFed
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

python scripts/make_partitions.py --outdir datasets --clients 5 --rows 1000 --seed 42
python run_sync_demo.py --clients 5 --timeout 5 --drop-prob 0.2 --max-delay 3
python run_async_demo.py --clients 5 --timeout 5 --drop-prob 0.2 --max-delay 3 --grace 1

On Windows, activate with .venv\Scripts\activate. Generate at least as many partitions as the demo's client count.

Each demo prints a readable summary and a machine-readable line:

STATS,mode=sync,clients_expected=5,received=...,dropped=...,duration=...,global_n=...,global_mean=...,global_var=...

dropped counts missing responses, including clients that did not arrive before collection stopped.

Compare the modes

Setting	Purpose
--clients	Number of local client processes
--drop-prob	Probability of dropping an update
--max-delay	Maximum simulated delay in seconds
--timeout	Collection deadline in seconds
--grace	Async only: allowed idle time after the last update

With all responses received, both modes should recover the dataset's mean and population variance. With missing responses, they summarize only the data received. Async can return earlier but may omit slow clients; it does not guarantee higher accuracy or more responses.

For custom data directories, sync uses --dataset-dir and async uses --data-dir.

Reproduce the experiment sweep

The runner uses 50 clients, a 2-second timeout, up to 4 seconds of delay, and drop probabilities 0.00 through 0.95, in steps of 0.05.

python scripts/make_partitions.py --outdir datasets --clients 50 --rows 1000 --seed 42
python scripts/run_experiments.py
python plot_results.py

The sweep overwrites results.csv. Plotting writes plot_duration_vs_drop.png, plot_success_vs_drop.png, and plot_mean_error_vs_drop.png in the repository root. Saved figures below live in figures/ and are not replaced automatically.

Runtime and received updates

Mean deviation and interpretation

The plot uses abs(global_mean) relative to the generating distribution's mean of zero. This is not error against the exact centralized mean of the finite dataset. A run receiving no data has no meaningful estimate.

Scope and limitations

• Communication uses a local queue; delays are simulated rather than measured over a network.
• Data generation is seeded; fault sampling and process scheduling are not fixed. The sweep runs each configuration once, without confidence intervals.
• Sync and async timeout clocks start at different points relative to process startup. Small runtime differences are not a controlled performance comparison.
• Histogram merging requires shared bin edges. The demos choose local ranges, so use mean/variance for comparisons; interpreting a global histogram requires a shared range.
• Sharing summaries provides no differential privacy or secure aggregation guarantee.

Repository map

SimuFed/client.py              Local statistics
SimuFed/coordinator.py         Synchronous collection
SimuFed/coordinator_async.py   Incremental aggregation
SimuFed/fault_simulator.py     Delay/dropout simulation
SimuFed/utils/aggregator.py    Summary merging
scripts/make_partitions.py    Seeded data generation
scripts/run_experiments.py    Parameter sweep
run_sync_demo.py              Synchronous CLI
run_async_demo.py             Asynchronous CLI
plot_results.py               CSV-to-figure workflow
figures/                      Saved experiment plots