[FEATURE] Support matpower and CGMES transmission grid examples

[m-mirz]

Describe the feature request

I am working on a benchmark of popular power system analysis tools using matpower and CGMES test configurations as datasets: https://m-mirz.github.io/grid-bench/

I have not been able yet to find a configuration or fix for pgm to converge on all datasets as you can see from the results page. Enabling or disabling Q-limits, reducing step size or delaying the PV->PQ switch did not seem to help.

Maybe this is not something worth looking into from your point of view since the focus of PGM is distribution grids but I wanted to mention it just in case.

Community Meeting

• I wish to discuss this feature in the bi-monthly PGM community meeting. Please join the PGM community meeting.

[mgovers]

Hi @m-mirz,

Thank you for reporting this. Recently, we have done some improvements that could solve some of the issues you have found. Can you please try to re-run the benchmark with the latest PGM version? We will also have a look ourselves, but it may be worthwile seeing what you find on your benchmark.

[m-mirz]

@mgovers thanks for looking into this. I have just bumped the pgm package to the latest pypi release. The results look pretty much the same since the previous version was already quite recent and only differed in the patch version.

[Jerry-Jinfeng-Guo]

Hi @m-mirz , we have performed some initial scan of the 9 failed cases and tried to convert the data to PGM native data. We did reproduce the same divergence. We will try to investigate this further. @scud-soptim could you also take a look at the data? We do suspect something has to do with PV node handling, among other things. Thanks!

[scud-soptim]

Ergänzte Fassung, die beiden Punkte sind als eigener Absatz drin:

---

Hi Markus,

nice work on the oracle approach, that is the part most benchmarks skip.

Would you consider adding Sparlectra.jl (https://github.com/Welthulk/Sparlectra.jl) as a tool under test? It is a Julia package with its own rectangular Newton-Raphson solver, no external solver library. Since 0.17 it reads all three families your benchmark uses: MATPOWER .m, power-grid-model JSON, and CGMES (bus-branch and node-breaker, with SV comparison). Flat start, single slack, Q-limits off and tolerance are plain configuration keys, so the "same problem for every tool" contract is easy to meet. Sienna already shows the juliacall path, a Sparlectra adapter would follow the same pattern (persistent Net, runpf! on it, bus voltages keyed by MATPOWER bus number). I can help with the adapter or the precompile workload if useful.

Two things worth keeping in mind when PGM and MATPOWER are compared on the same case:

• MATPOWER's reference bus is an ideal slack: the bus voltage is fixed at VM/VA. PGM has no slack bus; its source is a voltage source behind a short-circuit impedance (sk, rx_ratio), and u_ref sits on the internal node, not on the bus. With the default sk the difference is small but not zero, and on a heavily loaded case it shows up as a residual at the reference bus. An adapter has to push sk high enough that the oracle cannot tell the two apart, and say so.
• PV to PQ switching is a different mechanism in the two tools. MATPOWER (pf.enforce_q_lims) solves to convergence first, then converts violating PV buses to PQ (all at once or the largest one) and re-solves from the previous solution. PGM's PV support switches inside the iteration. Your protocol turns Q-limits off, so this should not affect the graded results, but it is a likely reason why PGM's PV path converges on case14 only and is worth stating in the adapter docstring.

One observation on the PEGASE cases from my own runs, which may or may not match what your oracle sees: on case1354pegase and case9241pegase the branch SHIFT values behave as if they were in radians with the opposite sign rather than in degrees, and it is not always obvious whether the tap RATIO has to be inverted. A branch-convention scan against the stored VM/VA picks "radians, opposite sign" as the only consistent reading on those files. If your oracle passes five tools on case9241pegase with degrees and the standard sign, then either the scan or the reference data is misleading me, and I would be glad to learn which. Either way, a file-level statement of the convention the oracle assumes would help every adapter author.

Regards,
Udo

[m-mirz]

Hi @scud-soptim and @mgovers ,

I've added Sparlectra :-)

The VM/VA columns are not used except for the matpower to pgm converter. I have updated that part in a branch and it slightly improves the pgm results but not in terms of convergence: m-mirz/grid-bench@9468166

So, you can just ignore VM/VA in the matpower cases.

I have continued to investigate different options with Claude and it looks like I might have found the issue. PGM does not seem to offer a flat start yet. Adding a flat start, improves convergence in most cases: https://github.com/m-mirz/power-grid-model/tree/feature/newton-raphson-flat-start

If you like, I can create a PR from that branch in my fork.

Cheers
Markus

[Jerry-Jinfeng-Guo]

Hi @m-mirz , thank you for the investigation and good to hear most issues are solved with this effort. We look forward to your PR.