Skip to content

Add proper pytesting - #51

Draft
juraph-dev wants to merge 5 commits into
mainfrom
jur/unit-testing
Draft

juraph-dev wants to merge 5 commits into
mainfrom
jur/unit-testing

Conversation

@juraph-dev

@juraph-dev juraph-dev commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

First pass for unit testing of our current architecture. These tests are sort of naive, there should be more rigor put into them, however as a first draft they server fairly well. Adding more tests will be fairly trivial.

Of note, two currently expected failures:

  1. XFAIL tests/jax_tests/test_plnet_lbdn.py::test_lbdn_init_output_zero - init_output_zero documents a zero output at init, but the output is non-zero (see PR notes)

  2. XFAIL tests/torch_tests/test_lipnets.py::test_unitary_no_bias_explicit - Unitary.direct_to_explicit assumes a bias exists

I'm going to treat these as bugs that we should resolve and unmark as xfail.

Resolves #36

First pass for unit testing of our current architecture. These tests are
sort of naive, there should be more rigor put into them, however as a
first draft they server fairly well. Adding more tests will be fairly
trivial.

Of note, two currently reported
failures:

XFAIL tests/jax_tests/test_plnet_lbdn.py::test_lbdn_init_output_zero - init_output_zero documents a zero output at init, but the output is non-zero (see PR notes)
XFAIL tests/torch_tests/test_lipnets.py::test_unitary_no_bias_explicit - Unitary.direct_to_explicit assumes a bias exists
Treating these as bugs. Resolve before merging.
Each network type should have it's own dedicated file. This allows for
easily adding additional tests.

Further, expected inputs/outputs have been moved into their own *.json
file, which has removed the requirement of the dedicated parity testing,
as input/outputs are shared between them
No longer have separate jax/torch testing files. I wasn't happy with the
likelihood of someone writing a test for one of the models, and
forgetting to implement it for the other. By having this unified schema,
one writes the test once, and both versions of the network are compared
against it.

A few notes:
1. Not all networks have been implemented in torch. Their files state
this explicitly, and they will need to be added to this testing once implemented.
2. There are often API differences between the two network backends that
definitely do not need to exist. We will address this with the refactor.
This is where the single backends.py is surprisingly useful.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set up proper testing

1 participant