Skip to content

ENH: Generalize the principal component flow penalty to all bijectiv… - #18

Merged
ntustison merged 1 commit into
mainfrom
pcf_reg2
Nov 12, 2026
Merged

ntustison merged 1 commit into
mainfrom
pcf_reg2

Conversation

@ntustison

Copy link
Copy Markdown
Member

…e flows.

Move the PCF penalty to antsnormflows.utils.pcf_penalty(model, x, num_dims=1, context=None, create_graph=None), which works for any flow with inverse_and_log_det: NormalizingFlow, ConditionalNormalizingFlow (Jacobian at fixed context), MultiscaleFlow and ConvFlow1d. MultiscaleFlow.pcf_regularizer now delegates to it (same interface and values). Layers that use batch statistics in training mode (nn.BatchNorm*, flow BatchNorm with detach_stats=False) are rejected, since one vector-Jacobian product serves the whole batch.

Tests: exact values against the full Jacobian for RealNVP, a conditional MAF and ConvFlow1d; unbiased stochastic estimate; delegation; BatchNorm rejection; finite gradients.

… flows.

Move the PCF penalty to antsnormflows.utils.pcf_penalty(model, x,
num_dims=1, context=None, create_graph=None), which works for any flow
with inverse_and_log_det: NormalizingFlow, ConditionalNormalizingFlow
(Jacobian at fixed context), MultiscaleFlow and ConvFlow1d.
MultiscaleFlow.pcf_regularizer now delegates to it (same interface and
values). Layers that use batch statistics in training mode (nn.BatchNorm*,
flow BatchNorm with detach_stats=False) are rejected, since one
vector-Jacobian product serves the whole batch.

Tests: exact values against the full Jacobian for RealNVP, a conditional
MAF and ConvFlow1d; unbiased stochastic estimate; delegation; BatchNorm
rejection; finite gradients.
@ntustison

Copy link
Copy Markdown
Member Author

Hey @stnava, here's another topic for discussion next week. I think you'd be interested and it would be useful to get your insight.

@ntustison
ntustison merged commit 56204ab into main Nov 12, 2026
4 checks passed
@ntustison
ntustison deleted the pcf_reg2 branch November 12, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant