PDE-Driven Spatiotemporal Disentanglement

HAL Id: hal-03181039

https://hal.archives-ouvertes.fr/hal-03181039

Submitted on 25 Mar 2021

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Distributed under a Creative Commons Attribution| 4.0 International License

PDE-Driven Spatiotemporal Disentanglement

Jérémie Donà, Jean-Yves Franceschi, Sylvain Lamprier, Patrick Gallinari

To cite this version:

Jérémie Donà, Jean-Yves Franceschi, Sylvain Lamprier, Patrick Gallinari. PDE-Driven Spatiotempo-ral Disentanglement. The Ninth International Conference on Learning Representations, May 2021, Vienne (virtual), Austria. �hal-03181039�

PDE-Driven Spatiotemporal Disentanglement

Jérémie Donà,

Jean-Yves Franceschi,

Sylvain Lamprier,

Patrick Gallinari

1

_{Sorbonne Université, CNRS, LIP6, F-75005 Paris, France}

_{Criteo AI Lab, Paris, France}

Motivation

• Disentanglement improves interpretability and prediction

• Prior spatiotemporal disentanglement works are often

complex and do seldom analysis of its meaning

• We aim at grounding spatiotemporal disentanglement on

stronger foundations

Separation of Variables as Disentanglement

• Consider the heat equation:

∂u

∂t = c

2 ∂2u

∂x2 , u(0, t) = u(L, t) = 0, u(x, 0) = f (x),

with separable solutions:

u(x, t) = µ sinnπ L x  | {z } φ(x) exp −cnπ L 2 t ! | {z } ψ(t) = ξ φ(x) × ψ(t)

• This method separates factors of variations and can be

generalized and applied to numerous systems

Separation of Variables for Observations

• State u_v : (x, t) 7→ u_v(x, t), observations v = v_t0, . . . , v_t1

corresponding to a spatial measurement of u:

v_t = ζ u_v(., t)

• Following the functional separation of variables, we

seek φ, ψ, U , ξ and ζ such that:

z = ξ φ(x), ψ(t)
, u(x, t) = U (z), v_t = ζ u(·, t)
,

with associated ODEs or PDEs on φ, ψ and U

• We abstract the unknown spatial coordinates:

v_t = (ζ ◦ U ◦ ξ) φ(·), ψ(t)
= D φ, ψ(t)

• In practice, we learn vectorial φ ≡ S and ψ ≡ T_t

• _{S and Tt0} are inferred with encoders E_S and E_T from

conditioning frames:

V_τ(t₀) = v_t0, . . . , v_t0+τ

Prediction Constraints

• T ≡ ψ is driven by an ODE: ∂Tt

∂t = f (Tt)

• Forecasting and alignment losses:

L_pred = X t k_bv_t − v_tk2₂ L_AE = D  S, E_T  V_τ t0
 − v_t0 2 2

Disentanglement Constraints

• From a strict S invariance constraint to a weaker one to

take into account variations of observable content:

∂E_S(V_τ(t)) ∂t = 0 ⇒ LS_reg = ES Vτ(t0)
− ES Vτ(t1 − τ )
2 2 • Disentanglement loss: LT_reg = T_t0 2 2 = ET Vτ(t0)
2 2 t=0 t=6 t=14 t=99 In pu t fr am es Co nt en t in pu t MI M DD PA E Ou rs Sw ap Tr ut h Ph yD Ne t Models

SST Mov. MNIST TaxiBJ Pred. Pred. Swap Pred.

MSE SSIM MSE

PhyDNet 1.27 0.3878 0.2839 41.9 DDPAE — 0.6446 0.6378 — MIM 0.91 0.6529 — 42.9 Ours 0.86 0.7990 0.7713 39.5 Ours (no S) 0.95 0.6707 — 43.7 t= 0 t= 4 t= 8 Truth Ours

Swap MIM PhyDNet

Co nt en t in pu t In pu t fr am es