FILTR: Extracting Topological Features from Pretrained 3D Models

Louis Martinez

Louis Martinez

Maks Ovsjanikov

Maks Ovsjanikov

Topology and 3D Deep Learning

  • 3D encoders are trained on semantic and geometric tasks
  • To make models topology-aware, we explicitly incorporate topological summaries along with the point cloud.
Topology-aware 3D learning illustration
Computing topological descriptors is costly.
Contrast with the data-driven principle of modern deep learning.

Some Usecases

Robust segmentation
Robust segmentation
Massive scalar fields characterization
Massive scalar fields characterization
Physically plausible reconstruction
Physically plausible reconstruction

Contributions

1

Topology Analysis

Evaluating topological understanding of 3D models.

Topology probing overview
2

FILTR: Filtration Transformer

Feed-forward topology prediction from frozen encoder features.

FILTR pipeline
3

DONUT: Dataset Of maNifold strUcTures

Benchmark with topological annotations.

DONUT benchmark samples

Quantifying Topological Understanding

Point cloudPoint cloud
Transformer encoderself-supervised reconstruction
Block 1
Block 2
Block 3
ReconstructionReconstruction
Underlying topologyUnderlying topology

Quantifying Topological Understanding

Probing

Global structure understanding

Probing illustration

Feature Space Similarity

Local/multiscale understanding

Feature space alignment illustration
DONUT benchmark samples
  • 30K meshes
  • Connected components and genus labels

Probing Results

Layer-wise probing results
Pretrained encoders have a limited understanding of the global structure of point clouds.

Quantifying Topological Understanding

?

Feature space similarity

Local/multiscale understanding

Feature space similarity illustration
What topological signature?

Persistent Homology: A Multiscale Topological Descriptor

Point cloud Persistence barcode H0 H1 Persistence diagram Birth Death Loop Connected components more salient structure Filtration Process

TERMINOLOGY ALERT

  • Filtration
  • Persistence diagram
  • The farther from the diagonal the more salient the corresponding feature

Feature Space Similarity

Point cloudPoint cloud
Persistence diagram
Encode
Vectorize
Encoder feature manifold Persistence feature manifold
Centered kernel Alignment
Aligned encoder feature manifold Aligned persistence feature manifold

Alignment Results

CKA vectorization summary
Encoder features correlate with vectorized persistence diagrams.
3D models might implicitly capture local/multiscale topological structure.

FILTR: Filtration Transformer

FILTR pipeline
  • Trained on DONUT
End-to-End Object Detection with Transformers, Carion et al., ECCV 2020

Quantitative Results

FILTR reconstruction W2 summary on DONUT
16.02 Point-MAEC
FILTR reconstruction W2 summary on ModelNet40 and ABC
39.80 PointGPTL
40.19 PointGPTL

Qualitative Results

Qualitative persistence diagram predictions
Predicted diagrams recover the main structures across ModelNet40, and ABC, although trained on DONUT
Predictions are approximate.
Most persistent features are not captured.

3 Key Contributions

1

Topology Analysis

3D models capture local topological information but struggle with the global structure.

Topology probing overview
2

FILTR

Extracts the topological information contained in 3D models.

FILTR pipeline
3

DONUT

Offers topological variety to train and evaluate models on topology.

DONUT benchmark samples

Project Page

DONUTS AT THE POSTER
QR code to the project page
FIRST COME FIRST SERVED

filtr-topology.github.io

Poster #2