Download experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md from Modularcomputing/Polymorphic: direct link, hf CLI and curl.
- Browser
- Download file 5.48 kB
-
https://huggingface.co/Modularcomputing/Polymorphic/resolve/main/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
- Command line
-
hf download hf://Modularcomputing/Polymorphic/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
-
curl -L -o FOURIER_SPLATTING_REPORT.md https://huggingface.co/Modularcomputing/Polymorphic/resolve/main/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
Fourier Splatting × AnySplat — Experiment Report
Date: 2026-10-13
Hardware: existing SF Compute H100 80 GB node; no new node created
Status: two-scene prototype, 6,000 scene-optimization iterations per scene
Question
Can the existing AnySplat video-to-3D pipeline be improved by converting its context-only geometry into Fourier-encoded planar primitives and optimizing the scene, instead of retaining a dense Gaussian cloud?
This experiment adapts the upstream Fourier Splatting implementation (https://github.com/mbjurca/FourierSplatting) to AnySplat geometry. It is a downstream, per-scene refinement experiment—not a new feed-forward model.
Evaluation protocol
- Apartment: 24 frames total; 12 context views for geometry initialization/training and 12 held-out target views.
- Kitchen: 32 frames total; 16 context views for geometry initialization/training and 16 held-out target views.
- All images are center-cropped and resized to 448×448.
- AnySplat context-only inference initializes the Fourier scene. All-frame AnySplat inference is used to obtain target camera poses and align them into the context reconstruction coordinate frame, matching the evaluation protocol. Held-out target pixels are not used for geometry initialization or Fourier training.
- Initial seed selection uses AnySplat opacity × maximum scale. Fourier Splatting uses K=6 frequencies with staged activation at iterations 300, 600, 900, 1,200, and 1,500.
- Each scene is optimized for 6,000 iterations. The final LOD sweep changes active Fourier frequency count while keeping primitive count fixed.
- Metrics: PSNR (higher is better), SSIM (higher is better), and VGG LPIPS (lower is better). Baseline and Fourier metrics use the same VGG LPIPS implementation.
Main held-out results
| Scene | Method | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | Primitives |
|---|---|---|---|---|---|
| Apartment | AnySplat | 19.1235 | 0.6612 | 0.3154 | 1,858,896 |
| Apartment | Fourier, K=6 | 20.3469 | 0.7041 | 0.3173 | 99,483 |
| Kitchen | AnySplat | 21.6155 | 0.6660 | 0.2678 | 1,983,229 |
| Kitchen | Fourier, K=6 | 25.3078 | 0.8212 | 0.2586 | 120,000 |
Changes vs AnySplat
- Apartment: +1.2234 dB PSNR, +0.0429 SSIM, LPIPS worsens by 0.0019; 18.68× fewer primitives.
- Kitchen: +3.6923 dB PSNR, +0.1552 SSIM, LPIPS improves by 0.0092; 16.53× fewer primitives.
The result is encouraging but not uniform across all metrics: apartment LPIPS is marginally worse, while kitchen improves on all three metrics. These are two indoor scenes, not a benchmark sufficient to claim state of the art.
Fourier level-of-detail sweep
Primitive count is held constant within each scene. Truncating Fourier frequencies reduces PLY size and trades off perceptual quality.
Apartment (99,483 primitives)
| K | PLY size (MB) | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ |
|---|---|---|---|---|
| 1 | 25.07 | 19.9081 | 0.7048 | 0.3831 |
| 2 | 25.87 | 20.0612 | 0.7056 | 0.3802 |
| 3 | 26.66 | 20.3420 | 0.7078 | 0.3551 |
| 4 | 27.46 | 20.3825 | 0.7071 | 0.3374 |
| 5 | 28.25 | 20.3771 | 0.7056 | 0.3236 |
| 6 | 29.05 | 20.3469 | 0.7041 | 0.3173 |
Kitchen (120,000 primitives)
| K | PLY size (MB) | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ |
|---|---|---|---|---|
| 1 | 30.24 | 21.9698 | 0.7638 | 0.3696 |
| 2 | 31.20 | 23.3089 | 0.7772 | 0.3599 |
| 3 | 32.16 | 24.3752 | 0.8006 | 0.3186 |
| 4 | 33.12 | 24.9584 | 0.8137 | 0.2889 |
| 5 | 34.08 | 25.1819 | 0.8190 | 0.2696 |
| 6 | 35.04 | 25.3078 | 0.8212 | 0.2586 |
Takeaway from LOD: K=1–3 is substantially worse on LPIPS. K=5 is a useful compromise; K=6 gives the best perceptual metric on both scenes. The K=1 to K=6 PLY-size increase is about 15–16%, while the scene representation remains 16–19× smaller in primitive count than the AnySplat output. This report does not claim an equivalent 16–19× reduction in serialized bytes; those AnySplat PLY bytes were not measured in this run.
Interpretation and limitations
- Promising direction: Fourier Splatting looks useful as an offline scene-refinement / compact-representation stage after AnySplat.
- Not feed-forward: every scene requires 6,000 optimization iterations. This has not yet been integrated into a low-latency video-to-3D inference path.
- No SOTA claim: only two indoor scenes have been tested. More scenes, motion, occlusion, camera-path diversity, runtime and byte-level comparisons are required.
- Research-code license: the upstream Fourier Splatting repository states non-commercial research/evaluation terms. Review licensing before using its implementation in a commercial release.
Reproduction commands
From the vendored Fourier Splatting repository, with the project environment and compiled rasterizer installed:
python train.py \
--source_path /path/to/fourier_scene \
--model_path /path/to/output \
--eval --iterations 6000 \
--test_iterations 3000 4500 6000 \
--save_iterations 3000 4500 6000 \
--checkpoint_iterations 3000 4500 6000 \
--densify_until_iter 0 --densification_interval 100000 \
--fourier_K 6 \
--fourier_freq_schedule 300,600,900,1200,1500 \
--no_dome --resolution 448
The bundle contains the scene transforms and images, seed point clouds, final checkpoints, Fourier PLYs, LOD variants, evaluation outputs, and the preparation/evaluation scripts. Training uses context-only geometry; target images are reserved for held-out evaluation.