|
Download experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md from Modularcomputing/Polymorphic: direct link, hf CLI and curl.
- Browser
- Download file 5.48 kB
-
https://huggingface.co/Modularcomputing/Polymorphic/resolve/main/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
- Command line
-
hf download hf://Modularcomputing/Polymorphic/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
-
curl -L -o FOURIER_SPLATTING_REPORT.md https://huggingface.co/Modularcomputing/Polymorphic/resolve/main/experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md
5.48 kB
| # Fourier Splatting × AnySplat — Experiment Report | |
| **Date:** 2026-10-13 | |
| **Hardware:** existing SF Compute H100 80 GB node; no new node created | |
| **Status:** two-scene prototype, 6,000 scene-optimization iterations per scene | |
| ## Question | |
| Can the existing AnySplat video-to-3D pipeline be improved by converting its context-only geometry into Fourier-encoded planar primitives and optimizing the scene, instead of retaining a dense Gaussian cloud? | |
| This experiment adapts the upstream Fourier Splatting implementation (https://github.com/mbjurca/FourierSplatting) to AnySplat geometry. It is a downstream, per-scene refinement experiment—not a new feed-forward model. | |
| ## Evaluation protocol | |
| - Apartment: 24 frames total; 12 context views for geometry initialization/training and 12 held-out target views. | |
| - Kitchen: 32 frames total; 16 context views for geometry initialization/training and 16 held-out target views. | |
| - All images are center-cropped and resized to 448×448. | |
| - AnySplat context-only inference initializes the Fourier scene. All-frame AnySplat inference is used to obtain target camera poses and align them into the context reconstruction coordinate frame, matching the evaluation protocol. Held-out target pixels are not used for geometry initialization or Fourier training. | |
| - Initial seed selection uses AnySplat opacity × maximum scale. Fourier Splatting uses K=6 frequencies with staged activation at iterations 300, 600, 900, 1,200, and 1,500. | |
| - Each scene is optimized for 6,000 iterations. The final LOD sweep changes active Fourier frequency count while keeping primitive count fixed. | |
| - Metrics: PSNR (higher is better), SSIM (higher is better), and VGG LPIPS (lower is better). Baseline and Fourier metrics use the same VGG LPIPS implementation. | |
| ## Main held-out results | |
| | Scene | Method | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | Primitives | | |
| |---|---|---:|---:|---:|---:| | |
| | Apartment | AnySplat | 19.1235 | 0.6612 | 0.3154 | 1,858,896 | | |
| | Apartment | Fourier, K=6 | 20.3469 | 0.7041 | 0.3173 | 99,483 | | |
| | Kitchen | AnySplat | 21.6155 | 0.6660 | 0.2678 | 1,983,229 | | |
| | Kitchen | Fourier, K=6 | 25.3078 | 0.8212 | 0.2586 | 120,000 | | |
| ### Changes vs AnySplat | |
| - **Apartment:** +1.2234 dB PSNR, +0.0429 SSIM, LPIPS worsens by 0.0019; 18.68× fewer primitives. | |
| - **Kitchen:** +3.6923 dB PSNR, +0.1552 SSIM, LPIPS improves by 0.0092; 16.53× fewer primitives. | |
| The result is encouraging but not uniform across all metrics: apartment LPIPS is marginally worse, while kitchen improves on all three metrics. These are two indoor scenes, not a benchmark sufficient to claim state of the art. | |
| ## Fourier level-of-detail sweep | |
| Primitive count is held constant within each scene. Truncating Fourier frequencies reduces PLY size and trades off perceptual quality. | |
| ### Apartment (99,483 primitives) | |
| | K | PLY size (MB) | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | | |
| |---:|---:|---:|---:|---:| | |
| | 1 | 25.07 | 19.9081 | 0.7048 | 0.3831 | | |
| | 2 | 25.87 | 20.0612 | 0.7056 | 0.3802 | | |
| | 3 | 26.66 | 20.3420 | 0.7078 | 0.3551 | | |
| | 4 | 27.46 | 20.3825 | 0.7071 | 0.3374 | | |
| | 5 | 28.25 | 20.3771 | 0.7056 | 0.3236 | | |
| | 6 | 29.05 | 20.3469 | 0.7041 | 0.3173 | | |
| ### Kitchen (120,000 primitives) | |
| | K | PLY size (MB) | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | | |
| |---:|---:|---:|---:|---:| | |
| | 1 | 30.24 | 21.9698 | 0.7638 | 0.3696 | | |
| | 2 | 31.20 | 23.3089 | 0.7772 | 0.3599 | | |
| | 3 | 32.16 | 24.3752 | 0.8006 | 0.3186 | | |
| | 4 | 33.12 | 24.9584 | 0.8137 | 0.2889 | | |
| | 5 | 34.08 | 25.1819 | 0.8190 | 0.2696 | | |
| | 6 | 35.04 | 25.3078 | 0.8212 | 0.2586 | | |
| **Takeaway from LOD:** K=1–3 is substantially worse on LPIPS. K=5 is a useful compromise; K=6 gives the best perceptual metric on both scenes. The K=1 to K=6 PLY-size increase is about 15–16%, while the scene representation remains 16–19× smaller in primitive count than the AnySplat output. This report does not claim an equivalent 16–19× reduction in serialized bytes; those AnySplat PLY bytes were not measured in this run. | |
| ## Interpretation and limitations | |
| 1. **Promising direction:** Fourier Splatting looks useful as an offline scene-refinement / compact-representation stage after AnySplat. | |
| 2. **Not feed-forward:** every scene requires 6,000 optimization iterations. This has not yet been integrated into a low-latency video-to-3D inference path. | |
| 3. **No SOTA claim:** only two indoor scenes have been tested. More scenes, motion, occlusion, camera-path diversity, runtime and byte-level comparisons are required. | |
| 4. **Research-code license:** the upstream Fourier Splatting repository states non-commercial research/evaluation terms. Review licensing before using its implementation in a commercial release. | |
| ## Reproduction commands | |
| From the vendored Fourier Splatting repository, with the project environment and compiled rasterizer installed: | |
| ~~~bash | |
| python train.py \ | |
| --source_path /path/to/fourier_scene \ | |
| --model_path /path/to/output \ | |
| --eval --iterations 6000 \ | |
| --test_iterations 3000 4500 6000 \ | |
| --save_iterations 3000 4500 6000 \ | |
| --checkpoint_iterations 3000 4500 6000 \ | |
| --densify_until_iter 0 --densification_interval 100000 \ | |
| --fourier_K 6 \ | |
| --fourier_freq_schedule 300,600,900,1200,1500 \ | |
| --no_dome --resolution 448 | |
| ~~~ | |
| The bundle contains the scene transforms and images, seed point clouds, final checkpoints, Fourier PLYs, LOD variants, evaluation outputs, and the preparation/evaluation scripts. Training uses context-only geometry; target images are reserved for held-out evaluation. | |