Abstract
3D Gaussian Splatting (3DGS) has emerged as a main stream for novel view synthesis, leveraging continuous aggregations of Gaussian functions to model scene geometry. However, 3DGS suffers from substantial memory requirements to store the multitude of Gaussians, hindering its practicality. To address this challenge, we introduce GaussianSpa, an optimization-based simplification framework for compact and high-quality 3DGS. Specifically, we formulate the simplification as an optimization problem associated with the 3DGS training. Correspondingly, we propose an efficient “optimizing-sparsifying” solution that alternately solves two independent sub-problems, gradually imposing strong sparsity onto the Gaussians in the training process. Our comprehensive evaluations on various datasets show the superiority of GaussianSpa over existing state-of-the-art approaches. Notably, GaussianSpa achieves an average PSNR improvement of 0.9 dB on the real-world Deep Blending dataset with 10× fewer Gaussians compared to the vanilla 3DGS.
Method Overview
-
Our GaussianSpa formulates 3DGS simplification as a constrained optimization problem with a sparsity
constraint:
\begin{equation}\label{eqn:obj-opacity-form}
\begin{aligned}
\min_{(\boldsymbol{a},\boldsymbol{\Theta})} ~&\mathcal{L}(\boldsymbol{a},\boldsymbol{\Theta}),
\end{aligned}
\end{equation}
\begin{equation}
\begin{aligned}
\text{s.t.}\hspace{0.2cm}&\|\boldsymbol{a}\|_0\leq\kappa.
\end{aligned}
\end{equation}
Variable \(\boldsymbol{\Theta}\) represent other GS variables except for the Gaussian opacity \(\boldsymbol{a}\),
and the sparsity constraint restricts the \(\ell_0\) norm of the Gaussian opacity to a specified number.
We present an efficient "optimizing-sparsifying" solution to solve this problem, consisting of two main
steps
(please refer to our paper for more details):
\begin{equation}\label{eqn:obj-optimizing} \textbf{ “Optimizing” Step:}\hspace{0.3cm}\min_{\boldsymbol{a},\boldsymbol{{\Theta}}}\mathcal{L}(\boldsymbol{a},\boldsymbol{\Theta})+\frac{\delta}{2}\|\boldsymbol{a}-\boldsymbol{z}+\boldsymbol{\lambda}\|^2. \end{equation} The "optimizing" step essentially optimizes over the original Gaussian variables and pushes the Gaussian opacity \(\boldsymbol{a}\) close to the auxiliary \(\boldsymbol{z}\) that is exactly sparse using a gradient-based approach. This way, it can simultaneously satisfy the 3DGS performance and the sparsity requirements, achieving a sweet point in the training process.
\begin{equation}\label{eqn:obj-sparsifying} \textbf{ “Sparsifying” Step:}\hspace{0.3cm}\min_{\boldsymbol{z}}h(\boldsymbol{z})+\frac{\delta}{2}\|\boldsymbol{a}-\boldsymbol{z}+\boldsymbol{\lambda}\|^2.\hspace{0.6cm} \end{equation} The "sparsifying" step essentially prunes the auxiliary variable \(\boldsymbol{z}\), which can be considered the exactly sparse version of \(\boldsymbol{a}\). By not directly operating the original Gaussians, this step avoids the irreversible loss of important information.
