RasterizeGaussians

Given 2D gaussians that are parametrized by their means \(μ'\) and covariances \(Σ'\) as well as their radii and conic parameters, the gsplat.rasterize_gaussians() function first sorts each gaussian such that all gaussians within the bounds of a tile are grouped and sorted by increasing depth \(z\), and then renders each pixel within a tile with alpha-compositing.

The discrete rendering equation is given by:

\[\sum_{t=n}^{N}c_{n}·α_{n}·T_{n}\]

Where

\[T_{n} = \prod_{t=m}^{M}(1-α_{m})\]

And

\[ \begin{align}\begin{aligned}α_{n} = o_{n} \exp(-σ_{n})\\σ_{n} = \frac{1}{2} ∆^{⊤}_{n} Σ'^{−1} ∆_{n}\end{aligned}\end{align} \]

\(σ ∈ R^{2}\) is the Mahalanobis distance (here referred to as sigma) which measures how many standard deviations away the center of a gaussian and the rendered pixel center is which is denoted by delta \(∆.\)

The python bindings support conventional 3-channel RGB rasterization as well as N-dimensional rasterization with gsplat.rasterize_gaussians().

gsplat.rasterize_gaussians(xys, depths, radii, conics, num_tiles_hit, colors, opacity, img_height, img_width, block_width, background=None, return_alpha=False)

Rasterizes 2D gaussians by sorting and binning gaussian intersections for each tile and returns an N-dimensional output using alpha-compositing.

Note

This function is differentiable w.r.t the xys, conics, colors, and opacity inputs.

Parameters:

  • xys (Tensor) – xy coords of 2D gaussians.

  • depths (Tensor) – depths of 2D gaussians.

  • radii (Tensor) – radii of 2D gaussians

  • conics (Tensor) – conics (inverse of covariance) of 2D gaussians in upper triangular format

  • num_tiles_hit (Tensor) – number of tiles hit per gaussian

  • colors (Tensor) – N-dimensional features associated with the gaussians.

  • opacity (Tensor) – opacity associated with the gaussians.

  • img_height (int) – height of the rendered image.

  • img_width (int) – width of the rendered image.

  • block_width (int) – MUST match whatever block width was used in the project_gaussians call. integer number of pixels between 2 and 16 inclusive

  • background (Tensor) – background color

  • return_alpha (bool) – whether to return alpha channel

Returns:

  • out_img (Tensor): N-dimensional rendered output image.

  • out_alpha (Optional[Tensor]): Alpha channel of the rendered output image.

Return type:

A Tensor