Abstract Rendering: Certified Rendering Under 3D Semantic Uncertainty

NeurIPS 2025 spotlight

Chenxi Ji^*, Yangge Li^*, Xiangru Zhong^*, Huan Zhang, Sayan Mitra

University of Illinois, Urbana-Champaign

Paper Code (Coming Soon)

Abstract

Rendering generates a two-dimensional image from the description of a three-dimensional scene and the camera parameters. To analyze how visual models, such as classifiers and pose estimators, and controllers behave with respect to changes in the scene or in the camera parameters, we need to propagate those changes through the rendering process. This is the problems of Abstract Rendering introduced and tackled in this paper. Abstract rendering computes provable pixel color bounds on all images that can be produced under variations in the scene and in the camera positions. The resulting sets of images are called abstract images which can then be propagated through neural networks verification tools such as CROWN to certify visual models under realistic semantic perturbations. For example, we can certify that a ResNet classifier will continue to detect and classify an airplane or a car correctly as the camera moves within a certain range of positions.

In this peper, we present an abstract rendering framework for scenes represeted by Gaussian splats (3DGS) and NeRFs. Our approach is based on computing piecewise-linear relational abstractions for each of the operations appearing in the standard rendering algorithms for 3DGS and NeRFs. By composing these relational abstractions, we get the abstract rendering algorithms that can compute pixel color bounds or abstract images. Experiments on classification (ResNet), object detection (YOLO), and pose estimation (GateNet) show that our abstract images provide guaranteed over-approximations with no more than 3% conservativeness error, enabling practical certification for safety-critical vision systems.

Abstract Images

An abstract image is a continuous range of images and it can also be seen as a matrix with a range (or interval) of possible colors for each pixel. Here we visualize abstract images by the lower and upper-bound of the pixel colors, that arise from all possible changes in the scene or camera. The camera-pose slider fixes a specific camera position within the scene, while the perturbation-range slider adjusts the extent of camera movement around that position. The reference image represents the rendering from the exact camera pose without any perturbation. Soundness of abstract rendering ensures that the abstract image, represented by the lower and upper bound images, contrains all the possible renderings from the specified range of camera movements.

Visual Model Verification with Abstract Rendering

Abstract rendering can be used to certify the robustness of visual models under 3D semantic uncertainty. For example, we can certify that a ResNet classifier will continue to detect and classify an object correctly as the camera moves around it. More realistically, we can identify the range of camera view angles for which the classifier correctly classifies the object (shown in green) and the range where the classification may be incorrect (shown in red). Similar analysis is performed for pose estimation models such as GateNet.

Video Overview

More details can be found in the paper. Cite this work as:

    
@inproceedings{AbstractRendering_Neurips2025,
    title={Abstract Rendering: Certified Rendering Under 3D Semantic Uncertainty},
    author={Ji, Chenxi and Li, Yangge and Zhong, Xiangru and Zhang, Huan and Mitra, Sayan},
    booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
    year={2025},
    month={December},
    address={San Diego, CA, USA}
}
    

Acknowledgments

Chenxi Ji, Yangge Li, and Sayan Mitra are supported by a research grant from The Boeing Company and NSF (FMITF-2525287). Huan Zhang and Xiangru Zhong are supported in part by the AI2050 program at Schmidt Sciences (AI2050 Early Career Fellowship) and NSF (IIS SLES-2331967, CCF FMITF-2525287). We thank Douglas Belgorod and Maya Cheshire for researching and developing applications of Abstract Rendering.