Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
•
A small neural network is augmented by a multiresolution hash table of trainable feature vectors whose values are optimized through stochastic gradient descent
◦
The multiresolution structure allows the network to disambiguate hash collisions, making for a simple architecture that is trivial to parallelize on modern GPUs
•
We leverage this parallelism by implementing the whole system using fully-fused CUDA kernels with a focus on minimizing wasted bandwidth and compute operations.
… Encodings, Hashing, Function Approximation
1 INTRODUCTION
•
Multiresolution hash encoding
◦
Hyperparameters
▪
# 파라미터
▪
최대 해상도
◦
Pros
▪
Adaptivity
we map a cascade of grids to corresponding fixed-size arrays of feature vectors.
At coarse resolutions, there is a 1:1 mapping from grid points to array entries.
At fine resolutions, the array is treated as a hash table and indexed using a spatial hash function, where multiple grid points alias each array entry.
Such hash collisions cause the colliding training gradients to average, meaning that the largest gradients—those most relevant to the loss function—will dominate.
The hash tables thus automatically prioritize the sparse areas with the most important fine scale detail.
Unlike prior work, no structural updates to the data structure are
needed at any point during training.
▪
경량
•
lookups are O (1)
•
hash tables for all resolutions may be queried in parallel
◦
Applications (MLP의 입출력 관점에서 비교)
1.
Gigapixel image
•
2D 좌표 → MLP → RGB (0~255 정수 값 3개)
2.
Neural signed distance functions (SDF)
•
3D 좌표 → MLP → 표면과의 거리 (유리수 값 1개)
3.
Neural radiance caching (NRC)
the MLP learns the 5D light field of a given scene from a Monte Carlo path tracer
4.
Neural radiance and density fields (NeRF)
•
시점(3D 좌표+방향 벡터) → MLP → 3D 밀도 + 5D light field
2 Prior neural network encodings
•
At machine learning: complex arrangements of data → encoding → linearly separable
◦
one-hot encoding
◦
kernel trick
•
frequency encodings
◦
frequency encodings followed by a linear transformation
▪
been used in other computer graphics tasks
•
approximating the visibility function [Annen et al. 2007; Jansen and Bavoil 2010]
◦
Transformers: identify the location it is currently processing
◦
randomly oriented parallel wavefronts [Tancik et al. 2020]
▪
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. NeurIPS (2020).
◦
level-of-detail filtering [Barron et al. 2021a]
▪
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. arxiv (2021).