Skip to content

Beginner's Guide to Nerfstudio

1. Introduction

Nerfstudio is an open-source tool designed to make NeRF (Neural Radiance Field) technology easy to use. In simple terms, NeRF is a way for AI to create 3D scenes from 2D images (like photos) by learning how light radiates from every point in a scene. Nerfstudio’s goal is to provide a user-friendly experience for working with NeRFs. It offers a simple end-to-end workflow for creating, training, and testing NeRF models, so users can turn ordinary pictures or videos into interactive 3D scenes without needing to build everything from scratch.

  • Modular framework developed by Berkeley’s KAIR Lab (Feb 2023).
  • Supports plug-and-play NeRF pipelines, extensive logging, real-time viewer, export options to video, point clouds, meshes.
  • Built for community contributions—supports continuous expansion.

At its core, Nerfstudio takes a collection of images (or a video) of a scene or object and reconstructs a 3D representation of that scene. It handles the entire process: from reading your input images, figuring out where each photo was taken from, training a neural network to represent the scene, and finally letting you view and use the result. This means you can feed Nerfstudio a bunch of photos of, say, your backyard or a sculpture, and it will produce a 3D scene that you can look at from different angles on your computer. The tool was initially developed by researchers at UC Berkeley and is now maintained by an active community, but you don’t have to be a researcher to use it – it’s meant to be accessible to beginners and hobbyists (not just PhDs), with plenty of tutorials and documentation to help users get started.

Key Built-in Models:

  • Nerfacto: Balanced speed‑quality default.
  • Instant‑NGP: Hash‑grid accelerated model
  • Splatfacto: Nerfstudio’s Gaussian Splatting implementation built on gsplat, fully open-source with Apache 2.0 license.

Splatfacto Highlights:

  • Real-time splatting backend (pip install gsplat)
  • Supports alpha culling, density densification, optional MCMC refinement.
  • Splatfacto‑W: Extends for photo collections with per-image appearance embedding, SH background model—+5 dB PSNR & 150× faster training than NeRF

Installation

bash
CopyEdit
pip install nerfstudio
pip install gsplat

  • gsplat compiles CUDA kernels first-time; consider Torch 2.1 if issues arise
  • Merge Splatfacto-W:
bash
CopyEdit
pip install git+https://github.com/KevinXu02/splatfacto-w
ns-install-cli

  • Multi-GPU: supported (distributed) by gsplat backend

Jawset Postshot

Jawset Postshot is an end-to-end software solution from Jawset Visual Computing that creates photorealistic 3D scenes from standard photos or videos. Using advanced radiance field techniques particularly Gaussian Splatting Postshot lets users reconstruct scenes and render images from new angles through a seamless workflow of tracking, training, editing, and rendering. jawset.com


Key Features:

  • Photorealistic Scene Reconstruction: Transform standard images or videos into detailed 3D scenes using state-of-the-art radiance field techniques.
  • Live Preview During Training: Watch and interact with your 3D scene as it develops, enabling quick adjustments during the training process.
  • Integration with Adobe After Effects: Import radiance fields directly into Adobe After Effects for seamless rendering and compositing.
  • Local Processing for Data Security: All processing happens on your computer, keeping your images secure without cloud uploads.

Getting Started with Postshot:

  1. Capture Media: Record a video or take multiple photos of your subject from different angles.
  2. Import into Postshot: Drag and drop your media into Postshot. The software will help you set up training parameters.
  3. Training and Reconstruction: Let Postshot process your media to create the 3D scene. Monitor and interact with the scene as it develops.
  4. Rendering and Export: After training, create rendered movies, animate camera movements, or export your 3D scene to other applications.

System Requirements:

  • Operating System: Windows 10 or later.
  • Graphics Processing Unit (GPU): Nvidia GPU GeForce RTX 2060, Quadro T400/RTX 4000, or higher.

Install nerfstudio
INSTALL WITH SETUP SCRIPT

🛠 How to use:

install_nerfstudio_with_colmap_windows.ps1

  1. Right-click the file and choose “Run with PowerShell” (or open PowerShell as Administrator).
  2. The script will:
    • Install Chocolatey
    • Install Miniconda
    • Set up a Python environment
    • Install PyTorch and tiny-cuda-nn
    • Clone and install Nerfstudio
    • Download and set up COLMAP
# Create a Windows PowerShell script for installing Nerfstudio with COLMAP
windows_script = """
# Nerfstudio + COLMAP Setup Script for Windows (PowerShell)

# === 1. Install Chocolatey if not installed ===
Write-Host "Checking for Chocolatey..."
if (!(Get-Command choco -ErrorAction SilentlyContinue)) {
    Write-Host "Installing Chocolatey..."
    Set-ExecutionPolicy Bypass -Scope Process -Force
    [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12
    iex ((New-Object System.Net.WebClient).DownloadString('<https://chocolatey.org/install.ps1>'))
} else {
    Write-Host "Chocolatey is already installed."
}

# === 2. Install Miniconda ===
Write-Host "Installing Miniconda..."
choco install miniconda3 --yes --params=\\"/AddToPath:1 /RegisterPython:1\\"

# === 3. Refresh environment variables ===
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine")

# === 4. Create Conda Environment ===
Write-Host "Creating Conda environment..."
conda create -y -n nerfstudio-env python=3.8
conda activate nerfstudio-env

# === 5. Install PyTorch with CUDA ===
Write-Host "Installing PyTorch..."
pip install torch torchvision torchaudio --index-url <https://download.pytorch.org/whl/cu116>

# === 6. Install tiny-cuda-nn ===
Write-Host "Installing tiny-cuda-nn..."
$env:TCNN_CUDA_ARCHITECTURES="86"
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

# === 7. Clone and Install Nerfstudio ===
Write-Host "Cloning Nerfstudio..."
git clone <https://github.com/nerfstudio-project/nerfstudio.git>
cd nerfstudio
pip install -e .

# === 8. Install COLMAP ===
Write-Host "Downloading COLMAP..."
Invoke-WebRequest -Uri "<https://colmap.github.io/install.html>" -OutFile "COLMAP.zip"
Expand-Archive -Path "COLMAP.zip" -DestinationPath "$env:ProgramFiles\\\\COLMAP"
$colmapPath = "$env:ProgramFiles\\\\COLMAP\\\\COLMAP.bat"
if (Test-Path $colmapPath) {
    Write-Host "COLMAP installed successfully at $colmapPath"
    [Environment]::SetEnvironmentVariable("Path", $env:Path + ";$env:ProgramFiles\\\\COLMAP", "Machine")
} else {
    Write-Host "COLMAP installation failed."
}

# === 9. Test Installation ===
Write-Host "Testing installation..."
ns-train --help
"""

# Save the script as a .ps1 file
script_path = "/mnt/data/install_nerfstudio_with_colmap_windows.ps1"
with open(script_path, "w") as file:
    file.write(windows_script)

script_path

1. Set Up Python Environment

It’s recommended to use a virtual environment to manage Python dependencies.

    • Using venv:Contents

      bash
      KopiërenBewerken
      python3 -m venv nerfstudio-env
      source nerfstudio-env/bin/activate  # On Windows: nerfstudio-env\\Scripts\\activate
      
      
    • Using conda:

      bash
      KopiërenBewerken
      conda create --name nerfstudio-env python=3.8
      conda activate nerfstudio-env
      
      

2. Install PyTorch

  • Install PyTorch compatible with your CUDA version.Contents

    • Linux:

      bash
      KopiërenBewerken
      pip install torch torchvision torchaudio --extra-index-url <https://download.pytorch.org/whl/cu116>
      
      
    • Windows:Contents

      bash
      KopiërenBewerken
      pip install torch torchvision torchaudio --extra-index-url <https://download.pytorch.org/whl/cu116>
      
      

    Replace cu116 with your specific CUDA version if different.

3. Install Tiny-CUDA-NN

  • Install the tiny-cuda-nn library, which provides efficient neural network primitives for CUDA.Contents

    • Linux:

      bash
      KopiërenBewerken
      pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
      
      
    • Windows:

      bash
      KopiërenBewerken
      set TCNN_CUDA_ARCHITECTURES=86  # Replace with your GPU's architecture code
      pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
      
      

    Determine your GPU’s architecture code from NVIDIA’s official list.Contents

4. Clone and Install Nerfstudio

  • Clone the Nerfstudio repository and install its dependencies.Contents

    bash
    KopiërenBewerken
    git clone <https://github.com/nerfstudio-project/nerfstudio.git>
    cd nerfstudio
    pip install -e .
    
    

7. Verify Installation

  • Run the following command to check if Nerfstudio is installed correctly:

    bash
    KopiërenBewerken
    ns-train --help
    
    

If the installation is successful, this command will display the help message for the ns-train command.Contents

5. Install Additional Dependencies (Optional)

  • For visualization and monitoring, you may want to install:
    • TensorBoard:

      bash
      KopiërenBewerken
      pip install tensorboard
      
      
    • Weights & Biases:

      bash
      KopiërenBewerken
      pip install wandb
      
      

6. Common Installation Issues and Solutions

  • ImportError: DLL load failed while importing _C:
    • This may occur due to a mismatch between your GPU’s architecture and the compiled binaries. Reinstall tiny-cuda-nn with the correct architecture code.Contents
  • No CUDA toolset found (Windows):
    • Ensure that Visual Studio is installed and that CUDA Visual Studio integration is enabled.Contents

To install Nerfstudio, a modular framework for Neural Radiance Field (NeRF) development, follow the detailed steps below. Ensure that your system meets the necessary prerequisites before proceeding.

1. Prerequisites

  • Operating System: Linux or Windows
  • Python: Version 3.8 or higher
  • CUDA Toolkit: Version 11.6 or higher (for GPU acceleration)
  • NVIDIA GPU: Recommended for optimal performance

2. Install System Dependencies

  • Linux: Ensure that build-essential and python3-dev packages are installed.
  • Windows: Install the latest version of Visual Studio with Desktop Development and C++ support.

Full Setup Script to install Nerfstudio whit COLMAP for window

Install Nerfstudio manually

2. Next Steps

  • After installation, you can proceed to train your first NeRF model using Nerfstudio. Detailed instructions are available in the Training Your First Model section of the documentation.

Step 1: Prepare a Dataset

Nerfstudio supports several data formats. For beginners, use a simple photo set or a provided dataset.

Option A: Use a Sample Dataset

bash
KopiërenBewerken
ns-download-data nerfstudio --capture-name=poster

This will download the “poster” dataset to ./data/nerfstudio/poster.

Option B: Use Your Own Images

If you have a folder of images:

  1. Place all images (JPG/PNG) in a folder, e.g., my_data/images.
  2. Run:
bash
KopiërenBewerken
ns-process-data images --data my_data/images --output-dir my_data/processed

This will use COLMAP to estimate camera poses and prepare the dataset.


✅ Step 2: Train the Model

Now you’re ready to train your first NeRF!

bash
KopiërenBewerken
ns-train nerfacto --data data/nerfstudio/poster

What happens:

  • Starts training the Nerfacto model (fast + high-quality)
  • Launches the Nerfstudio Viewer in your browser at http://localhost:7007
  • You’ll see the point cloud refine into a NeRF model over time

✅ Step 3: Explore in Viewer

Open your browser and go to:

arduino
KopiërenBewerken
<http://localhost:7007>

There you can:

  • Navigate the 3D scene
  • Toggle between RGB, Depth, and other render modes
  • View loss graphs, training stats, and step count

✅ Step 4: Export Your Trained Model

Once trained, export mesh or images:

bash
KopiërenBewerken
ns-export poisson --load-config outputs/nerfacto/poster/2024-*/config.yml

You can also export videos:

bash
KopiërenBewerken
ns-render camera-path --load-config outputs/nerfacto/poster/2024-*/config.yml --camera-path-filename path.json

  • ns-process-data for images and video:
bash
CopyEdit
ns-process-data images --data my_images/ --output-dir data/gallery
ns-process-data video --data my_vid.mp4 --output-dir data/gallery

  • Supports COLMAP SfM initialization, ready for Splatfacto or Nerfacto.
  • ns-download-data phototourism for sample datasets

  • Training Nerfacto:

    bash
    CopyEdit
    ns-train nerfacto --data data/gallery --vis viewer+wandb
    
    

    Training Splatfacto (Gaussian Splatting):

    bash
    CopyEdit
    ns-train splatfacto --data data/gallery --pipeline.model.cull_alpha_thresh=0.005
    
    

    Training Splatfacto‑W (in-the-wild):

    bash
    CopyEdit
    ns-train splatfacto-w --data data/gallery
    
    

    Convert NeRF to Splats (“NeRFGS”):

    bash
    CopyEdit
    ns-train nerfsh --data DATA_PATH
    ns-train nerfgs --data DATA_PATH
    
    

    Viewer: Launch with --vis viewer flag.

  • Export Options: .ply for splats or meshes; point cloud export for analysis.
  • Viewer & Sharing: viewer.nerf.studio web UI with shareable captures
  • Authoring Plugins:
    • Blender, Unity export from generated assets.
    • Community “Splatfacto in the Wild” demo.
    • Stepping-in/out through PDFs for large-scale aerial scenes.

Pros

  • Modular architecture, easy to extend
  • Realtime viewer and export to VFX pipelines
  • Supports Gaussian Splatting via Splatfacto & W, fast training, flexible plugins

Cons

  • Requires CUDA GPU (20+ GB VRAM for splats)
  • Splatfacto meshes export still maturing
  • Command-line pose errors possible on Windows (use WSL)

Q: Do I still need COLMAP?

A: It’s needed for splats initialization but not required for Nerfacto.

Q: Better than standalone NeRF libs?

A: Provides real-time viewer, easier plugins, more modular https://kevinxu02.github.io/splatfactow/?utm_source=chatgpt.com

https://docs.nerf.studio/nerfology/methods/splat.html?utm_source=chatgpt.com

https://arxiv.org/abs/2302.04264?utm_source=chatgpt.com

https://github.com/nerfstudio-project/nerfstudio?utm_source=chatgpt.com.

Q: Splatting here? Yes—just use ns-train splatfacto or splatfacto-w

https://docs.nerf.studio/nerfology/methods/splatw.html?utm_source=chatgpt.com.

  • Nerfstudio docs & quickstart
  • Splatfacto reference code
  • Splatfacto‑W paper+repo
  • gsplat technical paper (CUDA/PyTorch backend)
  • Aerial recon study comparing Nerfacto, In‑NGP, and Splatfacto
  • siggraph talk & modularity paper