jjrcn/panoworld

★ 16Audience · researcherComplexity · 5/5LicenseSetup · hard

Mindmap

mindmap
  root((PanoWorld))
    What it does
      Whole-house panoramas
      Room-by-room generation
      Geometry consistency
    Key components
      PanoWorld-LRM inference
      3D scene cache
      Floorplan guidance
    Resources
      HuggingFace models
      VR-Tour demo
      arXiv paper
    Setup
      Python 3.10
      PyTorch and CUDA
      RealSee3D dataset

mindmap root((PanoWorld)) What it does Whole-house panoramas Room-by-room generation Geometry consistency Key components PanoWorld-LRM inference 3D scene cache Floorplan guidance Resources HuggingFace models VR-Tour demo arXiv paper Setup Python 3.10 PyTorch and CUDA RealSee3D dataset

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Run inference with PanoWorld-LRM to reconstruct multi-room 3D geometry from a set of panoramic images.

USE CASE 2

Evaluate panorama synthesis quality using the 50 included RealSee3D scenes and the provided evaluation data.

USE CASE 3

Try the interactive VR-tour demo on HuggingFace Spaces to see whole-house panorama generation without local setup.

Tech stack

PythonPyTorchCUDA3DGSHuggingFace

Getting it running

Difficulty · hard Time to first run · 1day+

Requires Python 3.10, PyTorch 2.3.1, and CUDA 12.1, model weights must be downloaded from HuggingFace.

Apache 2.0: use freely in personal or commercial projects, including modifications, with attribution.

In plain English

PanoWorld is an AI research project that generates consistent 360-degree panorama images for every room in a building, working from a floorplan and a style reference image. It comes from researchers at Ke Holdings and is accompanied by a published paper on arXiv. The system works by treating whole-house panorama generation as a sequential process: it moves through the rooms one at a time in an order that matches how someone would navigate a real virtual property tour. For each room, it generates a new 360-degree view that stays geometrically consistent with the rooms already generated. To keep the visual memory of prior rooms intact, the system maintains a 3D scene cache that tracks geometry and materials across viewpoints, so a wall that appears in one room looks the same if visible from another angle. The repository currently releases the PanoWorld-LRM inference code, which is one component of the larger pipeline. PanoWorld-LRM reconstructs multi-room 3D geometry from panoramic images and is what you would use to run evaluations against the included test data. Model weights are hosted on HuggingFace in two resolutions: 1024x512 and 2048x1024. Inference requires Python 3.10, PyTorch 2.3, and CUDA 12.1. The project also offers two interactive demos on HuggingFace Spaces: one for the reconstruction model and one for a VR-tour experience. These let visitors test the output without setting up the local environment. Remaining parts of the pipeline, including the 2D generator inference code, full pipeline scripts, training code, and additional evaluation data, are described as coming soon. The code is released under Apache 2.0.

Copy-paste prompts

Prompt 1

How do I run PanoWorld-LRM inference on my own panoramic images using the 1024x512 model weights?

Prompt 2

Explain how PanoWorld maintains cross-room geometry consistency using its 3D scene cache.

Prompt 3

What format does PanoWorld expect for custom input data, and how do I organize my scene directory with map.json and extrinsics?

Prompt 4

How do I evaluate PanoWorld-LRM on the RealSee3D dataset and interpret the output metrics?

Open on GitHub → Explain another repo

← jjrcn on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.