English | 中文

trajectory_tools provides the release pipeline for InternScenes camera trajectories:

1. Generate camera pose trajectories from USD scene assets.
2. Render RGB, depth, instance segmentation, shading, and randomized-light shading in Isaac Sim.

This document only covers data contracts, configuration, and command usage. Environment setup is documented in the repository-level README files.

Commands in this document are intended to run from the repository root. Default paths are resolved relative to the current working directory.

data/
  source/
    GRScenes/
      part1/
        101_usd/
          scene_00001/
            scene.usd          # the only USD file in the scene directory
  env/
    default_no_specular.mdl
    WhiteMode.mdl
    default.png
output/
  camera_poses/
  scene_cache/
  render/

Each scene directory must contain exactly one direct .usd file. Both trajectory generation and rendering load that file. If a scene directory contains zero or multiple USD files, the scene is skipped instead of selecting an arbitrary asset.

data/env contains material resources used by rendering and material-reference repair.

Generate trajectories for every scene under data/source/{dataset}/part{part}/{usd}_usd/:

python trajectory_tools/generate_trajectory.py \
  --config trajectory_tools/configs/trajectory/strategy/astar_nav.yaml \
  --dataset GRScenes \
  --part 1 \
  --usd 101

Process a single scene:

python trajectory_tools/generate_trajectory.py \
  --config trajectory_tools/configs/trajectory/strategy/astar_nav.yaml \
  --dataset GRScenes \
  --part 1 \
  --usd 101 \
  --specific scene_00001

Use the straight-line forward-motion strategy:

python trajectory_tools/generate_trajectory.py \
  --config trajectory_tools/configs/trajectory/strategy/forward_motion.yaml \
  --dataset GRScenes \
  --part 1 \
  --usd 101

Override scalar YAML values from the command line:

python trajectory_tools/generate_trajectory.py \
  --config trajectory_tools/configs/trajectory/strategy/astar_nav.yaml \
  --dataset GRScenes \
  --part 1 \
  --usd 101 \
  --override camera.sample_number=5 strategy.params.minimum_distance=3.0

Common options:

Generation configs are YAML files. The loader first reads configs/trajectory/default.yaml, then deep-merges the selected strategy YAML, then applies --override.

scene_map:
  grid_resolution: 0.05
  safe_distance: 0.20
  ceiling_offset: 1.8
  cluster_eps: 0.2
  cluster_safe_threshold: 0.05
  cluster_min_points: 1000
  min_overlap_ratio: 0.3

strategy:
  name: astar_nav
  params: {}

camera:
  pitch_deg: 0
  pitch_deg_range: [0, 15]
  height: 1.8
  height_range: [1.2, 1.7]
  sample_number: 20

postprocess:
  min_step_distance: 0.3
  min_step_angle: 0.15
  min_turn_distance: 0.15
  bspline_degree: 3
  bspline_step: 100

Supported strategies:

When pitch_deg_range or height_range is used, the value is sampled once per floor before post-processing trajectories on that floor.

Trajectory files:

output/camera_poses/{dataset}/part{part}/{usd}_usd/
  generation_report_{strategy}.json
  {scene_name}/{strategy}/
    f{floor_idx}_t{traj_idx}.json

Each f{floor_idx}_t{traj_idx}.json file is a list of camera frames:

[
  [[x, y, z], [qw, qx, qy, qz]],
  [[x, y, z], [qw, qx, qy, qz]]
]

Positions are in meters. Quaternions use scalar-first order. The camera height is stored as absolute scene height: floor_height + camera_height.

The generation report is keyed by scene name:

{
  "scene_00001": {
    "status": "success",
    "floors_detected": 1,
    "trajectories_generated": 20,
    "trajectories_attempted": 20,
    "failure_reasons": [],
    "prim_filter": {
      "valid_count": 123,
      "noise_count": 4,
      "noise_prims": [],
      "survival_rates": {}
    }
  }
}

Skipped scenes may only contain status and reason.

Parsed-scene caches are stored separately:

output/scene_cache/{dataset}/part{part}/{usd}_usd/{scene_name}/
  scene_pcd.ply
  scene_metadata.json

Rendering must run in an Isaac Sim Python environment.

python trajectory_tools/render_trajectory.py \
  --dataset GRScenes \
  --part 1 \
  --usd 101 \
  --strategy astar_nav

By default, rendering outputs rgbd. Select other output types with flags:

# RGB only
python trajectory_tools/render_trajectory.py --part 1 --usd 101 --strategy astar_nav --rgb

# Depth only
python trajectory_tools/render_trajectory.py --part 1 --usd 101 --strategy astar_nav --depth

# RGBD plus instance segmentation
python trajectory_tools/render_trajectory.py --part 1 --usd 101 --strategy astar_nav --rgb --depth --instance

# White-material shading
python trajectory_tools/render_trajectory.py --part 1 --usd 101 --strategy astar_nav --shading

# White-material shading with randomized lights
python trajectory_tools/render_trajectory.py --part 1 --usd 101 --strategy astar_nav --shading-randlight

Common options:

Rendering loads:

data/source/{dataset}/part{part}/{usd}_usd/{scene_name}/{single_usd_file}.usd
output/camera_poses/{dataset}/part{part}/{usd}_usd/{scene_name}/{strategy}/f0_t0.json

Rendered outputs:

output/render/{dataset}/part{part}/{usd}_usd/{scene_name}/{strategy}/{trajectory_name}/
  camera_poses.json
  rgb/*.jpg
  depth/*.npy
  instance/*_seg.pkl
  shading/*.jpg
  shading_randlight/*.jpg
  rgb.mp4
  shading.mp4
  shading_randlight.mp4
  camera_intrinsic.json
  camera_intrinsic_rgb.json
  camera_intrinsic_depth.json
  camera_intrinsic_instance.json

Output details:

Re-running the same render command skips outputs that already contain the expected frame count and required metadata files.