DreamCar运行笔记

2025/4/20 5:17:32 来源：https://blog.csdn.net/xhtchina/article/details/141898145 浏览: 次关键词：DreamCar运行笔记

文章标题：DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstruction

DreamCar，能够在给定少量图像甚至单张图像的情况下重建高质量的3D汽车。为了使生成模型更具泛化性，我们收集了一个名为Car360的汽车数据集，包含超过5600辆汽车。利用这个数据集，我们使生成模型对汽车更加鲁棒。我们使用特定于汽车的生成先验，通过分数蒸馏采样来引导重建。为了进一步补充监督信息，我们利用汽车的几何和外观对称性。

1. 环境配置

1.1. 下载代码

git clone https://github.com/xiaobiaodu/DreamCar.git

1.2. 创建环境

conda create -n DreamCar python=3.9

sudo apt-get install libjpeg-dev libpng-dev

1.3. 安装pytorch

1.3.1. cuda 12.1

pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu121

1.3.2. cuda 11.8

pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu118

1.4. 安装官方依赖

pip install -r requirememnts.txtpip install xformers==0.0.22

1.5. 安装其它依赖

安装通用依赖库

pip install rembg envlight mediapipe nerfacc

安装tiny-cuda-nn

git clone https://github.com/NVlabs/tiny-cuda-nn.gitcd bindings/torchpip install .

安装nvdiffrast

git clone https://github.com/NVlabs/nvdiffrastcd nvdiffrastpip install .

安装clip

git clone https://github.com/openai/CLIP.gitcd CLIPpip install .

2. 下载预训练模型

omnidata
    ├── omnidata_dpt_depth_v2.ckpt
    └── omnidata_dpt_normal_v2.ckpt
zero123
    ├── dreamcar123.ckpt

2.1. Dreamcar123

2.2. Omnidata

3. 运行

3.1. 变量定义

export image_path="example_data/94b33ce331b844dcb991a2020742cebf"
export id=$(basename "$image_path")
export CUDA_VISIBLE_DEVICES=0

3.2. 预处理数据

python preprocess_image.py  "$image_path"

3.3. NeRF

python launch.py --train --config configs/dreamcar-coarse-nerf-nuscenes.yaml data.image_path="$image_path" data.random_camera.height=64 data.random_camera.width=64 data.height=64 data.width=64 system.guidance_3d.pretrained_model_name_or_path="load/zero123/dreamcar123.ckpt"

3.4. Neus

export ckpt=outputs/dreamcar-coarse-nerf/"$id"/ckpts/last.ckpt

python launch.py --train --config configs/dreamcar-coarse-neus-nuscenes.yaml system.weights="$ckpt" data.image_path="$image_path" data.random_camera.height=64 data.random_camera.width=64 data.height=64 data.width=64 system.guidance_3d.pretrained_model_name_or_path="load/zero123/dreamcar123.ckpt"

3.5. Geo

export ckpt=outputs/dreamcar-coarse-neus/"$id"/ckpts/last.ckpt

python launch.py --train --config configs/dreamcar-geometry-nuscenes.yaml system.geometry_convert_from="$ckpt" data.image_path="$image_path" system.guidance_3d.pretrained_model_name_or_path="load/zero123/dreamcar123.ckpt"