Week 1 - Discussion [17.4 - 22.4]

[git-bauerseb]

Discussion for Week 1

Here we can discuss our goals and milestones per week.

[git-bauerseb]

Sparse point cloud when using colmap

Reconstructed mesh when using Poisson surface reconstruction. I used Open3D reconstruction (as colmap dense reconstruction works only on CUDA devices):

import open3d as o3d

MODEL = 'model.ply'

pcd = o3d.io.read_point_cloud(MODEL)

# Estimate normals
if (not pcd.estimate_normals()):
    print('Normals could not be estimated')

# Apply Poisson
with o3d.utility.VerbosityContextManager(
        o3d.utility.VerbosityLevel.Debug) as cm:
    mesh, densities = o3d.geometry.TriangleMesh.create_from_point_cloud_poisson(
        pcd, depth=5)
print(mesh)
o3d.visualization.draw_geometries([mesh])

Result:

[doginono]

I got this dense reconstruction after applying poisson surface reconstruction, using COLMAP. I used the gpu in my local machine and not the one given by the university.

[doginono]

Training the initial model with the command: ns-train neus-facto --data data/sdfstudio-demo-data/dtu-scan65 . from nerstudio. It has some of the functionalities of the sdfstudio already built in https://docs.nerf.studio/en/latest/extensions/sdfstudio.html . I did not download sdfstudio separately as there was some library collisions, I do not know why.

[doginono]

[git-bauerseb]

Yes, I also encountered problems with sdfstudio and used nerfstudio instead. This is my result when training on the specified dataset. Next, I'll try to train it on the head images provided.

[git-bauerseb]

What I get when I train on the toy dataset:

I guess the camera poses are not correct or there are too few training images. I used ns-process-data for estimating the camera poses (which internally uses colmap)