How to integrate SMPLer-X for operation

[TDIT-haha]

Thank you very much for your work. In your paper, you mentioned that when used together with SMPLer-X, DPoserX further optimized initialization, especially in the hands, which is an area often overlooked by SOTA full body mesh restoration models. I want to find relevant code in the code, but I haven't found where to implement it? Can you provide it?

[moonbow721]

Hi, thank you for your great question and your interest in our work.

Since our method is optimization-based, it can theoretically leverage any task-specific model, like SMPLer-X, to provide an initial estimate for a given task. In the paper, we used SMPLer-X's predictions as the starting point for the whole-body mesh recovery experiments to demonstrate this capability.

We haven't integrated these specific regression-based models (like SMPLer-X) directly into our repository at this stage, mainly to keep the codebase and environment setup from becoming overly complex.

However, you can easily adapt the existing code to use SMPLer-X for initialization. For the whole-body mesh recovery task, you can refer to this script:

DPoser-X/run/tester/wholebody/demo_fit.py

Lines 120 to 134 in 566e3bd

	init_output = smpl(betas=init_betas,
	body_pose=smpl_poses[:, 3:N_POSES * 3],
	global_orient=smpl_poses[:, :3],
	trans=init_cam_t)
	# be careful: the estimated focal_length should be used here instead of the default constant
	smplify = SMPLify(body_model=smpl, step_size=1e-2, batch_size=batch_size, num_iters=100,
	focal_length=focal_length, args=args)
	results = smplify(init_output.full_pose.detach(),
	init_output.expression.detach(),
	init_output.betas.detach(),
	init_cam_t.detach(),
	camera_center,
	keypoints)

Currently, init_output is initialized using mean poses and shapes. To use SMPLer-X, you just need to replace this init_output object with the prediction results from the SMPLer-X model. The rest of the smplify optimization process will then refine that initial estimate using our DPoser-X prior.

I hope this helps you get started! Let us know if you have any more questions.