[MISC] Genesis version of ComFree solver

[Kashu7100]

Hi!

Thank you for open-sourcing a powerful solver! I’ve drafted a Genesis implementation of the ComFree solver in this PR: #2625.

I also conducted a comparative experiment between Newton and ComFree on the BeyondMimic task. You can find the full performance report here: WandB Report.

Blue: newton (dt=0.005, decimation=4)
Green: comfree (dt=0.005, decimation=4)
Red: comfree (dt=0.002, decimation=10)

Key Findings:

• Speed: ComFree demonstrates a clear advantage in computation time (when using same dt and decimation steps).
• Performance: Currently, the ComFree-trained model is underperforming compared to Newton (sim2sim cannot be stably performed). I suspect this is due to unoptimized hyperparameters (default stiffness and damping). Decreasing the dt helps improve the performance but it is still worse than Newton.

I will play with the stiffness and damping parameters but do you have any recommendation on how to optimize the solver while keeping the speed advantage over Newton?

Looking forward to your thoughts!

[wanxinjin]

Hi @Kashu7100,

Thank you very much for drafting the Genesis implementation and trying this out! It's incredibly exciting for us to see.

Regarding the performance, I'd love to share a few quick thoughts that might be helpful:

ComFree generally prefers smaller time-steps for stability in its current version. Your graphs actually illustrate this perfectly—the red curve (dt=0.002) is noticeably more stable, while the green curve (dt=0.005) shows a few spikes.

That said, you might be able to improve the performance by tuning the stiffness and damping. Here is what we've seen so far:

• Stiffness: We usually find that the default stiffness (0.1) needs to be adjusted depending on the dt you choose. As a general rule, if you use a larger dt, scaling down the stiffness helps. In our paper, we simply scaled it by 1/dt internally (Equations 11 and 12). A more mathematical approach would be $1/dt^2$, but we haven't implemented that in our API yet (since 0.1 can still converge nicely for some small dts). As an example: if you increase dt from 0.001 to 0.01, we'd suggest trying to scale the stiffness down to around 1/10th of its original.

• Damping: The damping default is okay for now. The current default value has worked well for most of the cases we have tested. But a larger one can make the system less stable. I haven't had time to test much about it.

Ultimately, all of these parameter behaviors depend on the heuristics in Equation (12), which we largely follow from MuJoCo. It is definitely possible that a better heuristic exists!

We are really excited to see how you and the Genesis team can further optimize it. We would love to discuss if you need our support anytime.

Team number @Zhi-Xian-Xie, who has some experience using it for RL training with MJlab, has some new takes to share in this threat.

[Kashu7100]

@wanxinjin
Recently, I ablated with two more runs with dt=0.005, decimation=4 and stiffness=0.05 (light green) vs stiffness=0.5 (pink).
Based on this result, it seems higher stiffness result match the performance with smaller dt one (dt=0.002, decimation=10; red). This seems opposite from your recommendation and wondering if I implemented something wrong.

[wanxinjin]

Hi @Kashu7100,

Thanks for these results! That is a very interesting observation.

To clarify my previous point: my recommendation to decrease stiffness with a larger $dt$ was from the perspective of numerical stability (i saw some pikes from the previous runs), rather than sim2sim task performance.

From your current results, it also makes sense to me that the task performance improves with larger stiffness (maybe higher contact stiffness makes it more physically accurate w.r.t. target sim). It seems your task can handle higher stiffness than I cautiously expected. I'd be curious to see if this holds generally in Genesis.