[Feature] `add_label` should display text in rendering plane

[alessandrofelder]

See brainglobe/brainglobe-heatmap#77 (comment)

[Adisauz]

is this issue still open?

[adamltyson]

I think so!

[alessandrofelder]

Yes!

[AlgoFoe]

Hi @adamltyson! I am working on reproducing this locally and exploring a fix in add_label / make_actor_label. Will update shortly.
Thanks.

[AlgoFoe]

Hey, quick update.
I was able to reproduce the issue locally and tried a couple of approaches in add_label / make_actor_label:

• keeping Text3D in the viewing plane (incl. follow_camera / rotations), and
• an alternative approach projecting label anchor points to screen space and rendering them as Text2D after render.

Both approaches have downsides (either labels drifting / disappearing after slicing, or clashing with the current label lifecycle in render.py), and I couldn’t arrive at a clean fix yet.

Before going further, do you have a preferred direction for this?
(like fixing this at brainrender level vs pushing it down to vedo, or treating labels as true 2D annotations)

Any pointers would be really helpful, thanks!

[alessandrofelder]

Hey @AlgoFoe - thanks for your update.

Before going further, do you have a preferred direction for this?

Not off the top of my head, unfortunately.

either labels drifting / disappearing after slicing

I think we should avoid this if possible

clashing with the current label lifecycle in render.py

Could you explain this more deeply?

[AlgoFoe]

By

clashing with the current label lifecycle in render.py

I meant that Scene.render() currently expects everything in self.labels to be a real actor with _mesh / _is_added. When I tried a screen-space (Text2D) approach, labels stop being true 3D actors and become per-render annotations, which doesn’t fit that lifecycle and leads to breakage.
So it feels like there are 2 models here:

• 3D labels managed as actors (current approach), and
• slice/view-aligned labels behaving more like 2D annotations.

I agree we should avoid labels drifting after slicing. I will move forward by focusing on keeping labels in the 3D actor system. If that turns out to be a dead end, I can revisit the 2D route more cleanly.

Thanks! I will update once I have something concrete.

[alessandrofelder]

Thanks for the helpful explanation @AlgoFoe - I understand now, and

I will move forward by focusing on keeping labels in the 3D actor system.

I think this makes sense (the per-render annotations route feels hacky at first sight 🤔 )