Actor -> ActorTemplate relationship

[zoez7]

Actor -> ActorTemplate relationship

Today we read some data from ActorTemplate and Actor objects while running. Should we:

1. Keep doing that, and prevent deletion of any AT that has an actor (foreign key like atespace)
2. Keep doing that, but not prevent AT deletion (which would break actors), and just tell users "don't do that" (this is where we are today, IIUC)
3. Copy all data from AT -> Actor, so it is self-contained?

CUJ	a) Actor references AT, with deletion restriction	b) Actor references AT, without deletion restriction	c) Copy all data from AT -> Actor, so it is self-contained
CreateActor (same as UpdateActor's tmpl)	Set actor.template=templ. Must ensure tmpl exists in database.	Set actor.template=templ. Must ensure tmpl exists in database.	Copy all data from templ, including golden snapshot. Must ensure tmpl exists in database.
ResumeActor (winner: c > a > b)	Look up the template by foreign key.	Look up the template by foreign key. May fail if the AT is deleted.	No need to look up the actor template.
Data at rest (winner: a = b > c, but for use cases with 1 actor per template (see #553 ), c is as good as a & b)	#templates × template size	#templates × template size	#actors × template size. Estimate: 1 billion actors × 1 KB per template =~ 1 TB
DeleteActorTemplate (winner: c > a > b)	Blocks until all actors are deleted. (Least flexible)	Does not block. Accidental deletion can cause outages. The user needs to either 1. ListActors() until pageToken=="", 2. Or "drain" a template by calling UpdateActor(replacementTempl) + ResumeActor(), then delete the template. After deleting the template, the above logic will still take effect, so ResumeActor will still succeed.	Does not block. Accidental deletion does not cause outages. Actors can still resume with their self-contained spec.
When can users delete an ActorTemplate?	When no actors reference it.	When the draining mechanism above is in place and has soaked.	When users will never call UpdateActor or CreateActor with the target template. May need the same draining logic to be confident.
Vulnerability scanning of container images (winner: a = b > c)	Iterate through all templates, and mark them as disabled. Subsequent ResumeActor calls using this template will fail. Customers must do UpdateActor(newTempl) + Resume.	Iterate through all templates, and mark them as disabled. If the template is already deleted, existing actors referencing it cannot resume without updating their template first.	Iterate through all Actors (and label them as "upgrade required").
List Actors by AT (winner: a = b > c)	Look up by foreign key.	Look up by foreign key while the AT still exists.	Table scan. Look up all Actors that reference the AT. ActorTemplates are immutable.

Conclusion

• Option c paves the road best for RL use cases that require creating an Actor without a template. See issue #553.
• Option b is error prone because "Accidental deletion of templates will cause outages."

[thockin]

Option A comes at a potentially signficant cost. Actor is the center of a spiderweb of cross-object dependencies. We currently inhibit deletion of Atespaces when some other resource is "in" them, but not much else. Foreign keys and RI are not free. If we open this door it will be hard to close, and will set precedent.

Option B is simple and is sort of "your break it, you buy it", but has sharp edges. If we do this, we establish a precendent, too.

Option C is expensive in a different way but does solve another problem.

Julian Gutierrez Oschmann (@juli4n) Jet Chiang (@supreme-gg-gg) Joe Betz (@jpbetz) Luiz Oliveira (@laoj2)

[jpbetz]

For Option B, the ability to bypass immutability by deleting and then creating a template with the same name feels like the sort of thing we want to disallow but that the design would encourage. So I'm only seriously considering A and C.

For Option C, if the template is deleted, what happens to the golden snapshot? If it get's deleted, any actors needing it are broken if they haven't created their own snapshot yet? Is this somehow addressed?

[zoez7]

For Option C, if the template is deleted, what happens to the golden snapshot

Good question. Once a template is deleted:

• Actors that don't need the golden snapshots are:
  • Any actors that has resumed at least once, AND are configured to use "full memory snapshot" or "DurDir only" during resume.
• Actors that will need the golden snapshots are:
  • Actors that are created, but hasn't resumed yet.
  • Actors that are configured to always use "Golden + DurDir" during resume at the template level.

If we go with Option C, deleting the template has to leak the golden snapshots, but GC still needs to know "when can golden snapshot be deleted". Then we need to keep track of the golden snapshot's refcount, and only delete it when the refcount drops to 0.