Remove IDs and Registries

[nical]

This is a meta issue to describe the long-ish term vision for wgpu-core's resource management and track the steps towards it.

Registries and IDs

Internally, all wgpu-core resources are stored behind Arcs. Externally, they are exposed as non-clonable structs containing an ID which conceptually corresponds to a single reference to the Arc of a resource.

The goal

Instead of having wgpu public types wrap a non-clonabe ID, we would like them to wrap the Arc to the internal resource directly (and be clonable).

TODO

• Resolve cycles in wgpu-core using weak references #5120
• Remove remaining internal use of the IDs.
• Make wgpu-core less generic (brainstorm) #5124
• Rewrite the record-reply functionality Add back support for tracing (trace feature) to wgpu #5974
• Move ErrorSink into wgpu-core #8911
• (Firefox): Move the wgpu-core part of command recording out of the content process
• Move logic present in the Global's methods into methods on wgpu-core objects
• (Firefox): Stop providing the IDs during resource creation and do ID mapping in the gecko-side bindings.
• Remove the registries
• Consider whether serde_object_reference_struct can be removed (AL: when originally added, I had in mind to possibly simplify it/remove macro_rules_attribute after we got rid of registries, although macro_rules_attribute is now also used by ConstDefault.)

Other Related Issues

• IDs of resources used in render bundles are not resolved until finish() #8234
• Tracing does not support render bundles #8837

[sagudev]

(Firefox): Stop providing the IDs during resource creation and do ID mapping in the gecko-side bindings.

How exactly would that look like? Would that be something like hashmap that would map id to Arc of wgpu object that would live in WebGPU thread?

I'm mostly asking for servo, as it would be nice to share ID mapping stuff.

[nical]

(Firefox): Stop providing the IDs during resource creation and do ID mapping in the gecko-side bindings.

How exactly would that look like? Would that be something like hashmap that would map id to Arc of wgpu object that would live in WebGPU thread?

Yes, that's the general idea. Depending on how you generate the IDs you could get away with arrays instead of hash maps (like wgpu-core currently does).

[sagudev]

There are some cases where this would also be useful in servo, for example when doing swapchain present we always need to send new encoder id from content process so we can create new encoder in wgpu_thread, but removing registries would allow us to have concept of wgc types that are not owned by JS (content process).

I think making descriptors generic over IDs/Arced Resources would be beneficial as that would allow servo to still reuse such types for IPC. Something like this is already done in some parts using Arc* variants descriptors:

wgpu/wgpu-core/src/command/timestamp_writes.rs

Lines 8 to 25 in ad194a8

	pub struct PassTimestampWrites {
	/// The query set to write the timestamps to.
	pub query_set: id::QuerySetId,
	/// The index of the query set at which a start timestamp of this pass is written, if any.
	pub beginning_of_pass_write_index: Option<u32>,
	/// The index of the query set at which an end timestamp of this pass is written, if any.
	pub end_of_pass_write_index: Option<u32>,
	}
	/// Describes the writing of timestamp values in a render or compute pass with the query set resolved.
	pub struct ArcPassTimestampWrites {
	/// The query set to write the timestamps to.
	pub query_set: Arc<crate::resource::QuerySet>,
	/// The index of the query set at which a start timestamp of this pass is written, if any.
	pub beginning_of_pass_write_index: Option<u32>,
	/// The index of the query set at which an end timestamp of this pass is written, if any.
	pub end_of_pass_write_index: Option<u32>,
	}

[teoxoy]

Relevant to this issue: #8129 (comment) (possible shape for wgpu-core resources, rather than the current Snatchable + SnatchLock).

[sagudev]

Move the wgpu-core part of command recording out of the content process

This is about render bundles, right? They are currently built in content process (using IDs) and send over IPC and then resolved, but that is wrong AFAIK and will cause problem in migration. I read somewhere that this also complicates firefox as it needs to keep dom object of used resources alive while encoder is still open. I propose we go to same semantics as used by other commands/passes (at least the one that is then also used in servo; IIRC in firefox there is additional buffering of Pass command which are then unrolled on finish) so for every RenderBundleEncoder command we expose a global method that will resolve the id's and then store resolved commands (so all used resources will be kept alive by arc!).

[teoxoy]

Yeah, that item is now mainly about bundles.

IIRC in firefox there is additional buffering of Pass command which are then unrolled on finish

We want to remove this additional buffering as well. We shouldn't need to store the commands in another Vec that on finish gets sent to the GPU process.

so for every RenderBundleEncoder command we expose a global method that will resolve the id's and then store resolved commands

Yes this would be the ideal architecture and we should do this for pass encoders as well. IIRC doing it for passes was also necessary/blocking removal of registries but it might not be.

[teoxoy]

I think I might have some time next week to try to refactor Firefox's passes to no longer buffer in the content process.

[sagudev]

I will prepare PR with global methods for RenderBundleEncoder (but still using existing RenderBundleEncoder) just so we get "public" API ready so we can start doing internal changes.

As for the other passes it might be beneficial to still have the id and stuff in global (in servo we currently do hashmap) tho thing are complicated by &mut (we would need mutex around each renderpass).

My final goal is to have Global doing all and only remoting (for servo and firefox) other users would be able to use resources directly.