Automatically generate foley sounds for all (or all tagged) 3D assets in a Unity SYNK project using AudioLDM and a local LLM (several models tested) to create a meaningful audioLDM prompt from the metadata of a 3D asset.
The flow is as follows
AudioLDM to generate AudioFiles
Guidance to guardrail the local LLM to return structured data only
LlamaCCP a localLLM inference implementation for prompt generation
python to make an interactive CLI application and interact with the FTRACK data