Game Support (FFXIV): Emulate XIV Playback

[Meowchestra]

New one-click setup in Game Support Final Fantasy XIV section (top right toggle button?) to automatically adjust settings and use playback restrictions / custom adjustments to closer emulate xiv?

• Enable FluidSynth Synthesizer (if disabled)
• Download & Enable the FFXIV soundfont (standard version?)
• Enable track-based program changes (compatibility mode)
  simulating track based instrument switching and only care about guitar program changes on guitar tracks for instrument tone changing
• Set polyphony to 16 (global active voice limit, verify fluidsynth polyphony cuts oldest audio first)
• Disable FluidSynth Reverb & Chorus options
• One voice per track (monophonic tracks/channels, bottom-up emulated 50ms chord arpeggiation, cut off/steal oldest note on that specific track)
• No rapid voices (50ms minimum array)
• Ignore midi note velocity (always 100) & ignore control change/key pressure/pitch bend events
• Forced minimum note sample length / held sounds even if the note is much shorter in midi (including piano playback)
• Maximum held sample length of 4s

Reference

internal static partial class Utils
{
    /// <summary>
    /// Convert the Midi for siren
    /// </summary>
    /// <param name="song"></param>
    /// <returns></returns>
    internal static async Task<(MidiFile, Dictionary<int, Dictionary<long, string>>)> GetSynthMidi(this BmpSong song)
    {
        var file = new MidiFile { Division = 600 };
        var events = new AlphaSynthMidiFileHandler(file);
        events.AddTempo(0, 100);

        var trackCounter = byte.MinValue;
        var veryLast = 0L;

        var midiFile = await song.GetProcessedMidiFile();
        var trackChunks = midiFile.GetTrackChunks().ToList();

        var lyrics = new Dictionary<int, Dictionary<long, string>>();
        var lyricNum = 0;

        //Skip first track, it's just track 0 / "None"
        foreach (var trackChunk in trackChunks.GetRange(1, trackChunks.Count - 1))
        {
            var instr = Instrument.None;
            var trackOctaveShift = 0;
            using var manager = trackChunk.ManageTimedEvents();
            var trackName = trackChunk.Events.OfType<SequenceTrackNameEvent>().First().Text;
            var rex = MyRegex();
            if (rex.Match(trackName) is { } match)
            {
                if (!string.IsNullOrEmpty(match.Groups[1].Value))
                {
                    instr = Instrument.Parse(match.Groups[1].Value);
                }

                if (int.TryParse(match.Groups[2].Value, out var os))
                    trackOctaveShift = os;
            }
            var instrumentMap = new Dictionary<float, List<KeyValuePair<NoteEvent, Instrument>>>();

            foreach (var _event in manager.Objects)
            {
                var lyricsEvent = _event.Event as LyricEvent;
                var programChangeEvent = _event.Event as ProgramChangeEvent;

                if (_event.Event is NoteEvent noteEvent && _event.Event.EventType == MidiEventType.NoteOn)
                {
                    {
                        var timeKey = (float)Math.Round((decimal)_event.Time, 3); // Round to milliseconds

                        if (!instrumentMap.TryGetValue(timeKey, out var value))
                        {
                            value                  = [];
                            instrumentMap[timeKey] = value;
                        }

                        value.Add(new KeyValuePair<NoteEvent, Instrument>(noteEvent, instr));
                    }
                }

                if (programChangeEvent != null)
                {
                    if (instr.InstrumentTone.Equals(InstrumentTone.ElectricGuitar))
                        instr = Instrument.ParseByProgramChange(programChangeEvent.ProgramNumber);
                }

                if (lyricsEvent != null)
                {
                    if (lyrics.Count < lyricNum+1)
                        lyrics.Add(lyricNum, new Dictionary<long, string>());
                    lyrics[lyricNum].Add(_event.Time, lyricsEvent.Text);
                    lyricNum++;
                }
            }

            foreach (var note in trackChunk.GetNotes())
            {
                var instrument = instr;
                if (instrumentMap.TryGetValue(note.Time, out var notes))
                {
                    var matchingNote = notes.FirstOrDefault(n => n.Key.NoteNumber == note.NoteNumber);
                    if (matchingNote.Key != null)
                    {
                        instrument = matchingNote.Value;
                    }
                }

                var noteNum = note.NoteNumber + 12 * trackOctaveShift;
                noteNum = Math.Clamp(noteNum, 0, 127);
                var dur = (int)MinimumLength(instrument, noteNum - 48, note.Length);
                var time = (int)note.Time;

                events.AddProgramChange(trackCounter, time, trackCounter, (byte)instrument.MidiProgramChangeCode);
                events.AddNote(trackCounter, time, dur, (byte)noteNum, DynamicValue.FFF, trackCounter);

                trackCounter = trackCounter == byte.MaxValue ? byte.MinValue : (byte)(trackCounter + 1);
                veryLast     = Math.Max(veryLast, time + dur);
            }
            instrumentMap.Clear();
        }
        trackChunks.Clear();
        events.FinishTrack(byte.MaxValue, (byte)veryLast);

        // Force garbage collection to clean up temporary objects
        GC.Collect();
        GC.WaitForPendingFinalizers();

        return (file, lyrics);
    }

	/// <summary>
	/// Converts Melanchall track chunks
	/// Uses the same normalized format as the BmpSong converter:
	///   Division=375, Tempo=160 BPM → 1 tick ≈ 1 ms
	///   All times are relative to the first note (firstNote alignment)
	/// </summary>
	internal static MidiFile ConvertToSirenMidi(List<TrackChunk> tracks, MidiFile_Melanchall sourceMidiFile)
	{
		var tempoMap = sourceMidiFile.GetTempoMap();

		long firstNoteMs = 0;
		var allNotes = tracks.SelectMany(t => t.GetNotes()).OrderBy(n => n.Time).FirstOrDefault();
		if (allNotes != null)
		{
			firstNoteMs = (long)((MetricTimeSpan)TimeConverter
				.ConvertTo<MetricTimeSpan>(allNotes.Time, tempoMap)).TotalMilliseconds;
		}

		// Division=375 + Tempo=160 BPM
		// 160 BPM and 375 ticks/quarter: 1 tick = 60000/(160*375) ≈ 1ms
		var sirenFile = new MidiFile { Division = 375 };
		var handler = new AlphaSynthMidiFileHandler(sirenFile);
		handler.AddTempo(0, 160);

		var trackCounter = byte.MinValue;
		var veryLast = 0L;

		foreach (var trackChunk in tracks)
		{
			// Read instrument from TrackName
			Instrument instr = Instrument.None;
			int octaveShift = 0;

			var trackNameEvent = trackChunk.Events
				.OfType<SequenceTrackNameEvent>().FirstOrDefault();
			if (trackNameEvent != null)
			{
				var rex = new Regex(@"^([A-Za-z _:]+)([-+]\d)?");
				if (rex.Match(trackNameEvent.Text) is Match m)
				{
					if (!string.IsNullOrEmpty(m.Groups[1].Value))
						instr = Instrument.Parse(m.Groups[1].Value.Replace(":", ""));
					if (int.TryParse(m.Groups[2].Value, out int os))
						octaveShift = os;
				}
			}

			// Track guitar program change switching
			var instrumentMap = new Dictionary<long, Instrument>();
			var currentInstr = instr;

			var timedEvents = trackChunk.GetTimedEvents()
				.OrderBy(te => te.Time)
				.ThenBy(te => te.Event is ProgramChangeEvent ? 0 : 1)
				.ToList();

			foreach (var te in timedEvents)
			{
				if (te.Event is ProgramChangeEvent pc)
				{
					if (currentInstr.InstrumentTone.Equals(InstrumentTone.ElectricGuitar))
						currentInstr = Instrument.ParseByProgramChange(pc.ProgramNumber);
					continue;
				}
				if (te.Event is NoteOnEvent noteOn && noteOn.Velocity > 0)
					instrumentMap[te.Time] = currentInstr;
			}

			// Convert notes
			foreach (var note in trackChunk.GetNotes().OrderBy(n => n.Time))
			{
				var instrument = instrumentMap.TryGetValue(note.Time, out var mapped)
					? mapped : instr;

				int noteNum = (int)note.NoteNumber + (12 * octaveShift);
				noteNum = Math.Max(0, Math.Min(127, noteNum));

				// Absolute milliseconds via TempoMap (all tempo changes considered)
				long startMs = (long)((MetricTimeSpan)TimeConverter.ConvertTo<MetricTimeSpan>(note.Time, tempoMap)).TotalMilliseconds;
				long endMs = (long)((MetricTimeSpan)TimeConverter.ConvertTo<MetricTimeSpan>(note.Time + note.Length, tempoMap)).TotalMilliseconds;
				long lenMs = Math.Max(1, endMs - startMs);

				// firstNote alignment
				long alignedStart = startMs; // Math.Max(0, startMs - firstNoteMs);

				// Apply minimum length
				long finalLen = MinimumLength(instrument, noteNum - 48, lenMs);

				handler.AddProgramChange(trackCounter, (int)alignedStart, trackCounter, (byte)instrument.MidiProgramChangeCode);
				handler.AddNote(trackCounter, (int)alignedStart, (int)finalLen, (byte)noteNum, DynamicValue.FFF, trackCounter);

				if (trackCounter == byte.MaxValue) trackCounter = byte.MinValue;
				else trackCounter++;

				long end = alignedStart + finalLen;
				if (end > veryLast) veryLast = end;
			}

			instrumentMap.Clear();
		}

		handler.FinishTrack(byte.MaxValue, (byte)Math.Min(veryLast, byte.MaxValue));
		return sirenFile;
	}

    private static long MinimumLength(Instrument instrument, int note, long duration)
    {
        switch (instrument.Index)
        {
            case 1: // Harp
                return note switch
                {
                    <= 19 => 1338,
                    <= 28 => 1334,
                    _     => 1136
                };

            case 2: // Piano
                return note switch
                {
                    <= 18 => 1531,
                    <= 25 => 1530,
                    <= 28 => 1332,
                    _     => 1531
                };

            case 3: // Lute
                return note switch
                {
                    <= 14 => 1728,
                    <= 28 => 1727,
                    _     => 1528
                };

            case 4: // Fiddle
                return note switch
                {
                    <= 3  => 634,
                    <= 6  => 632,
                    <= 11 => 633,
                    <= 15 => 634,
                    <= 18 => 633,
                    _     => 635
                };

            case 5: // Flute
            case 6: // Oboe
            case 7: // Clarinet
            case 8: // Fife
            case 9: // Panpipes
                if (duration > 4500) return 4500;
                return duration < 500 ? 500 : duration;

            case 10: // Timpani
                return note switch
                {
                    <= 15 => 1193,
                    <= 23 => 1355,
                    _     => 1309
                };

            case 11: // Bongo
                return note switch
                {
                    <= 7  => 720,
                    <= 21 => 544,
                    _     => 275
                };

            case 12: // BassDrum
                return note switch
                {
                    <= 6  => 448,
                    <= 11 => 335,
                    <= 23 => 343,
                    _     => 254
                };

            case 13: // SnareDrum
                return 260;

            case 14: // Cymbal
                return 700;

            case 15: // Trumpet
            case 16: // Trombone
            case 17: // Tuba
            case 18: // Horn
            case 19: // Saxophone
            case 20: // Violin
            case 21: // Viola
            case 22: // Cello
            case 23: // DoubleBass
            case 24: // ElectricGuitarOverdriven
                return duration > 4500 ? 4500 : duration < 300 ? 300 : duration;
            case 25: // ElectricGuitarClean
            case 27: // ElectricGuitarPowerChords
                if (duration > 4500) return 4500;
                return duration < 300 ? 300 : duration;
            case 26: // ElectricGuitarMuted
                return note switch
                {
                    <= 18 => 186,
                    <= 21 => 158,
                    _     => 174
                };
            case 28: // ElectricGuitarSpecial
                return 1500;

            default: return duration;
        }
    }

Disabling the button toggle should reset everything to default. (Poly 256, reverb & chorus enabled, xiv soundfont disabled, no forced sample duration / limitations.)

[rvaidya]

Hello! Since I had to deal with this recently, here are a couple notes when you get to implementing this so you don't have to rediscover them:

• You will probably want to apply note attack compensation if the soundfont samples are directly extracted from the game's resources. Here are the timings that Midibard2 uses. Fast songs with a mix of slow attack and fast attack instruments will sound odd without it. https://github.com/reckhou/MidiBard2/blob/60d066d0812b3b79c0213880f57e5de05e561d96/Midibard/Managers/EnsembleManager.cs#L185
• FFXIV instruments differ in release/sustain - some instruments play the entire sample (drums, piano, some guitar modes) regardless how long you press the key, and some sustain linearly for how long the key is held - I clamped this at 300...1000ms. So if an Oboe note is held for 700ms, on release the note will sustain/fade out over approximately 700ms. The game seemingly does clamp sustain itself to about 1s (didn't measure this exactly, sort of approximated it in my head while pressing keys ingame).

[Meowchestra]

Hi @rvaidya! I should have a good grasp on the engine limitations from my involvement with the bard community but appreciate you looking out. ❤️

I already apply sample compensation (via startAddrsOffset) in my XIV soundfont so that shouldn't be a concern. I calculate the padding differently than midibard but MB2 does contain my updated clarinet correction.
https://github.com/Meowchestra/FFXIV-SoundFont/releases

Sustains have a hard cutoff after ~4s (noteoff event) regardless of how long a sustained instrument is held. Instruments that do not sustain I was just going to copy how BardMusicPlayer / LightAmp's preview synthesizer handles note durations since I didn't care to double check the exact release timing. (See MinimumLength in above Reference.)