Ags4 feature external dialog

[ericoporto]

Fix #1008

Individual dialogues are saved with their name + ".asd" extension in a directory named Dialogs.

Still need to do:

• Figure how to write the dialogue upgrade process in ags4 way (need help with actual migration code)
• Proper handling of renaming a dialogue (which should rename the file!)
• File Listener: move things to ScriptEditorBase or duplicate in DialogEditor?
• Right before this is merged the AGS 4 version check needs to have the version corrected.
• Should the other dialog xml stuff like the dialog prompts be migrated too? (like to a dDialogName.xml file in the dir, like the rooms stuff)

[ivan-mogilko]

I am having a hard time building ags native because of these errors below

Try switching from "Build + Intellisense" to "Build". Intellisense has its own hiccups sometimes in the mixed code.

OR, you may change this kind of lines above:
using AGS::Common::GUIButton;
to
typedef AGS::Common::GUIButton = AGSButton
and use AGSButton where makes sense below.

[ericoporto]

oh, you are right about intellisense. After your mention, I tried closing the file and running build again and it made the build work (!!!).

@ivan-mogilko , if you want to play with my code, feel free, because I am lost what do I need to do for the loading in the upgrade game files and in the Dialog and DialogScript for it to work in the correct ags4 way. I wouldn't mind some light here of where to change what in my code...

I think that old data should be read and converted by a upgrade task. This is a one-time operation, and modern Dialog class should not know anything about old format.

You made this comment but I have no idea how to load the cdata xml stuff without the Dialog class knowing the old format.

The main problem is the upgrade has Execute(Game game, IWorkProgress progress, CompileMessages errors);, which has a game parameter. So I understood that game is already loaded... If it is already loaded... How do I load it again? I still don't understand this...

I think I would need some way to access the XmlNode that contains the Dialogues before they are iterated, and then I would apply this on the node, and then I can get the dialog script with something like node.SelectSingleNode("Script").InnerText.

[edmundito]

Can you describe the xml schema of the file in the PR description?

[ericoporto]

Not sure what you mean but currently there isn’t, the current CDATA contents from the Dialog under the Script property is what would instead be migrated to its own file. Below is an example from don’t give up the cat

https://github.com/ericoporto/dont-give-up-the-cat/blob/fc2810d9d580f6e1b51742489fa388fbd96cdbd0/dont-give-up-the-cat/Game.agf#L3265

would put a file dIntro.asd in the Dialogs directory with that content.

// Dialog script file
@S  // Dialog startup entry point
TheNarrator: The cat meowed at home.
TheNarrator: And the cat got outside...
TheNarrator: In the cold of the night.
  bool_in_dialog = false;
stop

@ivan-mogilko , I had one idea but not sure about it, but what if there was a property with a name like Legacy or Unprocessed, something like that, that had a type XmlNode that could store the unprocessed xml or point to it somehow, then the Upgrade process could pick it and use it directly. I don’t have an actual idea of how to deal with it. Also I don’t know how to fail the dialog node creation… This is because the upgrade process runs after Game object is created.

[ivan-mogilko]

The title and description in this PR may be misleading, you say that "Individual dialogues are saved with their name + ".asd" extension in a directory named Dialogs.", but it's not dialogs that are saved there, it's dialog scripts. The dialog objects are still in Game.agf.

Or do you plan to move whole dialogs to separate files too?

[ivan-mogilko]

I think that old data should be read and converted by a upgrade task. This is a one-time operation, and modern Dialog class should not know anything about old format.

You made this comment but I have no idea how to load the cdata xml stuff without the Dialog class knowing the old format.

I could have made a mistake. As a variant, it may load script data from xml if it's present. But it needs to test if it's present, and dont treat the missing node as error.

The process may be this:

• When game is loaded from XML, each dialog tries if there's a legacy xml node with dialog script. If there's one then it loads it into the Script property.
• When game upgrade is performed, the upgrade step function takes scripts out of the Dialogs and saves them into the Dialogs/name.asd file. They may probably dispose the Script objects after that, in order to free memory.
• Dialogs should no longer save their scripts back to xml (this must be double checked).

The main problem is the upgrade has Execute(Game game, IWorkProgress progress, CompileMessages errors);, which has a game parameter. So I understood that game is already loaded... If it is already loaded... How do I load it again? I still don't understand this...

You dont need to load Game again.
If the first variant mentioned above is not used, then, as a second variant, this upgrade process may load Game.agf explicitly searching for dialog nodes, and extracting scripts out to the asd files.
I do not have a defined idea on which variant is the best. We may start with the first one, since it will just reuse the existing code that loads CDATA node.

[ericoporto]

It is currently only dialog scripts, but would it make sense to also have a pair xml file with the rest? This would ease following up version changes of the dialogs (in git).

[ivan-mogilko]

Whether it makes sense to have them separated from the project file is a question of project design.

So far the rooms are only ones that have such organization to be stored in separate files. The rooms have their entries in Game.agf, that are loaded up on game load as UnloadedRoom objects, while Room objects are read from the external room file (it was the same both in 3.x with crm files and in 4.x with room folders). If dialogs follow the room-like organization, then the Game.agf should have a dialog reference that is loaded into some kind of a temporary object (UnloadedDialog?? or the Dialog object that has a indication that it is only partially loaded) which only has a dialog name and script name; and the actual Dialog object is created in memory by loading that external file later (either right after Game.agf finishes loading, or when the dialog is opened for editing / game is compiling).

[ericoporto]

I can keep only the DialogScript separate as is, but the thing I really don’t like being in Game.agf is the Dialog Options, to me if they were just there in the directory they would be much easier to check in version control, merge and author. I don’t mind they not being part of the dialog script itself because I guess having them present separately allows easier check of consistency of the script.

For now I don’t use them at all - I can’t just Dialog.Start(option), so what I have is a big switch clause and my own dialog system built on top, so I don’t particularly need them to be easy for me.

[edmundito]

I agree that dialog should be separate for working with multiple people. Each should have its own xml and script file, which will make it easier to track changes and reduce conflicts with game.agf.

Now when it comes to bundling it all together it can be treated as it was before, where all the dialog data is available on game load (unlike rooms.)

[ivan-mogilko]

Loading on start is easy to do. You need to clarify what happens if the xml got modified externally while the project is loaded. Should it update dialog itself similar to how it updates scripts?

[ericoporto]

Loading on start is easy to do. You need to clarify what happens if the xml got modified externally while the project is loaded. Should it update dialog itself similar to how it updates scripts?

I think it should be, but in my mind I was only thinking about dialog options, I have no idea how the rest of things could impact with their changes…

[ivan-mogilko]

Please add a todo note for the project upgrade: if "Dialogs" folder already exists, then it should be renamed. We do this for Rooms and Fonts alike.

ags/Editor/AGS.Editor/Components/FontsComponent.cs

Lines 583 to 588 in e6fe8b8

	// If the fonts directory we want to write to already exists then backup
	if (Directory.Exists(FONT_FILES_DIRECTORY))
	{
	string backupRootDir = Utilities.MakeUniqueDirectory(_agsEditor.CurrentGame.DirectoryPath, FONT_FILES_DIRECTORY, "Backup-");
	Utilities.SafeMoveDirectoryFiles(FONT_FILES_DIRECTORY, backupRootDir);
	}

[ericoporto]

I already copy pasted that code there in the dialog components. But it runs AFTER the game is loaded, so not sure yet what happens if someone opens a project, let the upgrade run, and then don’t save the game and come back to the upgrade process, I think because of this they would lose their dialogs?

[ivan-mogilko]

so not sure yet what happens if someone opens a project, let the upgrade run, and then don’t save the game and come back to the upgrade process,

There's exact same issue with Rooms and Fonts.
That's why the game must be saved automatically after upgrade process. I think this is already done, but must double check that it's true.

[edmundito]

Looks like common list beat us to the file extension (https://asdf.common-lisp.dev/asdf.html,) and we do have conflicts with .asc, and .ash, so wonder if we can do better this time. I am looking at the Linguist manifest to see what's there:
https://github.com/github-linguist/linguist/blob/af6f772786199696e4d07d618c9c5b625a1a03f0/lib/linguist/languages.yml#L1397

From what I gather, "asc"/"ash" in our case was originally intended as "adventure script source/header" (as in .c/.h from C). This is an annotated but restricted "AGS Script" type of file, AGS Dialog Script? .adsc? .dasc? .ascd? (all not taken), for what it's worth, you can call .dasc a "dask" file, the same way I call the files "ask files" and "ash files"