Any interest left in afp?

[camprevail]

I've spent quite a bit of time digging in afp shit lately with museca. I've kinda figured out what some of the unknown flags like 0x800000000 and 0x2000000000 are doing because I've come across them on scenes that afputils wasn't able to fully render. They appear to be morphing shapes and maybe masking, doing things like linear and radial wipes. I tried looking for some of these in afp-core and kinda found them but it's really hard to see what it's doing with them. Wasn't sure if you had any interest left but I'd love to pick your brain if you do.

Also, the current theory we have is that konami is just converting swf files created in flash or animate (based on the converter string in ifs version.xml and all the similarities in the format), so I've been pondering on the idea of trying to create that, maybe parsing an swf with a python swf lib and adding support to afputils to convert the objects and then serialize to afp. Obviously there are many challenges with that idea, like figuring out how the unknown flags & such correlate to flash functions, as well as converting the vector shapes to mesh and dumping them to GE2D, but it sounds easier than trying to create a whole custom studio software.

Anyways, curious if you have any thoughts on all this and if maybe you'd be interested in attempting to create a conversion utility. It's probably a long shot in asking but I learn, understand and write code at a snails pace compared to you lol.

[jorpz]

A conversion utility would be the holy grail of custom content for many, whether that be porting/creating custom pop'n characters or museca plus animations. At one point SOMEONE was working on doing something with this but I have not been able to find that project in a number of years. I don't know how far they got with it but I would assume at this point that entire project is defunct.

[camprevail]

Interesting! I haven't heard about that one, but I did see a video of a wip AEP renderer maybe 2 years ago for like jubeat or gita. But yeah that's kinda my goal is making custom animations for M+. Recreating animations from scratch sounds terrifying though, some of them are extremely complicated, plus bytecode lol. Maybe you could go from afp to swf and then swf to FLA to import it into flash. Looks like jpexs decompiler has an swf to fla feature, I haven't tested it though. It's either that or try to inject sprites into afp files, which also sounds like a pain.

Also, might was well mention that I created a pattern file for parsing a deobfuscated afp file in ImHex, makes it pretty easy to explore the binary and make very simple edits to transforms or scales. That can be found here

[DragonMinded]

In super old versions of AFP there's even checks for SWF headers and error messages saying that the file must be converted, so it's quite possible that they're running SWF files through some internal tooling. They used to also support SWF bytecode directly as well as their own bytecode with a flag on the bytecode header as found in the file. Additionally, older AFP versions recognize SWF tags directly. All of that's been since removed from the engine, and the last time I looked at an AFP DLL it was to understand the 3D camera placement tag as well as perspective-correct matrixes, both of which Konami has added to the format. By that time, the bytecode engine could only run the basics (bytecode is where looping happens, as well as where layer masks are set, hence why the renderer has the beginnings of a bytecode engine) and all of the SWF tags, legacy bytecode support and all of the interactive stuff had been stripped.

It was originally the intention to be able to repack AFP files, and you can see that the TXP2 container can be unpacked and repacked. I was even going to go as far as tracking the resulting parsed data back to locations in the file much like you've done, specifically to help with minor adjustments for a Bishi Bashi translation project that never got finished.

Note that one of the things the AFP renderer can do is spit out JSON of the instructions instead of rendering. At some point somebody was using that with a script they wrote to import into Adobe AfterEffects to use it to render the files instead of the AFP renderer. Turns out that since AE is based on flash at its core, all the transforms and effects were still accurate so it was able to render out the files with some massaging. Lends more credence to the idea that these are just converted from SWF internally. It does beg the question: how the hell are they making files that have actual 3D transforms and a camera if SWF only supports affine transforms? Are they fixing up the SWF with some post processing?

That said, I'm burned out on this whole project. Everyone stole what they could from me for a decade and I've also moved on interest-wise, so I'm unlikely to want to get back into it again. But, if somebody were to work on an SWF to AFP converter, it would really force a complete understanding of the file format and fill in a lot of blanks. Like yeah, the AFP code in this repo can render out an astonishing number of files, strip textures, repack TXP2 files, and give insight into code such as the Bishi Bashi levels. But, it's missing a lot. Things that probably are important to the AFP renderer in Konami games but didn't seem important to just playing files back. Making a SWF->AFP converter and then seeing how the game renders it would reveal a LOT.

[camprevail]

First time I've heard of that - someone using AE to render afp. AE does support some of these wtf effects like linear and radial wipes, while flash/animate has no way to easily create them. I wonder if they were using one or the other, doing it in actionscript or just bouncing the file between AE and Flash?

As for 3D, apparently Stage3D support was added to flash player in 2011 with more additions in following years, so there are tools to add 3D animations to flash but it's really only supported in actionscript? Idk I'm just doing a little research on it.
Stage3D objects depart from the traditional SWF rendering model in that they cannot be added to SWF's display lists, instead they must be instantiated via ActionScript
So I wonder if they were just converting the AS3 for 3D stuff into matrices instead of having their bytecode engine handle it. Same with the linear/radial wipes, those appear at the end of the tag, not in the bytecode section.

https://airsdk.dev/docs/development/display/working-in-three-dimensions/understanding-3d-display-objects-in-flash-player-and-the-air-runtime

https://en.wikipedia.org/wiki/Stage3D

[DragonMinded]

Huh, interesting. I genuinely do not know that much about Flash. Most of my knowledge was referencing docs and RE'd details when understanding AFP since the original implementation I was reversing was still rather close. So, yeah, a lot of that seems plausible.

[camprevail]

After plotting the series of shorts from the 0x800000000 flag in AP2_PLACE_OBJECT tags as XY coordinates, we can now really see how it's performing mask effects à la After Effects. This is the radial wipe from the museca title screen.
I couldn't figure out how they exported this effect though—After Effects stopped supporting SWF export after CS6 (which was the only hope really because all other export formats are video formats), and the only way to export them in SWF is to rasterize them because they are "unsupported features". So maybe they were using some other program to generate them.

0x1000000000 and 0x2000000000 appear to be placing AP2_IMAGE objects using the same coords of the Place Object tag. The weird thing though is that every usage I've seen of this in museca is redundant in the sense that the image is already placed by a shape object reference in the same tag. Disabling this flag didn't seem to affect the render in the afp viewer, so I'm not sure what the purpose is, but it seems to be placing the ap2 image on top of the shape image because changing the values of this flag (an ap2 image object id and two UV coordinates) revealed other areas of a texture canvas.

I'll include these findings in the full format documentation I'm making for my afptools repo, but I wanted to share it here for lurkers too. I'll see if I can figure out how to integrate this into afputils too but no promises.

[DragonMinded]

They might actually have an AE native file converter then? Like, if they have their own entire forked flash engine with custom bytecode and a bunch of new tags and stuff, they could very well have done that in conjunction with a converter.

[camprevail]

Well I tried making a really long, slow radial wipe in AE and the file size didn't change, so it likely only saves the basic information to create the effect

[DragonMinded]

They have to have SOME tool that is capable of generating these AFP files, and I doubt they've authored their own SWF-like studio. So, it's quite possible somebody wrote something that can make the files but doesn't have a UI, and then they could make it load AE files directly. We've talked about having a script-based or JSON-based encoder, with the first goal of being able to take an AFP file and convert it to JSON, and then take that JSON and the images and convert it back to an AFP that played the same animation (and ideally would be byte for byte identical). I know it's possible to take the JSON it spits out and import it to AE since somebody I used to work with wrote a script to do just that and render AFP files using AE directly. And, if this code was capable of converting JSON to an AFP then the "only" thing missing would be authoring tools. Hence the utility to convert from AE to that.

At least, that's my best guess as to how they might be doing things internally.

[Bottersnike]

SEGA have a custom editor for building all their surfboard assets, so I wouldn't put it out of the question that konami have a similar tool

[camprevail]

I looked into AE json stuff and it looks like it's mainly leveraged by plugins like Lottie/Bodymovin. Starting to wonder now if they wrote their own AE plugin to be able to export transition effects, and maybe even more