Python bindings?

[KeygenOld]

I know this hasn't been updated in a while, but most MLV stuff is old anyway.

I checked your PDF docs but there don't seem to be any python bindings listed. Am I missing something or are we expected to just dig through the .py files? 😬

Btw, this seems like a great tool. I've been using MLV for a long time and I'm working on a python script so I can just output the image sequence formats I need in one go. ffmpeg can sort of do it, but it leaves things green because it doesn't apply the white balance metadata from the MLV and getting 16-bit or float output is proving to be a pain.

Specifically, I'd like to test outputting uint16 TIFF and float32 RGB EXR files. Point in the right direction would be appreciated for how to do this in python, unless we're just supposed to use subprocess or os.system commands with it.

Thanks

[so-rose]

Hey @KeygenLLC,

Cool to see someone's still using my old program!

Solution

I'll start by solving your problem! You can easily get float16 EXR sequence output with

convmlv -i file.mlv

which uses camera WB and writes Linear sRGB. If you want uint16 TIFF, just add -t 1. You can also set start and end frames. See convmlv -h for all the variations thereof.

If it's too slow dumping DNGs, use MLVFS to get DNGs "instantly", and use -k to avoid dumping DNGs. Something I've been planning on adding to be automatic, but haven't gotten around to yet.

There's also no good reason to use float32 for the output of camera footage. No camera making MLV footage is sensing 14 stops.

Bindings?

From your description, convmlv can already meet your needs without needing a wrapper script. But if you still want bindings:

convmlv is (unfortunately!) implemented in bash, not Python. Even if it weren't, it would still depend heavily on a shell interface. All the magic happens in the big bad convmlv.sh script 😇. So, in sum: No Python bindings. But, being a script with a well-defined interface (see convmlv -h), subprocess.run is going to work pretty nicely. It can be made very clear and readable what is going on 😊

The Python scripts you see just perform supporting functions too hard in raw bash, like Grey World AWB.

If you're interested I could see about adding some bindings, but I hope it can meet your needa until then😊

[KeygenOld]

Thanks for the info and the quick reply.

float16 should work. My pipeline is just designed around float32 EXR. Don't want you to go to the trouble for nothing. Main thing is not to use int and grab as much data as possible from the raw sensor data and make sure things don't clip or have the wrong gamma applied or a green cast like ffmpeg.

I was tinkering around recently with MLV App, which is pretty nice and helped reduce noise and smooth out things, just very laggy (doesn't seem to use GPU) and only seems to output regular Tiffs. I'm still searching for a tool like that with float and 32-bit support—something like Adobe Camera Raw that supports floating point images and 32-bit and supports image sequences. It's hard to get the stylized ACR looks I've developed for projects using other tools, or directly in After Effects without it, but nothing I've found can read the Adobe presets and apply similar effects and most everything lacks a clarity control (midtone contrast). I probably need to just hire someone to code that at some point. Even Lumetri lacks a few tools from camera raw that would make it possible.

That's great though. It's fine if it uses bash. I can easily execute bash commands from Python. It's not about wanting a wrapper, it's because I'm building various Python tools that are manipulating the images after conversion, so it would be the first step to convert the MLV (in a blocking manner), then read in the images for further processing. I'm on macOS and there seems to be a bug in python where the blocking subprocess stuff doesn't block (though using os.system commands will make it block).

I have to read through your docs some more, but do they happen to allow the applying of a 3d lut, or perhaps applying a gamma curve that's more log-like if images were shot without a log-like curve? So far I've only found one github repo with a python script that allows the applying of 3d lut .cube, uses cmake to build, but it crashes when trying to build on mac Mojave. I typically use the Cinestyle curve from Technicolor when shooting MLV on Canon 5D III. MLV App is only thing that reads the raw data properly and keeps that curve intact. Everything else wants to apply an sRGB curve that makes it way too dark.

Anyway, sorry for the rambling. Have a nice holiday and I will definitely try out convmlv.

[so-rose]

Sounds like a cool project! Yeah, MLV App is a lovely bit of software. The tonemapping and interactivity is especially lovely; it's so easy to get a nice final result, and the color controls are handy.

Tl;dr convmlv has both 3D LUTs and log-like gamma curves!

What it does and doesn't do

convmlv is entirely headless, made explicitly to never ever lose data that the sensor made. The output strives to be unopinionated and technically sound, meaning not neccesarily pretty. That's the footage I at least always want as the blank slate to grade against, when I boot up ex. Resolve.

Your midtone slider is, for example, something I philosophically don't believe belongs in a raw footage developer. Handy as ACR is, I don't want artistic decisions split between incompatible apps. Let the fancy grading suites take care of that.

convmlv has thus essentially no unique features, inheriting instead the features of the tools it glues together (dcraw, ffmpeg, mlv_dump, etc.). For example, there are 5 quality noise reduction methods, astrophotography-based darkframe subtraction, deshake, several demosaic methods, bad pixel removal, etc., all courtesy of the underlying tool's capabilities.

In practice, it can be pseudo-interactive; you can develop one frame, change the command, run again, the frame auto-reloads, etc. . This is how I test settings.

(There's also a config file format allowing global, project-local & per-filename adjustments. I've used that to define how to develop an entire project from one config.)

More on Curves

In terms of curves, RAW data is always just as linear as the sensor. Sensors have natural clip points too, so float vs int doesn't matter in the slightest; # of values possible per pixel is all that matters.

So when MLV App "preserves the curve", it's actually developing linear data, then making an "artistic" decision to apply the log curve mentioned in the MLV header. convmlv doesn't care about art, only data.

But, sometimes you need pipeline integration - a technical, not artistic, curve. So, convmlv will gladly transform your linear data to something more logarithmic for you, if you ask for it. The -g options lets you output SLog3, C-Log, Cineon, ~2.2, and/or linear values. With the -G option you can combine any gamma function with your non-sRGB gamut of choice (dcraw outputs XYZ from an estimated camera color space, convmlv LUTs it on to whatever space).

If you like your custom curve: With exiftool you can likely read the curve function/1D LUT off of the header of an MLV file that had the curve applied, and make a new 1D LUT out of it.

1D LUTs aren't supported by convmlv (only 3D), but if you're already juggling float values you should pretty easily be able to transform the values after outputting linearly from convmlv.

LUTs

convmlv does however take any number of 3D LUTs (as well as a host of other ffmpeg filters for ex. deshake, denoising, etc.), with the caveat that they cannot be larger than 64x64x64. The docs include a command to convert from the standard x65 to x64 painlessly.

This is really only made for color space transforms. But you can cheat and apply an artistic 3D LUT too 😊

A little tip; you can actually capture any global (every pixel runs the exact same function) effect from ACR and any other software, if you use a HALD CLUT. Just generate an image where each pixel is one of the values in the identity 3D LUT. You pass that through whatever fancy global effect you want, and read the resulting values right into a text formatted ex. 3dl file.

[KeygenOld]

Sounds like a cool project! Yeah, MLV App is a lovely bit of software. The tonemapping and interactivity is especially lovely; it's so easy to get a nice final result, and the color controls are handy.

Yeah, I wish I could use it as a second stage AFTER getting the files out of the MLV, so I have the "raw" starting point to always go back to without needing to export again.

Tl;dr convmlv has both 3D LUTs and log-like gamma curves!

👍🏻

The output strives to be unopinionated and technically sound, meaning not neccesarily pretty. That's the footage I at least always want as the blank slate to grade against, when I boot up ex. Resolve.

Same here, I just lack the GPU power for Resolve, so I manually do things in AE and bounce pre-comps as needed. Not ideal but works for now. That's why I use Cinestyle. I like to push and pull like mad in post using a linear workflow. Give me the pedestrian, blue-collar, technical output to begin with that is plain, drab and boring, and I'm a happy guy.

Your midtone slider is, for example, something I philosophically don't believe belongs in a raw footage developer. Handy as ACR is, I don't want artistic decisions split between incompatible apps. Let the fancy grading suites take care of that.

Agreed. I do that non-destructively (usually) and then I'll spit an EXR sequence out after I have the grade nailed so the CPU doesn't have a stroke. I work with very dense, nested compositions in AE, so I innevitably have to start taking precomps and exporting them as float32 EXR sequences to make sure I can work fast but pull things back if needed.

Only downside is if I need to change any of that baked in stuff, I have to go back and re-export, but it's the only reasonable way to work IMO or else things get too sluggish after a while and AE is not very optimized for GPU.

convmlv has thus essentially no unique features, inheriting instead the features of the tools it glues together (dcraw, ffmpeg, mlv_dump, etc.). For example, there are 5 quality noise reduction methods, astrophotography-based darkframe subtraction, deshake, several demosaic methods, bad pixel removal, etc., all courtesy of the underlying tool's capabilities.

I really want to try out the darkframe reduction. Sometimes if footage is grabbed and it was just a little too dark, Canon sensor just adds so much garbage artifacts/noise and streaking that it can only be used of stylized.

In practice, it can be pseudo-interactive; you can develop one frame, change the command, run again, the frame auto-reloads, etc. . This is how I test settings.

This is the plan. Figure I'll create a few loops in Python and have it spit out a ton of image sequences to get an idea of what each one does. Will eventually probably add an arg with argparse so I can set a --test flag, that will spit out a short range or a single image using a variety of settings. Then remove the flag and do a final pass to export all. Already built a framework when testing ffmpeg with progress bars and all that jazz.

(There's also a config file format allowing global, project-local & per-filename adjustments. I've used that to define how to develop an entire project from one config.)

Very cool idea. I will have to try that. You've added a lot of features for a one-man show.

So when MLV App "preserves the curve", it's actually developing linear data, then making an "artistic" decision to apply the log curve mentioned in the MLV header. convmlv doesn't care about art, only data.

What I meant is that other apps (like ACR) will try to force their curve upon your images, typically some crappy sRGB curve that crushes everything and makes it harder to manipulate without pushing some sliders too extremes—defeating the purpose of using an in-camera curve in the first place. I used one of Adobe's lab tools years ago to try and avoid this, but it never was perfect, even for still CR2.

But, sometimes you need pipeline integration - a technical, not artistic, curve. So, convmlv will gladly transform your linear data to something more logarithmic for you, if you ask for it. The -g options lets you output SLog3, C-Log, Cineon, ~2.2, and/or linear values. With the -G option you can combine any gamma function with your non-sRGB gamut of choice (dcraw outputs XYZ from an estimated camera color space, convmlv LUTs it on to whatever space).

This is great. I want to run some tests to see what will work best—no curve, Cinestyle, or one of the above. Then bring those into PS/AE and test what I can do, and also run through Python tools and see how things fare and some 3rd party scripts.

If you like your custom curve: With exiftool you can likely read the curve function/1D LUT off of the header of an MLV file that had the curve applied, and make a new 1D LUT out of it.

Was trying to figure this out yesterday infact—to grab the curve and white balance kelvin value to use in Python. Failed miserably... 😬

1D LUTs aren't supported by convmlv (only 3D), but if you're already juggling float values you should pretty easily be able to transform the values after outputting linearly from convmlv.

I'm sure numpy could manage something. It's built for manipulating matrices.

convmlv does however take any number of 3D LUTs (as well as a host of other ffmpeg filters for ex. deshake, denoising, etc.), with the caveat that they cannot be larger than 64x64x64. The docs include a command to convert from the standard x65 to x64 painlessly.

👍🏻

This is really only made for color space transforms. But you can cheat and apply an artistic 3D LUT too 😊

I really would love to figure out a way to build a LUT from ACR settings so I could apply that in float32 space to my converted MLV sequences (i.e. python most likely) and then export a float32 RGB EXR sequence to do minor push pull in AE. Unfortunately I'm not the most technical guy and try to remain an artist. At times like these, some more technical know-how would help.

I have been knee-deep in machine learning software recently, and am going to do some tests also. Having some decent results generating slow-mo interpolation this way, and super-resolution, instead of using Magic Lantern where I have to kill the frame size. It's not 100% accurate yet, but promising. Also mapping out idea for my own ML algorithms and such because there are so many possibilities.

A little tip; you can actually capture any global (every pixel runs the exact same function) effect from ACR and any other software, if you use a HALD CLUT. Just generate an image where each pixel is one of the values in the identity 3D LUT. You pass that through whatever fancy global effect you want, and read the resulting values right into a text formatted ex. 3dl file.

I will need to figure out how to do this. Then how to use numpy/opencv to apply that stuff.

Thanks for the in-depth explanation. It's going to be a busy week, most likely filled with failures :)

Have a happy holiday.

[KeygenOld]

btw, found an error in your docs on the top of page 5.

this: sudo python3 -m pip install $(./convmlv -Y)
should be: sudo python3 -m pip install $(./convmlv.sh -Y) (with the .sh)

It requires it on macOS, anyway. Fails without it.

I personally don't like using sudo for package installs, but I also use Anaconda. If using a conda env, you can just do the following:
pip install $(./convmlv.sh -Y) if you don't have numpy and tifffile installed.

If I have some time, I'm going to reformat the PDF with a dark background because it's blinding me. I'll pass it along if you want to post it for others.

P.S. This tool does a lot, man. If you ever want to update it, let me know and I'll help with testing. I'm going to be doing more shooting on my 5D III this year and this tool seems like it will alleviate much headache.

[KeygenOld]

🎄Merry Christmas 🎄

Here's a LUT Generator. It makes the HALD image @ 125 or 512 16-bit. You can apply color correction or ACR (just not localized stuff) and re-save the image, and generate a .cube with the same app. Tested in After Effects and works by applying the Apply Color LUT plugin and after quick test it seems accurate.

I also found darktable: download here

darktable is like ACR but it also works on float32 images (including EXR) and processing everything at 128-bit. Only problem I'm finding is that even with GPU acceleration, it is pretty sluggish.

Still on the hunt for something that can come close to the local contrast enhancement of ACR's "clarity" for use in After Effects and Python.

[KeygenOld]

mlv2badpixels.sh is no longer available at the link provided. Repo is gone. Still scouring the web trying to find a copy.

cr2hdr mac binary (and entire repository) is also missing. Maybe bitbucket did some overhaul and trashed old stuff?

mlv_dump my version must be different. The one you referenced is also gone from bitbucket.

If you can upload them, would appreciate it. I can startup convmlv, and it seems to copy the mlv to a temp folder, but no images are generated.

[so-rose]

Hi again, and thanks!

If you download the bundled release, all the relevant files should be included. Have you tried that one?