It should be possible to use the math extensions.
In HTML and PDF formulas don't get rendered and the plugin just shows a warning:
asciidoctor: WARNING: conversion missing in backend pdf for stem
There need to be a similar example as it does exist for the asciidocj-diagram extension.
Putting :stem: into the document header or <stem/> into the <attribute>-section of the plugin configuration does not help.
A non working sample could be provided.