New-M365DSCReportFromConfiguration -Type HTML does not work with huge source/destination data

[myC-Hive]

Description of the issue

for any reason, creating reports works perfectly if i use just a few DSC Ressources.
As soon as i have a larger Dataset e.g.: 6MB RAW Data (Source and Dest) it just does not work. No errors but it seems that PS hangs somehow.
If you need the sourcedata i can send them over to you, but i cannot upload it public here. thanks a lot.
FYI: my customer has about 25k Users, lots of groups and lots of everything ;-)

Microsoft 365 DSC Version

1.26.218.1

Which workloads are affected

other

The DSC configuration

NA

Verbose logs showing the problem

NA

Environment Information + PowerShell Version

OsName               : Microsoft Windows 11 Enterprise
OsOperatingSystemSKU : EnterpriseEdition
OsArchitecture       : 64-Bit
WindowsVersion       : 2009
WindowsBuildLabEx    : 26100.1.amd64fre.ge_release.240331-1435
OsLanguage           : en-US
OsMuiLanguages       : {en-US, de-DE, fr-FR, it-IT}

Key   : PSVersion
Value : 5.1.26100.7462
Name  : PSVersion

Key   : PSEdition
Value : Desktop
Name  : PSEdition

Key   : PSCompatibleVersions
Value : {1.0, 2.0, 3.0, 4.0...}
Name  : PSCompatibleVersions

Key   : BuildVersion
Value : 10.0.26100.7462
Name  : BuildVersion

Key   : CLRVersion
Value : 4.0.30319.42000
Name  : CLRVersion

Key   : WSManStackVersion
Value : 3.0
Name  : WSManStackVersion

Key   : PSRemotingProtocolVersion
Value : 2.3
Name  : PSRemotingProtocolVersion

Key   : SerializationVersion
Value : 1.1.0.1
Name  : SerializationVersion

[FabienTschanz]

Jep, that's a limitation of the DSCParser that we use under the hood. If you want to share the big file with me, you can reach me on Linkedin. I have a PR open which will refactor the DSCParser to use C# under the hood, massively improving the performance (see microsoft/DSCParser#71). Until that one is merged and released, we unfortunately can't do more.

If you're interested, I can measure the time when all optimizations start kicking in. I have my own module stack already up to date with the new parser version and some other improvements that are coming once the DSCParser is publicly available.

[myC-Hive]

Thanks for your reply! Given this, I don’t think it makes much sense to send you any data, since you’ve already identified that this is a limitation.
I’ll live with it and probably write my own parser. I plan to export each action as a single element, so the data shouldn’t be as large as before.
After that, I’ll need to figure out how to generate delta reports that consolidate everything in one place.

[FabienTschanz]

Feel free to take a look in the functions that we provide. Some of them are pretty complicated though, so if you have any questions, reach out to us. We're happy to help. I hope that the updated DSCParser will be released during late March or April.

[myC-Hive]

First Shot -- i didn't care a lot about the existing functions and so on and tried to create my own diff report by using just the dsc files and it actually worked pretty good. for sure it's v0.1 (no error handling, no formatting, no nothing) but a good start:

i store my dsc exports in a format like c:\M365DSC#COMPONENT#Date#
this script reads the data from all Components based on the most current date and the date before.
based on that information it creates a delta report in html form.
until now its super fast. the whole script took less than a minute for config files with a datasize at about 50mb.

#region generate html report

================================

CONFIGURATION

================================

$basePath = "C:\M365DSC"
$reportPath = "C:\M365DSC-Reports\Report_$(Get-Date -Format 'yyyy-MM-dd').html"
$FlowUrl = "this flow uri is a webhook that sends the data over to a ms teams channel"

$combinedResults = @()

================================

FUNCTIONS

================================

function Parse-M365DSCConfig {
param([string]$Path)

if (-not (Test-Path $Path)) { return @{} }

$content = Get-Content $Path -Raw

$resourcePattern = '(?ms)(?<Type>[A-Za-z0-9]+)\s+"(?<Name>[^"]+)"\s*\{(?<Body>.*?)\}'
$matches = [regex]::Matches($content, $resourcePattern)

if ($matches.Count -eq 0) { return @{} }

$resources = @{ }

foreach ($m in $matches) {

    $type = $m.Groups["Type"].Value.Trim()
    $name = $m.Groups["Name"].Value.Trim()
    $body = $m.Groups["Body"].Value

    $key   = "$type::$name"
    $props = @{ }

    $propPattern = '^\s*(?<Prop>[A-Za-z0-9_]+)\s*=\s*(?<Value>.+?)\s*;?\s*$'

    foreach ($line in ($body -split "`r?`n")) {

        if ($line -match $propPattern) {

            $propName = $Matches["Prop"]
            $rawValue = $Matches["Value"].Trim()

            if ($rawValue -match '^@\((.*)\)$') {
                $inner = $Matches[1]
                $items = $inner -split "," | ForEach-Object { $_.Trim().Trim('"') }
                $props[$propName] = $items -join ","
            }
            else {
                $props[$propName] = $rawValue.Trim('"')
            }
        }
    }

    $resources[$key] = $props
}

return $resources

}

function Build-GroupedSection {
param (
[string]$Title,
[string]$Color,
[array]$Data,
[scriptblock]$GroupBy
)

$html = "<h2 style='color:$Color'>$Title ($($Data.Count))</h2>"

if (-not $Data -or $Data.Count -eq 0) {
    return $html + "<p>No changes detected.</p>"
}

$groups = $Data | Group-Object $GroupBy | Sort-Object Name

foreach ($group in $groups) {
    $html += "<h3>$($group.Name) ($($group.Count))</h3>"
    $html += ($group.Group | ConvertTo-Html -Fragment)
}

return $html

}

================================

DISCOVERY

================================

$policyFolders = Get-ChildItem -Path $basePath -Directory

foreach ($policy in $policyFolders) {

$subFolders = Get-ChildItem -Path $policy.FullName -Directory

$foldersWithDates = foreach ($f in $subFolders) {
    if ($f.Name -match '\d{8}$') {
        [PSCustomObject]@{
            FullName = $f.FullName
            Date     = [datetime]::ParseExact($f.Name.Substring($f.Name.Length - 8, 8), 'yyyyMMdd', $null)
        }
    }
}

if (-not $foldersWithDates) { continue }

$sorted = $foldersWithDates | Sort-Object Date -Descending

$latestDate = $sorted[0].Date
$latest     = $sorted | Where-Object { $_.Date -eq $latestDate } | Select-Object -ExpandProperty FullName

$previousDate = ($sorted | Where-Object { $_.Date -lt $latestDate } | Select-Object -First 1).Date
$previous     = @()

if ($previousDate) {
    $previous = $sorted | Where-Object { $_.Date -eq $previousDate } | Select-Object -ExpandProperty FullName
}

$combinedResults += [PSCustomObject]@{
    PolicyName       = $policy.Name
    LatestFolders    = $latest
    PreviousFolders  = $previous
}

}

================================

DIFF ENGINE

================================

$AddedProps = @()
$RemovedProps = @()
$ModifiedProps = @()

foreach ($r in $combinedResults | Where-Object { $_.PreviousFolders }) {

$oldFile = Join-Path $r.PreviousFolders "M365TenantConfig.ps1"
$newFile = Join-Path $r.LatestFolders  "M365TenantConfig.ps1"

$OldConfig = Parse-M365DSCConfig $oldFile
$NewConfig = Parse-M365DSCConfig $newFile

$OldKeys = $OldConfig.Keys
$NewKeys = $NewConfig.Keys

# Resource-level added
foreach ($k in $NewKeys | Where-Object { $_ -notin $OldKeys }) {
    $AddedProps += [PSCustomObject]@{ Resource = $k; Property = "*Added Resource*"; NewValue = "Exists in new configuration" }
}

# Resource-level removed
foreach ($k in $OldKeys | Where-Object { $_ -notin $NewKeys }) {
    $RemovedProps += [PSCustomObject]@{ Resource = $k; Property = "*Removed Resource*"; OldValue = "Existed in previous configuration" }
}

# Existing resources
foreach ($key in ($OldKeys | Where-Object { $_ -in $NewKeys })) {

    $oldProps = $OldConfig[$key]
    $newProps = $NewConfig[$key]

    foreach ($prop in ($oldProps.Keys + $newProps.Keys | Sort-Object -Unique)) {
        $oldVal = if ($oldProps.ContainsKey($prop)) { $oldProps[$prop] } else { $null }
        $newVal = if ($newProps.ContainsKey($prop)) { $newProps[$prop] } else { $null }

        if (-not $oldVal -and $newVal) {
            # Added property
            $AddedProps += [PSCustomObject]@{
                Resource = $key
                Property = $prop
                NewValue = $newVal
            }
        }
        elseif ($oldVal -and $oldVal -ne $newVal) {
            # Modified only if old value exists
            $ModifiedProps += [PSCustomObject]@{
                Resource = $key
                Property = $prop
                OldValue = $oldVal
                NewValue = $newVal
            }
        }
    }
}

}

================================

HTML REPORT

================================

$summary = @"

Microsoft365DSC Configuration Change Report

Generated: $(Get-Date)

---

Summary

• Added Properties/Resources: $($AddedProps.Count)
• Removed Properties/Resources: $($RemovedProps.Count)
• Modified Properties: $($ModifiedProps.Count)

"@

$addedPropsTable = Build-GroupedSection -Title "Added Properties/Resources"
-Color "green" -Data $AddedProps
-GroupBy { ($_."Resource" -split "::")[0] }

$removedPropsTable = Build-GroupedSection -Title "Removed Properties/Resources"
-Color "red" -Data $RemovedProps
-GroupBy { ($_."Resource" -split "::")[0] }

$modifiedPropsTable = Build-GroupedSection -Title "Modified Properties"
-Color "orange" -Data $ModifiedProps
-GroupBy { ($_."Resource" -split "::")[0] }

$style = @"

<style> body { font-family: Segoe UI, Arial, sans-serif; margin: 30px; } h1 { color: #3366; } table { border-collapse: collapse; width: 100%; margin-bottom: 30px; } th, td { border: 1px solid #ccc; padding: 8px 12px; text-align: left; } th { background-color: #f2f2f2; } tr:nth-child(even) { background-color: #fafafa; } </style>

"@

$html = ConvertTo-Html -Head $style -Body "$summary $addedPropsTable $removedPropsTable $modifiedPropsTable"

$html | Out-File -FilePath $reportPath -Encoding UTF8

Invoke-RestMethod -Method Post -Uri $FlowUrl -Body $html -ContentType "String/html"

#endregion

[FabienTschanz]

Pretty decent, I have to admit. Regex works well if you always have the exact same format of the configuration, but one missing semicolon might turn it into a nightmare if I read the discovery regex correctly. If you write your own configuration, then semicolons are missing most of the time. But keep it going, and if the improved comparison engine comes out, you might want to give it a try.

Is it okay if I close the issue?

[myC-Hive]

For sure i will go with the improved version! And yes… pls close the issue ! Von: Fabien Tschanz ***@***.***> Gesendet: Freitag, 27. Februar 2026 14:49 An: microsoft/Microsoft365DSC ***@***.***> Cc: Thomas Bernhard ***@***.***>; Author ***@***.***> Betreff: Re: [microsoft/Microsoft365DSC] New-M365DSCReportFromConfiguration -Type HTML does not work with huge source/destination data (Issue #6932) [https://avatars.githubusercontent.com/u/71251572?s=20&v=4]FabienTschanz left a comment (microsoft/Microsoft365DSC#6932)<#6932 (comment)> Pretty decent, I have to admit. Regex works well if you always have the exact same format of the configuration, but one missing semicolon might turn it into a nightmare if I read the discovery regex correctly. If you write your own configuration, then semicolons are missing most of the time. But keep it going, and if the improved comparison engine comes out, you might want to give it a try. Is it okay if I close the issue? — Reply to this email directly, view it on GitHub<#6932 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRFEUWNP7PIFC5JVBYFAQYT4OBDN3AVCNFSM6AAAAACV6IBN4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNZTGA2TQNJUGM>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>