Skip to content

Fix Get-DbaDbTable performance regression#10141

Open
potatoqualitee wants to merge 1 commit intodevelopmentfrom
prfix
Open

Fix Get-DbaDbTable performance regression#10141
potatoqualitee wants to merge 1 commit intodevelopmentfrom
prfix

Conversation

@potatoqualitee
Copy link
Member

Resubmit PR from @PeterSawatzki-MGGM which keeps his commits but hopefully is able to merge.

Fix Get-DbaDbTable performance regression when using -Schema or -Table parameters

The ClearAndInitialize optimization introduced in 2.7.7 loads ALL tables in the database before filtering client-side. This causes severe performance issues (30+ seconds vs <1 second) when only specific tables are requested.

This fix adds server-side filtering by passing an XPath-style URN filter to ClearAndInitialize when -Schema or -Table parameters are specified.

Example filter: [@Schema='dispo' and @name='t_auftraege']

Performance improvement:

  • Before: ~30 seconds (loads 1000+ tables)
  • After: <1 second (loads only requested table)

Type of Change

  • Bug fix (non-breaking change, fixes performance regression in 2.7.7)
  • New feature (non-breaking change, adds functionality)
  • Breaking change (affects multiple commands or functionality)
  • Ran manual Pester test and has passed (Invoke-ManualPester)
  • Adding code coverage to existing functionality
  • Pester test is included
  • If new file reference added for test, has is been added to github.com/dataplat/appveyor-lab ?
  • Unit test is included
  • Documentation
  • Build system

Purpose

Fix a severe performance regression introduced in version 2.7.7 where Get-DbaDbTable with -Schema or -Table parameters loads ALL tables in the database before filtering client-side, causing execution times to increase from <1 second to 30+ seconds on databases with many tables.

Approach

The ClearAndInitialize optimization added in 2.7.7 uses an empty filter string, which causes SMO to load all tables:

$db.Tables.ClearAndInitialize('', [string[]]$properties)

This fix constructs an XPath-style URN filter when -Schema or -Table parameters are specified, enabling server-side filtering:

# Example: -Schema "dispo" -Table "t_auftraege"
$db.Tables.ClearAndInitialize("[@Schema='dispo' and @Name='t_auftraege']", [string[]]$properties)

The filter syntax follows the SMO ClearAndInitialize documentation.

Commands to test

# Test with -Schema and -Table (should be fast now)
Measure-Command { Get-DbaDbTable -SqlInstance $server -Database "YourDB" -Schema "dbo" -Table "YourTable" -Verbose }

# Test with -Schema only
Measure-Command { Get-DbaDbTable -SqlInstance $server -Database "YourDB" -Schema "dbo" -Verbose }

# Test with -Table only
Measure-Command { Get-DbaDbTable -SqlInstance $server -Database "YourDB" -Table "YourTable" -Verbose }

# Test without filters (unchanged behavior - loads all tables)
Measure-Command { Get-DbaDbTable -SqlInstance $server -Database "YourDB" }

# Test with multiple schemas
Get-DbaDbTable -SqlInstance $server -Database "YourDB" -Schema "dbo","sys" -Verbose

Screenshots

SQL Server Profiler comparison:

Metric 2.7.6 (fast) 2.7.7 (slow) With Fix
Trace file size 88 KB 4,981 KB ~88 KB
Queries to sys.tables 1 1,150 1
Queries to sys.partitions 0 1,146 0
Execution time <1 sec ~30 sec <1 sec

Learning

…e parameters

The ClearAndInitialize optimization introduced in 2.7.7 loads ALL tables in the
database before filtering client-side. This causes severe performance issues
(30+ seconds vs <1 second) when only specific tables are requested.

This fix adds server-side filtering by passing an XPath-style URN filter to
ClearAndInitialize when -Schema or -Table parameters are specified.

Example filter: [@Schema='dispo' and @name='t_auftraege']

Performance improvement:
- Before: ~30 seconds (loads 1000+ tables)
- After: <1 second (loads only requested table)
@potatoqualitee
Copy link
Member Author

This somehow fixed the other branch. I don't know but I'll close it after merging his changes.

@PeterSawatzki-MGGM
Copy link

Thanks looking forward to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants