Feature/azdo by SebastianClaesson · Pull Request #1371 · maester365/maester

SebastianClaesson · 2025-12-11T08:06:32Z

Description

Fixes #1368

Adding 31 Azure DevOps security tests.
Each of the tests is added to the new folder azdo in maester\public\maester

All of the tests have a markdown file, with :

Rationale
Remediation action
Related links (Official links / blog posts describing the security issue)

Contribution Checklist

Before submitting this PR, please confirm you have completed the following:

📖 Read the guidelines for contributing to this repository.
🧪 Ensure the build and unit tests pass by running /powershell/tests/pester.ps1 on your local system.

…aester into feature/azdo

SamErde · 2026-01-21T11:23:06Z

tests/Maester/AzureDevOps/Test-AzdoAllowRequestAccessToken.md

+3. Select Policies, locate the Request Access policy and toggle it to off.
+4. Provide the URL to your internal process for gaining access. Users see this URL in the error report when they try to access the organization or a project within the organization that they don't have permission to access.
+
+**Results:**


Shouldn't the detailed 401 go to users in the organization and the 404 go to users not in the organization?

I do agree!
Seems the experience has changed since;

MicrosoftDocs/azure-devops-docs@91c4410#diff-a28e8dd823f1493651d0c6322e6b2dd976bccab79e279de5aea876ce20272729R44

I'll update with the new information the article.

powershell/public/maester/azdo/Test-AzdoAllowRequestAccessToken.md

powershell/public/maester/azdo/Test-AzdoAllowTeamAdminsInvitationsAccessToken.md

Copilot

Pull request overview

This pull request adds 31 Azure DevOps security tests to the Maester project, providing comprehensive security assessments for Azure DevOps organizations. Each test includes both PowerShell implementation and markdown documentation describing the security rationale and remediation steps.

Changes:

Added 31 new Azure DevOps security test functions covering authentication, access control, pipeline security, and resource management
Created comprehensive markdown documentation for each test with remediation guidance
Updated the module manifest to export all new test functions
Added a test runner file that orchestrates all Azure DevOps security tests

Reviewed changes

Copilot reviewed 65 out of 65 changed files in this pull request and generated 28 comments.

Show a summary per file

File	Description
tests/Maester/Azdo/Test-Azdo.Tests.ps1	Test runner that executes all 31 Azure DevOps security tests
tests/Maester/Azdo/README.md	Overview documentation for Azure DevOps tests
powershell/public/maester/azdo/*.ps1	31 PowerShell functions implementing security checks
powershell/public/maester/azdo/*.md	31 markdown documentation files with rationale and remediation steps
powershell/Maester.psd1	Module manifest updated to export new functions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

...ll/public/maester/azdo/Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipeline.ps1

tests/Maester/Azdo/Test-Azdo.Tests.ps1

powershell/public/maester/azdo/Test-AzdoOrganizationStageChooser.md

powershell/public/maester/azdo/Test-AzdoSSHAuthentication.md

Corrected typos and formatting in the documentation.

Removed unnecessary commas and spaces. Added line breaks for MD linting.

Corrected grammatical errors and added punctuation for clarity.

Updated description to clarify the function's purpose and improve readability.

Corrected grammatical errors and improved clarity in remediation instructions.

Clarified the purpose of the Azure DevOps tests and updated the reference link.

tests/Maester/Azdo/Test-Azdo.Tests.ps1

…thorizationScopeNonReleasePipeline.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

powershell/public/maester/azuredevops/Test-AzdoFeedbackCollection.ps1

powershell/public/maester/azuredevops/Test-AzdoLogAuditEvent.ps1

...c/maester/azuredevops/Test-AzdoOrganizationAutomaticEnrollmentAdvancedSecurityNewProject.ps1

powershell/public/maester/azuredevops/Test-AzdoOrganizationBadgesArePrivate.ps1

powershell/public/maester/azuredevops/Test-AzdoOrganizationCreationClassicBuildPipeline.ps1

powershell/public/maester/azuredevops/Test-AzdoPublicProject.ps1

powershell/public/maester/azuredevops/Test-AzdoResourceUsageProject.ps1

powershell/public/maester/azuredevops/Test-AzdoResourceUsageWorkItemTag.ps1

powershell/public/maester/azuredevops/Test-AzdoSSHAuthentication.ps1

powershell/public/maester/azuredevops/Test-AzdoThirdPartyAccessViaOauth.ps1

SamErde · 2026-02-18T17:48:41Z

Sorry for all of the comments! This is awesome work!

…aester into feature/azdo

SebastianClaesson · 2026-02-19T07:48:39Z

@SebastianClaesson this is top notch work! Love it.

I would suggest two additions.

Add doc - Update the installation doc to include the optinal module to install and the command for connecting website/docs/installation.md . See https://maester.dev/docs/installation#optional-modules-and-permissions

Add a blog post that outlines that calls out the new tests to website/blog See https://maester.dev/blog

Thank you @merill , I'll add the docs and a blog post to showcase / outline the new tests!

Copilot

Pull request overview

Copilot reviewed 68 out of 68 changed files in this pull request and generated 12 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-19T17:38:06Z

tests/Maester/AzureDevOps/Test-AzdoProjectCollectionAdministrator.ps1

+        if ($Member.subjectKind -eq 'group') {
+            Write-Verbose "Finding members in group '$($Member.DisplayName)' - Descriptor '$($_.Descriptor)'"
+            Get-ADOPSMembership -Descriptor $Member.descriptor -Direction 'down' | Foreach-object {
+                Write-Verbose "Processing member '$($_.DisplayName)' - Descriptor '$($_.Descriptor)'"


Inside Get-NestedAdoMembership, the first Write-Verbose message references $($_.Descriptor) even though $_ isn’t set in that scope; this will log an empty/incorrect descriptor. Use $Member.Descriptor there to match the rest of the function.

powershell/public/maester/azuredevops/Test-AzdoOrganizationTaskRestrictionsDisableNode6Task.ps1

tests/maester-config.json

Copilot · 2026-02-19T17:38:07Z

tests/Maester/AzureDevOps/Test-AzdoEnforceAADConditionalAccess.ps1

+    Checks the status of when you sign in to the web portal of a Microsoft Entra ID-backed organization,
+    Microsoft Entra ID always performs validation for any Conditional Access Policies (CAPs) set by tenant administrators.
+
+    https://learn.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops


The comment header link in the description points to audit streaming (.../audit/auditing-streaming...) which doesn’t match this test’s purpose (Conditional Access enforcement). Updating it to the Conditional Access policy documentation would avoid confusing users reading the help.

Suggested change

https://learn.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops

https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/manage-conditional-access?view=azure-devops&tabs=preview-page

tests/Maester/AzureDevOps/Test-Azdo.Tests.ps1

Copilot · 2026-02-19T17:38:08Z

tests/Maester/AzureDevOps/Test-AzdoOrganizationTriggerPullRequestGitHubRepository.ps1

+        $data = @'
+            Prevent pipelines from making secrets available to fork builds is set to '{0}'\
+            Prevent pipelines from making fork builds have the same permissions as regular builds is set to '{1}'\
+            Require a team member's comment before building a pull request is set to '{2}' ({3})
+'@ -f $settings.enforceNoAccessToSecretsFromForks, $settings.enforceJobAuthScopeForForks, $settings.isCommentRequiredForPullRequest, $AdditionalInfo


The here-string includes trailing backslashes at the end of lines, which will be emitted literally in the result output (they don’t act as line continuations inside a here-string). If the intent is multiline formatting, remove the trailing \ characters and rely on the here-string newlines (or build the string with explicit newlines).

powershell/public/maester/azuredevops/Test-AzdoAllowRequestAccessToken.ps1

powershell/public/maester/azuredevops/Test-AzdoThirdPartyAccessViaOauth.ps1

powershell/public/maester/azuredevops/Test-AzdoSSHAuthentication.ps1

powershell/public/maester/azuredevops/Test-AzdoExternalGuestAccess.ps1

SamErde · 2026-02-24T11:09:35Z

@copilot open a new pull request to apply changes based on the comments in this thread

powershell/public/maester/azuredevops/Test-AzdoRestrictPersonalAccessTokenLifespan.ps1

powershell/public/maester/azuredevops/Test-AzdoValidateSshKeyExpiration.ps1

Corrected punctuation and improved clarity in the installation documentation.

…aester into feature/azdo

…verted logic as co-pilot do not like it.

…aester into feature/azdo

SebastianClaesson and others added 9 commits May 11, 2025 23:22

adding azdo test suit and preview module

de3bb1d

Adding readme

90404c8

adding updated readme

8d9f3c2

Adding Azdo tests

aa8b936

Updating readme and setting functions in the correct folder

8d7318a

Merge branch 'maester365:main' into feature/azdo

94d7523

Relocating parent folder

fc83a14

Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…

13c55d8

…aester into feature/azdo

updating readmes and powershell scripts

86125a8

SebastianClaesson requested review from a team as code owners December 11, 2025 08:06

SamErde requested a review from Copilot January 21, 2026 11:19

Copilot started reviewing on behalf of SamErde January 21, 2026 11:20 View session

SamErde added enhancement New feature or request maester-test Related to a Maester test labels Jan 21, 2026

SamErde reviewed Jan 21, 2026

View reviewed changes

powershell/public/maester/azdo/Test-AzdoAllowRequestAccessToken.md Outdated Show resolved Hide resolved

SamErde reviewed Jan 21, 2026

View reviewed changes

powershell/public/maester/azdo/Test-AzdoAllowTeamAdminsInvitationsAccessToken.md Outdated Show resolved Hide resolved

SamErde reviewed Jan 21, 2026

View reviewed changes

powershell/public/maester/azdo/Test-AzdoAllowTeamAdminsInvitationsAccessToken.md Outdated Show resolved Hide resolved

Copilot AI reviewed Jan 21, 2026

View reviewed changes

SamErde added 7 commits January 21, 2026 06:29

Fix typos and improve formatting in documentation

0ba07d8

Corrected typos and formatting in the documentation.

Refine grammar in Test-AzdoArtifactsExternalPackageProtectionToken.md

c12ab8b

Removed unnecessary commas and spaces. Added line breaks for MD linting.

Fix typo in description of security policy check

bdfa2ff

Fix grammar and punctuation in Test-AzdoAuditStream.md

f4f68c7

Corrected grammatical errors and added punctuation for clarity.

Refine description for Test-AzdoAuditStream function

ca1d7da

Updated description to clarify the function's purpose and improve readability.

Fix grammar and clarity in Test-AzdoEnforceAADConditionalAccess.md

6005abd

Corrected grammatical errors and improved clarity in remediation instructions.

Update README.md with clearer test descriptions

0c4b342

Clarified the purpose of the Azure DevOps tests and updated the reference link.

SamErde reviewed Jan 21, 2026

View reviewed changes

tests/Maester/Azdo/Test-Azdo.Tests.ps1 Outdated Show resolved Hide resolved

SamErde reviewed Jan 21, 2026

View reviewed changes

tests/Maester/Azdo/Test-Azdo.Tests.ps1 Outdated Show resolved Hide resolved

Update powershell/public/maester/azdo/Test-AzdoOrganizationLimitJobAu…

25dcaff

…thorizationScopeNonReleasePipeline.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

SamErde requested changes Feb 18, 2026

View reviewed changes

SebastianClaesson added 2 commits February 19, 2026 08:39

Added severity configuration to maester-config instead

9667b94

Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…

4bff7a1

…aester into feature/azdo

SebastianClaesson added 6 commits February 19, 2026 13:16

Fixed connection check, Fixed typos, Fixed grammar issues

0fdb4f6

Fixed more grammar issues

08f7481

Changing title to Maester contributor

d1c6100

Adding rowbreak in between to make the message look nicer

4e3521c

Adding whitespace

19a7c57

Fixed grammar and new instructions for Azure DevOps suite

7c484f8

SebastianClaesson requested a review from a team as a code owner February 19, 2026 14:18

SamErde requested a review from Copilot February 19, 2026 17:29

Copilot started reviewing on behalf of SamErde February 19, 2026 17:29 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

SebastianClaesson added 2 commits February 25, 2026 14:29

Changing severity and adding new tests

e30c0e5

Adding more tests.

a724202

SamErde reviewed Feb 25, 2026

View reviewed changes

powershell/public/maester/azuredevops/Test-AzdoRestrictPersonalAccessTokenLifespan.ps1 Outdated Show resolved Hide resolved

SamErde reviewed Feb 25, 2026

View reviewed changes

powershell/public/maester/azuredevops/Test-AzdoValidateSshKeyExpiration.ps1 Outdated Show resolved Hide resolved

SebastianClaesson and others added 10 commits February 25, 2026 15:23

Adding WIP

6142352

Fix punctuation and wording in installation.md

9b7c637

Corrected punctuation and improved clarity in the installation documentation.

Adding blogpost and proof read content

0d0b169

Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…

c109973

…aester into feature/azdo

Adding another iteration

edca956

Reverting maester-config

3a6ecdd

Adding azdo.1036

5f499e2

Fix typo in maester-config.json title

4138d8f

Adding updated files, with grammar checks and spellchecks. Removed in…

70946d7

…verted logic as co-pilot do not like it.

Merge branch 'feature/azdo' of https://github.com/SebastianClaesson/m…

e46b514

…aester into feature/azdo

	https://learn.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops
	https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/manage-conditional-access?view=azure-devops&tabs=preview-page

Conversation

SebastianClaesson commented Dec 11, 2025

Description

Contribution Checklist

Uh oh!

SamErde Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

SebastianClaesson Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SamErde commented Feb 18, 2026

Uh oh!

SebastianClaesson commented Feb 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SamErde commented Feb 24, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants