Recursive fetching for Google Groups causes timeouts ( >1m auth req times)

[codeknight03]

Preflight Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.
• I am not looking for support or already pursued the available support channels without success.

Version

2.41.1

Storage Type

Kubernetes

Installation Type

Official Helm chart

Expected Behavior

This recursive function to fetch google groups makes requests for users with more than 100 groups take longer than standard timeout bounds ( > 30s)

For instance here
argoproj/argo-cd#21420 (comment)

Expected is to be able to not take this long in returning with the groups by either providing a parameter for depth to which groups should be fetched or not to give sub groups in the default response.

Actual Behavior

The recursive fetching takes longer and causes timeouts.

Steps To Reproduce

To Reproduce

• Ensure user has membership in more than 100 Google Workspace groups. Based on observations, users with fewer than 90 groups successfully complete the authentication flow, while those with more than 110 groups fail.

• Ensure Dex is enabled with the Google connector and configured to fetch transitive group memberships as instructed in the Argo CD documentation.

• Attempt to log in to the Argo CD dashboard using Google as the authentication provider.

Though this the core way to reproduce but as pointed out the recursive code would exhibit this behavior for any client be it Argo or not.

Additional Information

No response

Configuration

Logs

[codeknight03]

I have filed this issue in intention to start writing a DEP for it. I am looking to see if this is an issue the community would like to be addressed then I can start a DEP for this.

[nabokihms]

Hello, @codeknight03. Sorry for the long delay.|

I believe we do not need a DEP to fix the problem. It would be enough just to discuss the solution briefly in this issue.

[codeknight03]

Hi, Apologies on my end for the delay. But I am actively looking into options. Should have something solid within the coming week.

[codeknight03]

@nabokihms ,
After looking for ways where I could stop any major disruptions, I have come to the conclusion that there aren't going to be any free lunches on this thing.

Overall, there are two ways to go about it :

1. Do the transitive fetches in parallel,
   • A thread safe map for checkedGroups instead of the current map[string]struct{}.
   • A thread safe implementation of the userGroups list, so appends are safe.

Pseudo Code:

// As we are mostly setting one time and reading many times, I think sync.Map is better than handling locking. 
var checkedGroups sync.Map

// mutex to be passed to getGroups to make userGroups list safe. 
mu sync.Mutex

func getGroups (mu, checkedGroups) {
  userGroups []string

  if FetchTransitiveGroupMembership { 
      go getGroups()
  }
}

2. Allow the users to configure the depth, they want. Stop the recursion at the given depth Something like,

type Config struct {

FetchTransitiveGroupMembership bool `json:"fetchTransitiveGroupMembership"`

//Optional
//if FetchTransitiveGroupMembership  is true, set depth for transitive group fetch.
FetchTransitiveGroupsDepth int `json: "fetchTransitiveGroupDepth`

}

[codeknight03]

I am inclined to go with 1 as there is no user apparent change and it will be a true performance gain.

2 feels more like moving the problem to the user for him to solve.

I would start the implementation as soon as a member can weigh in.

[codeknight03]

Hey @nabokihms , Do you think it's okay to start implementation for the concurrent fetching ?

[bzurkowski]

Any update on this one?

[codeknight03]

I'd be happy to raise a PR with transitive fetches in parallel but I am not sure if that is what @nabokihms has in mind.

[pgmgb]

We would also be intrested in an update here

[codrutpanea]

@nabokihms can you add some input here? We are waiting 10-15 seconds for a login with dex when using nested groups from google. Fetching the groups in parallel should improve this time

[nabokihms]

If there is an easy algorithmic optimization, like querying groups in parallel, I would be happy to accept it. I have no environment to test it myself, though.

[codrutpanea]

I'd be happy to test this in my environment. @codeknight03 are you still up for a PR with your proposed improvement?

[codeknight03]

Absolutely. I must have an old branch ready with code but will review , zip it up and raise.

[codrutpanea]

i implemented the suggested pseudo code from #3929 (comment) and the loading times went form 10-15 seconds down to 2-3 seconds. I tested the changes with 20 nested groups. So just fetching the groups in parallel makes a huge difference

[codeknight03]

Hey @codrutpanea, I’m going to be unavailable for another week, so I won’t be able to raise the PR in that time. If you’d like to go ahead and open a PR to get this moving sooner, that would be great. Otherwise, I can take it up once I’m back.