New failover implementation

[pbrezina]

This pull request is intended to be a start of a "failover" feature branch where other developers will be able to contribute.

The main failover logic works, compiles and can be tested using a "minimal" provider that is included as an example. The purpose of the "minimal" provider is only to test the failover without the need to port full provider code and itwill be removed prior pushing the contents to the master branch. See how to set it up in minimal-provider-notes.txt and see the switch to new failover in commit minimal: switch to new failover for service lookup and user authentication - this is the minimal set of changes to get it working, but the real port should get and will require more refactoring.

The work is still not finished and there is missing functionality. This functionality, however, can be implemented in small areas of code and should not require larger changes or glues in the whole code base, so this is ready for review. Remaining work is tracked at [1]. Feel free to take any of these tickets and open new tickets when you find something missing.

When reviewing, you can start with src/providers/failover/readme.md that provides high level documentation of the code. And of course do not forget the design page [2].

• [1] https://redhat.atlassian.net/browse/IDM-647
• [2] https://docs.google.com/document/d/1LfGsum35aj4CdFfbN3uLLRWbSin247ct-rlkiI4eMLw/edit?tab=t.0

Thanks, Pavel

[gemini-code-assist]

Code Review

This pull request implements a new failover mechanism for SSSD, introducing prioritized server groups, parallelized candidate server discovery, and a transaction-based API for automated retries. It also provides a minimal provider implementation to demonstrate the new architecture. Critical logic bugs were identified in the server group resolution logic, where duplicate detection causes premature loop exit, and in the address change detection function, which currently returns inverted results.

[gemini-code-assist]

There's a logic error in the loop that merges discovered servers and removes duplicates. The break statement on line 349 will cause the loop to terminate prematurely if a duplicate server is found, preventing any subsequent discovered servers from being added to the list.

This should be a continue, but simply changing it would also be incorrect due to the i++ in the loop's post-increment step, which would create a hole in the out array.

The loop should be refactored to correctly handle duplicates without terminating early.

    for (j = 0; state->group->discovered_servers[j] != NULL; j++) {
        found = false;
        current = state->group->discovered_servers[j];
        for (k = 0; out[k] != NULL; k++) {
            if (sss_failover_server_equal(out[k], current)) {
                found = true;
                break;
            }
        }

        if (found) {
            continue;
        }

        out[i] = talloc_reference(out, current);
        if (out[i] == NULL) {
            DEBUG(SSSDBG_CRIT_FAILURE, "Out of memory!\n");
            talloc_free(out);
            return ENOMEM;
        }
        i++;
    }

[gemini-code-assist]

The logic in sss_failover_server_resolve_address_changed appears to be inverted. The function name suggests it should return true if the address has changed, but the implementation memcmp(...) == 0 returns true if the addresses are the same. This will cause incorrect behavior where address changes are not detected.

    return memcmp(server->addr->binary, hostent->addr_list[0]->ipaddr,
                  server->addr->binary_len) != 0;