[Question] Feature flags overwritten when loading from multiple AppConfigurations via provider

[kimpenhaus]

---

When loading feature flags from multiple AppConfigurations using the provider, entries are overwritten in the ConfigurationManager. As a result, only the last ConfigMap entries remain, and previously defined feature flags are lost.

---

We have the following setup:
Using the provider, we generate ConfigMaps and Secrets from 4 different AppConfigurations.

When working with feature flags, we ran into an issue: if we load them into the ConfigurationManager via AddJsonFile, entries get "overwritten" — since they are stored by index in the ConfigurationManager underlying providers.

For example, if we have the following entries in two ConfigMaps:

"feature_management": {
    "feature_flags": [
        {
            "conditions": {
                "client_filters": []
            },
            "enabled": true,
            "id": "T1"
        },
        {
            "conditions": {
                "client_filters": []
            },
            "enabled": true,
            "id": "T2"
        }
    ]
}

"feature_management": {
    "feature_flags": [
        {
            "conditions": {
                "client_filters": []
            },
            "enabled": true,
            "id": "T3"
        }
    ]
}

The result in the feature manager will be only T3 and T2 — T1 is no longer present.

Our current workaround is to keep all feature flags in a single AppConfiguration.

I’m not sure if this could be called an issue and if so where it should be addressed, since from our perspective three components are involved:

• the appconfiguration kubernetes provider
• the feature manager
• the configuration manager

From what we can see, the problem already occurs in the ConfigurationManager, as it overwrites entries. We also already found this issue: dotnet/runtime#118204 but not sure if this addresses the same behavior.

Is there a possible solution for this in the provider?

[zhiyuanliang-ms]

Hey, @kimpenhaus

Thanks for reaching out! This is a pitfall of .NET configuration system.

I am assuming that:

1. You are using App Configuration k8s provider which will load kvs/ffs and store them in the config maps.
2. You have a .NET application where you have the similar code like below:

var builder = new ConfigurationBuilder();
builder.AddJsonFile("configmap1")
           .AddJsonFile("configmap2");

Each AddJsonFile call will register a ConfigurationSource.

3. You are using .NET Feature Management library to consume the feature flags.

In the latest version, .NET Feature Management library, we present a custom configuration merge behavior which will concatenate feature_flags array from different ConfigurationSource #536
You can follow this doc to enable it. I think it will fix your problem.

We have a full discussion about this configuration index merging issue in these two issues: #507 #550

[kimpenhaus]

hey @zhiyuanliang-ms,

thanks for your quick reply and the attached example.

Interestingly, when I noticed the issue for the first time, I had already seen the two tickets you mentioned and had accordingly configured them during registration:

        serviceCollection
            .Configure<ConfigurationFeatureDefinitionProviderOptions>(
                o => o.CustomConfigurationMergingEnabled = true)
            .AddFeatureManagement();

When running the code I had still the wrong behavior - that's why we went with the workaround to just use a single appconfiguration for the feature flags.

However, when I took another look at your sample yet, I noticed that we are using ConfigurationManager instead of ConfigurationBuilder.

Do you know if there is different behavior in that case?

I switched to ConfigurationBuilder on our end, and that seemed to work with CustomConfigurationMergingEnabled. I then wanted to run a counter-test and switched back to ConfigurationManager — after that, it also produced the correct result. Now I’m a bit confused…

From your perspective, could I also use ConfigurationManager to achieve the same behavior?

Thanks and best regards,
M.

[zhiyuanliang-ms]

Hey, @kimpenhaus

I added a testcase for ConfigurationManager in this PR

I didn't find any problem of merging feature flags from different configuration sources under ConfigurationManager.

I then wanted to run a counter-test and switched back to ConfigurationManager — after that, it also produced the correct result.

This sounds weird. I have no idea about it. But if you cannot reproduce it, I will just let it go.

From your perspective, could I also use ConfigurationManager to achieve the same behavior?

Yes. We depends on IConfigurationRoot.Providers (code) and the ConfigurationManager implements IConfigurationRoot interface.

However, when I took another look at your sample yet, I noticed that we are using ConfigurationManager instead of ConfigurationBuilder.

Do you know if there is different behavior in that case?

ConfigurationManager is more modern than ConfigurationBuilder.

The biggest difference between them is that ConfigurationManager will build configuration source immediately when adding the source. For more details, please check ref1 ref2 ref3 The .Build() call of ConfigurationManager is a no-op.

However, the classic ConfigurationBuilder only build/load data from configuration source when .Build() is called.

[zhiyuanliang-ms]

cc @jimmyca15

[kimpenhaus]

@zhiyuanliang-ms thanks again for the detailed explanations - we will go for this. thanks!