Dynamic configuration updates in .NET using Parameter Store and Secrets Manager

Loading configurations and secrets in .NET applications is a common practice. However, it comes with challenges in storing and accessing them securely and dynamically, without the need for application restart.

AWS Systems Manager Parameter Store provides a centralized solution for storing and managing configurations and secrets data. This blog post explores an advanced approach to managing these in .NET applications with a focus on:

Referencing AWS Secrets Manager secrets through Parameter Store
Loading configurations from both Parameter Store and Secrets Manager
Implementing automatic configuration reloading without application restarts

Solution Overview

You can retrieve secrets stored in Secrets Manager directly from Parameter Store using the following format:

/aws/reference/secretsmanager/<secret-path>

By following this format, Systems Manager understands that it needs to fetch the secret from the Secrets Manager instead of the Parameter Store.

For example, to reference a secret named dev/DbPassword, you would use:

/aws/reference/secretsmanager/dev/DbPassword

See Referencing AWS Secrets Manager secrets from Parameter Store parameters for more information.

This solution uses Parameter Store and Secrets Manager, along with .NET configuration and IOptionsMonitor pattern, to address the following common configuration challenges:

Unified Access: Use Parameter Store as a single-entry point for both configurations and Secrets Manager secrets.

Security: Sensitive data is stored in the Secrets Manager and referenced in the Parameter Store, ensuring proper encryption and access control.

Dynamic Updates: Implement automatic reloading of configuration values without application restarts.

Environment Segregation: Utilize hierarchical structure in Parameter Store to separate configurations by environment

Prerequisites

AWS Account with required privileges
.NET 8 SDK

Architecture

Consider an application running on EC2 instance that requires access to both configuration parameters and sensitive secrets during its operation. The application assumes an IAM role with the necessary permissions and retrieves values from Parameter Store for both configuration data and secrets.

When the application requests a configuration parameter, Parameter Store returns the value directly.
If the parameter is a reference to a secret stored in Secrets Manager, Parameter Store fetches the secret from Secrets Manager and returns it to the application.

Diagram of solution architecture showing EC2, Parameter Store, and Secrets Manager

Figure 1: Solution Architecture

Walkthrough

Let’s explore how to fetch configuration values from AWS services in the above application. The application needs to retrieve two types of data:

System parameters like the Database Hostname and API Endpoint from Parameter Store
Sensitive information such as the Database Password and API Key from Secrets Manager.

In this walkthrough, you will build a .NET application that securely accesses configuration values from both AWS services.

Step 1: Configure parameters and secrets

1. Store parameters in AWS Systems Manager

Set up parameters in AWS Systems Manager using the following hierarchical structure:

Development environment	Production environment
/dev/DbHostname	/prod/DbHostname
/dev/ApiEndpoint	/prod/ApiEndpoint

2. Store secrets in AWS Secrets Manager

Set up secrets in AWS Secrets Manager using the following hierarchical structure:

Development environment	Production environment
/dev/DbPassword	/prod/DbPassword
/dev/ApiKey	/prod/ApiKey

3. Create IAM Role to access configuration parameters and secrets

To access the parameters and secrets, you need to create an IAM Role with the following permissions. This role will grant the necessary permissions to access the required resources.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "VisualEditor0",
			"Effect": "Allow",
			"Action": [
				"ssm:GetParameter",
				"ssm:GetParameters",
				"ssm:GetParametersByPath"
			],
			"Resource": [
				"arn:aws:ssm:${Region}:${AccountId}:parameter/*"
			]
		},
		{
			"Sid": "VisualEditor1",
			"Effect": "Allow",
			"Action": "secretsmanager:GetSecretValue",
			"Resource": [
				"arn:aws:secretsmanager:${Region}:${AccountId}:secret:*"
			]
		}
	]
}

Step 2: .NET Application Setup

1. AWS Services Extension Configuration

Create two extension methods to configure AWS services as follows.

The following extension method adds the application configuration from Parameter Store based on the path provided.

public static void AddParameterStoreConfiguration(this WebApplicationBuilder builder)
{
    var configuration = builder.Configuration;

    // Retrieve current environment (e.g., /dev, /prod)
    var appSettingsPath = configuration["Environment"];

    configuration.AddSystemsManager(options =>
    {
        options.Path = appSettingsPath;
        options.ReloadAfter = TimeSpan.FromMinutes(5); // Reloads for every 5 minutes
    });
}

The following extension method adds Secrets Manager configuration using System Manager reference path.

public static void AddSecretsManagerConfiguration(this WebApplicationBuilder builder)
{
    var configuration = builder.Configuration;

    // Retrieve current environment (e.g., /dev, /prod)
    var environmentPath = configuration["Environment"];

    var secretPath = $"/aws/reference/secretsmanager/{environmentPath}/<SecretName>";
    configuration.AddSystemsManager(options =>
    {
        options.Path = secretPath;
        options.ReloadAfter = TimeSpan.FromMinutes(5);
        options.Optional = true;
    });
}

2. AppSettings (Optional)

For local testing purposes, please configure your appSettings.json file as below.

"AppConfig": {
  "DbHostname": "",
  "ApiEndpoint": ""
},
"Secrets": {
  "DbPassword": "",
  "ApiKey": ""
}

3. Configuration Model

A model is used for the binding of configuration values from the Secrets Manager and Parameter Store. Add the following two classes to your project:

public class SecretSettings
{
    public string DbPassword { get; set; }
    public string ApiKey { get; set; }
}

 public class AppConfig
{
    public string DbHostname { get; set; }
    public string ApiEndpoint { get; set; }
}

4. Startup configuration

// Add AWS Configuration.
builder.AddParameterStoreConfiguration();
builder.AddSecretsManagerConfiguration();

builder.Services.Configure<AppConfig>(builder.Configuration.GetSection("AppConfig"));
builder.Services.PostConfigure<SecretSettings>(options =>
{
    options.ApiKey = builder.Configuration["ApiKey"];
    options.DbPassword = builder.Configuration["DbPassword"];
});

5. Dynamic configuration usage

Use IOptionsMonitor<T> to access the latest configuration values and react to changes. This offers flexibility to refresh the secrets without re-deploying or re-starting the application.

public class MyService
{
    private readonly IOptionsMonitor<AppConfig> _appConfig;
    private readonly IOptionsMonitor<SecretSettings> _secretSettings;

    public MyService(
       IOptionsMonitor<AppConfig> appConfig,
       IOptionsMonitor<SecretSettings> secretSettings)
    {
        _appConfig = appConfig;
         _secretSettings = secretSettings;
    }

    public void DoSomething()
    {
        // Always get the latest configuration
        var config = _appConfig.CurrentValue;
        var apiEndpoint = config.ApiEndpoint; //Access parameter store value.
        var dbPassword = _secretSettings.CurrentValue.DbPassword; //Access secret value.
    }
}

How It Works

Here is how the application gets the values from Secrets Manager and Parameter Store:

When the application starts, it loads configurations from Parameter Store for the specified environment.
The application loads configurations and secrets (using reference path) directly from Parameter Store.
The AWS SDK for .NET resolves these references automatically, fetching the actual secret value from the Secrets Manager.
The configuration is reloaded every 5 minutes (as specified in ReloadAfter).
IOptionsMonitor<T> ensures that your application always uses the most up-to-date configuration values.
Any changes to parameters in Parameter Store or secrets in Secrets Manager are reflected in your application within 5 minutes, without restart.

Best Practices and Considerations

Follow these best practices for managing your configuration in .NET:

Environment Segregation: Use separate paths for different environments to maintain clear separation.
Least Privilege: Ensure your IAM roles have the minimum necessary permissions to access Parameter Store and Secrets Manager.
Encryption: Use AWS Key Management Service (AWS KMS) keys to encrypt sensitive data.
Error Handling: Add robust error handling for scenarios where configuration loading fails.
Caching: Consider implementing a caching layer for frequently accessed rarely changing values to reduce API calls.
Audit Trail: Enable AWS CloudTrail to maintain an audit log of parameter and secret access.

Cleanup

To avoid ongoing charges and maintain a clean AWS environment, follow these steps to delete the resources created:

1. Delete the Parameter Store parameters:

Open the AWS Systems Manager console.
In the navigation panel, choose Parameter Store.
On the My parameters tab, select the check box next to each parameter to delete.
Choose Delete.
On the confirmation dialog, choose Delete parameters.

2. Delete the Secrets Manager secrets:

Open the AWS Secrets Manager console
In the list of secrets, choose the secret you want to delete.
In the Secret details section, choose Actions, and then choose Delete secret.
In the Disable secret and schedule deletion dialog box, in Waiting period, enter the number of days to wait before the deletion becomes permanent. The Secrets Manager attaches a field called DeletionDate and sets the field to the current date and time, plus the number of days specified for the recovery window.
Choose Schedule deletion.

Conclusion

This approach provides a powerful, secure, and flexible way to manage configurations in .NET applications using AWS services. By leveraging Parameter Store and Secrets Manager together, you can centralize configuration management, securely handle sensitive information, dynamically update configurations without application restarts, and maintain clear separation between environments. This integration offers a robust solution for managing application settings and secrets in a cloud-native context, enhancing both security and operational efficiency. To learn more, refer to Referencing AWS Secrets Manager secrets from Parameter Store parameters in the Systems Manager User Guide.

.NET on AWS Blog