Integrating Custom Keycloak Protocol Mappers with ASP.NET Core Minimal APIs

June 20, 2024 by Binny Kanjur

---

Keycloak is an open-source identity and access management solution designed to secure applications and services with minimal effort. It offers features like single sign-on, identity brokering, and user federation, making it a powerful tool for managing authentication and authorization.

In many scenarios, it is necessary to add additional claims to the tokens issued by Keycloak. Claims are pieces of information about a user, such as their tenant, role or permissions. While Keycloak allows adding static claims, there are cases where these claims need to be dynamically retrieved from an external API. By doing so, the token contains the most up-to-date information, ensuring effective enforcement of access policies and seamless integration with external systems.

Protocol mappers in Keycloak are used to map user attributes and roles to claims in tokens. They play a crucial role in customizing the information included in tokens. In this article, we will develop a custom RESTful protocol mapper that fetches claims from a configured REST endpoint, allowing dynamic claim retrieval.

In this article, you will:

• Build a REST endpoint using ASP.NET Core minimal API that will return a collection of customized claims which would be added to the Keycloak token.
• Implement a Custom Protocol Mapper & deploy it to Keycloak.
• Dockerize both ASP.NET Core app & Keycloak; orchestrated by Docker Compose.

TL;DR

If you're short on time or just want to get your hands dirty quickly, follow these steps to run the complete solution from my GitHub repository. This setup includes Keycloak and an ASP.NET Core Minimal API, all orchestrated via Docker Compose. No extra configuration is needed—just Docker installed on your machine.

Prerequisites

• Docker Community Edition.

1. Clone the GitHub repository:

If you don't have Git installed, download the source zip here.

git clone https://github.com/binnykanjur/keycloak-protocol-mapper-sample.git
cd keycloak-rest-protocol-mapper-sample

2. Start the services with Docker Compose:

docker-compose up -d

3. Login to the Keycloak admin console:

• Open your browser and go to http://localhost:5000.
• Login with the credentials:
  • Username: admin
  • Password: admin

4. Explore the custom protocol mapper:

You will use the Keycloak admin console to navigate to the configured client and check the claims included in the access token. A visual guide is provided below, navigate through the images to see each step.

For detailed instructions and more information, continue reading the full article below.

Setting Up the Development Environment

Install the following prerequisites:

• .NET 8+ SDK.
• Docker Community Edition.
• Java Development Kit (JDK).
• Gradle.
• A working folder for ASP.NET Core Minimal API, Custom protocol mapper & the docker-compose.yml. In this article, the name keycloak-protocol-mapper-sample is used as the working folder.

---

On Windows 10 or above, you can use Windows Package Manager to install the prerequisites:

1. Install PowerShell, Docker Desktop, .NET 8 & OpenJDK 21:
   • Download the Winget Configuration File.
   • Navigate to the downloaded folder & execute:

winget configure -f configuration.dsc.yaml

2. Download and Configure Gradle:
   • Download the PowerShell script file.
   • Open PowerShell in the Administrator mode.
   • Navigate to the downloaded folder & execute:

.\setup-gradle.ps1

The PowerShell script downloads the Gradle archive, extracts it to c:\Gradle & sets the required environment variables.

Creating the ASP.NET Core Minimal API

In this section, you will build an ASP.NET Core Minimal API with a GET endpoint /api/v1/claims that receives an objectId query parameter and returns custom claims. The endpoint is simple yet demonstrative of the process required to integrate with Keycloak. In production, these claims could be retrieved from a database or another external source.

1. Setting Up the Project

Open your terminal, create a working folder (keycloak-protocol-mapper-sample) if you haven't already, and enter it. In the working folder, run the following command to create a new Minimal API project & navigate into its directory.

dotnet new web -n ClaimsProvider.Api
cd ClaimsProvider.Api

2. Defining the Claims Endpoint

Open Program.cs & replace the file with the following code that defines a new GET endpoint /api/v1/claims & accepts an objectId query parameter:

using System.Text.Json.Serialization;

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.MapGet("/api/v1/claims", (string objectId) => {
    return Results.Ok(
        new ClaimsResponse {
            UserId = objectId,
            Role = $"Role_{Random.Shared.Next(1, 100)}"
        });
});

app.Run();

internal class ClaimsResponse {
    [JsonPropertyName("__user__")]
    public required string UserId { get; set; }
    public required string Role { get; set; }
}

3. Testing the Endpoint

Run the application

dotnet run --urls=http://localhost:8080/

Use a browser to send a GET request to http://localhost:8080/api/v1/claims?objectId=123. You should see a JSON response like below.

{
    "__user__": "123",
    "role": "Role_25"
}

4. Containerizing the API

Create a file named Dockerfile in the directory containing the .csproj and open it in a text editor.

Here's the code snippet to add to your Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
USER app
WORKDIR /app
EXPOSE 8080
EXPOSE 8081

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["ClaimsProvider.Api.csproj", "."]
RUN dotnet restore "./ClaimsProvider.Api.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "./ClaimsProvider.Api.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./ClaimsProvider.Api.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ClaimsProvider.Api.dll"]

With the ClaimsProvider.Api set up and the /api/v1/claims endpoint ready, we have laid the groundwork for integrating custom claims into Keycloak tokens. The next steps will involve creating the custom protocol mapper in Keycloak and configuring it to use this endpoint.

Implementing the Custom Protocol Mapper in Keycloak

In this section, you will implement a custom protocol mapper and configure it to call our ClaimsProvider.Api to retrieve additional claims.

1. Setting Up the Project

• Begin by creating a new directory named restful-claims-provider in the working directory & enter into it.
• Initialize the project by executing gradle init with parameters to generate a Java library.
• Delete the files not required for this article.

mkdir restful-claims-provider; cd restful-claims-provider

gradle init --type java-library --dsl kotlin --test-framework junit --package "com.binnykanjur.keycloak.protocolmapper" --project-name restful-claims-provider --no-split-project --no-incubating --use-defaults

Remove-Item -Path "lib\src\test" -Recurse -Force; Remove-Item -Path "lib\src\main\java\com\binnykanjur\keycloak\protocolmapper\Library.java"

2. Configuring Gradle for the Protocol Mapper

Next, lets update the build.gradle.kts file include the necessary dependencies & other configurations.

Key Points from build.gradle.kts

1. Java Compatibility:
   • sourceCompatibility: JavaVersion.VERSION_11
   • targetCompatibility: JavaVersion.VERSION_11
2. Repositories:
   • mavenCentral(): Specifies Maven Central as the repository for dependencies.
3. Dependencies:
   • org.json:json:20240303: For parsing JSON returned from the /api/v1/claims endpoint.
   • Keycloak dependencies for core functionality:
     • org.keycloak:keycloak-core:25.0.0
     • org.keycloak:keycloak-services:25.0.0
     • org.keycloak:keycloak-server-spi:25.0.0
     • org.keycloak:keycloak-server-spi-private:25.0.0
4. Tasks:
   • copyToLib: Copies the org.json runtime dependency to the keycloak/providers directory within the working directory.
   • jar: Configures the JAR task to depend on copyToLib, sets destination directory, and names the archive file.

Here's the lib\build.gradle.kts file:

plugins {
    `java-library`
}

group = "com.binnykanjur.keycloak.protocolmapper"
version = "1.0.0"
description = "Keycloak RESTful Claims Provider"

java {
    sourceCompatibility = JavaVersion.VERSION_11
    targetCompatibility = JavaVersion.VERSION_11
}

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.json:json:20240303")
    implementation("org.keycloak:keycloak-core:25.0.0")
    implementation("org.keycloak:keycloak-services:25.0.0")
    implementation("org.keycloak:keycloak-server-spi:25.0.0")
    implementation("org.keycloak:keycloak-server-spi-private:25.0.0")
}

tasks {
    val copyToLib by creating(Copy::class) {
        into("$rootDir/../keycloak/providers")
        from(configurations.runtimeClasspath) {
            include("json-*")
        }
    }

    jar {
        dependsOn(copyToLib)
        destinationDirectory = File("$rootDir/../keycloak/providers")
        archiveFileName.set("${rootProject.name}-${project.version}.jar")
    }
}

This configuration ensures that all necessary dependencies are included and prepares the project for building the custom protocol mapper.

3. Implementing the Protocol Mapper

• Create a new Java class that extends org.keycloak.protocol.oidc.mappers.AbstractOIDCProtocolMapper & implement the necessary methods to define your custom mapper.
• Within the mapper, use the ProviderConfigProperty class to define a configuration property for Service URL.
• Override setClaim() to make an HTTP request to the service URL, parse the JSON response, and add the retrieved claims to the token.

Here's the Java code to add to your lib\src\main\java\com\binnykanjur\keycloak\protocolmapper\RESTfulClaimsProvider.java file:

package com.binnykanjur.keycloak.protocolmapper;

import org.keycloak.models.ClientSessionContext;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.ProtocolMapperModel;
import org.keycloak.models.UserSessionModel;
import org.keycloak.protocol.oidc.mappers.AbstractOIDCProtocolMapper;
import org.keycloak.protocol.oidc.mappers.OIDCAccessTokenMapper;
import org.keycloak.protocol.oidc.mappers.OIDCAttributeMapperHelper;
import org.keycloak.protocol.oidc.mappers.OIDCIDTokenMapper;
import org.keycloak.provider.ProviderConfigProperty;
import org.keycloak.representations.IDToken;
import org.json.JSONObject;
import java.io.StringReader;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class RESTfulClaimsProvider extends AbstractOIDCProtocolMapper
        implements OIDCAccessTokenMapper, OIDCIDTokenMapper {

    private static final List<ProviderConfigProperty> configProperties = new ArrayList<>();

    public static final String INCLUDED_SERVICE_URL = "included.service.url";
    public static final String INCLUDED_SERVICE_URL_LABEL = "Service Url";
    public static final String INCLUDED_SERVICE_URL_HELP_TEXT = "Endpoint to get claims from";

    public static final String PROVIDER_ID = "oidc-RESTful-claims-mapper";

    static {
        ProviderConfigProperty property = new ProviderConfigProperty();
        property.setName(INCLUDED_SERVICE_URL);
        property.setLabel(INCLUDED_SERVICE_URL_LABEL);
        property.setHelpText(INCLUDED_SERVICE_URL_HELP_TEXT);
        property.setType(ProviderConfigProperty.STRING_TYPE);
        property.setRequired(true);
        property.setDefaultValue("");
        configProperties.add(property);

        OIDCAttributeMapperHelper.addIncludeInTokensConfig(configProperties, RESTfulClaimsProvider.class);

        // Don't include claims in ID Token by default
        for (ProviderConfigProperty prop : configProperties) {
            if (OIDCAttributeMapperHelper.INCLUDE_IN_ID_TOKEN.equals(prop.getName())) {
                prop.setDefaultValue("false");
            }
        }
    }

    @Override
    public List<ProviderConfigProperty> getConfigProperties() {
        return configProperties;
    }

    @Override
    public String getId() {
        return PROVIDER_ID;
    }

    @Override
    public String getDisplayType() {
        return "RESTful Claims Provider";
    }

    @Override
    public String getDisplayCategory() {
        return TOKEN_MAPPER_CATEGORY;
    }

    @Override
    public String getHelpText() {
        return "Adds custom claims returned from a call to the Service Url";
    }

    @Override
    protected void setClaim(IDToken token, ProtocolMapperModel mappingModel, UserSessionModel userSession,
            KeycloakSession keycloakSession, ClientSessionContext clientSessionCtx) {
        String serviceUrl = mappingModel.getConfig().get(INCLUDED_SERVICE_URL);
        if (serviceUrl == null || serviceUrl.isBlank()) {
            return;
        }

        String userId = token.getSubject();
        if (userId == null) {
            return;
        }

        // Fetch data from the remote GET endpoint using the authenticated user's ID
        Map<String, Object> customClaims = fetchCustomClaimsFromApi(serviceUrl, userId);

        // Add the retrieved key-value pairs as custom claims to the token
        for (Map.Entry<String, Object> entry : customClaims.entrySet()) {
            Object claimValue = entry.getValue();
            if (claimValue == null) {
                continue;
            }

            token.getOtherClaims().put(entry.getKey(), claimValue);
        }
    }

    private Map<String, Object> fetchCustomClaimsFromApi(String baseEndpointUrl, String userId) {
        Map<String, Object> customClaims = new HashMap<>();
        HttpClient client = HttpClient.newHttpClient();

        // Encode the userId to ensure it is URL-safe
        String encodedUserId = URLEncoder.encode(userId, StandardCharsets.UTF_8);
        // Construct the complete endpoint URL with the userId as a query parameter
        String endpointUrl = String.format("%s?objectId=%s", baseEndpointUrl, encodedUserId);

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(endpointUrl))
                .GET()
                .header("Accept", "application/json")
                .build();

        try {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            String json = response.body();

            JSONObject jsonResponse = new JSONObject(json);
            for (String key : jsonResponse.keySet()) {
                Object value = jsonResponse.get(key);
                customClaims.put(key, value);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

        return customClaims;
    }

}

4. Register the Protocol Mapper

Next, we need to create a META-INF directory and a services directory within it. Inside the services directory, create a file named org.keycloak.protocol.ProtocolMapper.

Here's the text to add to your lib\src\main\resources\META-INF\services\org.keycloak.protocol.ProtocolMapper file.

com.binnykanjur.keycloak.protocolmapper.RESTfulClaimsProvider

5. Build the project

Now that everything is set up, build the project.

To build the project, run the following command in the terminal from the project root folder (restful-claims-provider):

gradle clean build

Build Output: After running the build command, you will find the compiled JAR file keycloak-rest-protocol-mapper-sample-1.0.0.jar and json-20240303.jar in the keycloak/providers directory within the working directory.

By following these steps, you will have your custom protocol mapper JAR & the necessary dependencies ready to be deployed into Keycloak.

Configuring Docker Compose for Keycloak and ASP.NET Core API

In this section, you will set up Docker Compose to allow external access to Keycloak while blocking external access to the REST API. This should be the case to make Keycloak reachable for user authentication and management tasks, while the API remains internal and accessible only to services (keycloak) within the Docker network.

Key Components of the Docker Compose Setup

• Services
  • claimsprovider-keycloak:
    • Ports: Exposes port 8080 internally and maps it to port 5000 on the host.
    • Command: start-dev to start Keycloak in development mode.
    • Volumes: Mounts directories from the host to the container. The ./keycloak/providers directory is used to include custom protocol mapper providers. Keycloak will automatically detect and deploy the new protocol mapper upon startup.
    • Networks: Connected to both default and internal-network allowing it to be accessed from outside and to communicate with the internal claimsprovider.api service.
  • claimsprovider.api:
    • Networks: Connected only to internal-network, ensuring it is not accessible from outside the Docker host.
• Networks:
  • Internal Network: internal-network is defined as internal, restricting access to only containers within this network.

Enter the working folder (keycloak-protocol-mapper-sample) & update the docker-compose.yml file content with:

version: '3.4'

name: keycloak-rest-protocol-mapper-sample
services:
  claimsprovider-keycloak:
    image: quay.io/keycloak/keycloak:25.0.0
    container_name: claimsprovider-keycloak
    restart: always
    ports:
      - 5000:8080
    environment:
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
    command: start-dev
    volumes:
      - ./keycloak/providers:/opt/keycloak/providers
    networks:
      - default
      - internal-network

  claimsprovider.api:
    image: ${DOCKER_REGISTRY-}claimsproviderapi
    container_name: claimsprovider.api
    build:
      context: ClaimsProvider.Api
      dockerfile: Dockerfile
    depends_on:
      - claimsprovider-keycloak
    networks:
      - internal-network

networks:
  internal-network:
    internal: true

Configuring Keycloak to Use the Custom Mapper

In this section, we'll go through the steps to configure the custom protocol mapper in Keycloak; from running Docker Compose to setting up the realm, user, client, and protocol mapper.

1. Run Docker Compose

Run the following command in your terminal to start the Keycloak and ClaimsProvider API services:

docker-compose up -d

2. Verify the Setup

1. Check Keycloak Admin Console:
   • Open your browser and navigate to http://localhost:5000.
   • Log in with the admin credentials (admin/admin).
   • Ensure the Keycloak admin console is accessible.
2. Check Claims Provider API:
   • In your browser, navigate to http://localhost:8080/api/v1/claims?objectId=123.
   • Confirm that the ClaimsProvider API is not accessible.
3. Check Custom Protocol Mapper Installation:
   • After you log in to the Keycloak console with the admin credentials (admin/admin), make sure you are able to see the newly added oidc-RESTful-claims-mapper in Provider Info tabs, protocol-mapper section.

3. Set Up Realm, User, and Client

In this step, you will create and configure a Realm along with a test User & a Client. For detailed steps, a visual guide is provided below. Navigate through the images to see each step.

4. Configure Protocol Mapper

Once you have the realm, user & a client created, configure the custom protocol mapper for the sample-client client. A visual guide is also provided below.

• Navigate to the realm sample-realm. This is where we add the custom protocol mapper.
• Go to Clients and select the client sample-client.
• In the client details page, navigate to the Client scopes tab & click on the sample-client-dedicated scope to get into its Dedicated scopes page.
• Select the Mappers tab & Click Configure a new mapper.
• Select RESTful Claims Provider from the Configure a new mapper popup.
• Configure the mapper:
  • Mapper Type: This should be the name of our custom protocol mapper - RESTful Claims Provider.
  • Name: Give mapper a descriptive name. - oidc-RESTful-claims-mapper
  • Service URL: Enter the internal URL of the REST endpoint that the mapper will call to retrieve additional claims. - http://claimsprovider.api:8080/api/v1/claims. Here, claimsprovider.api is the name of the API container and 8080 is the port it listens on.
  • Add to ID token: Turn it Off.
  • Add to access token: As we want the claims to be added to the access token, turn it On.
  • Add to lightweight access token: Turn it Off.
  • Click Save.

Visual guide of the above steps:

Testing the Integration

To make sure the solution working as expected, you will check the generated access token & ensure it contains the claims added by the custom protocol mapper. Refer the image below for details.

Exporting and Importing Keycloak Configuration

After successfully testing the integration, it's important to ensure that your Keycloak configurations can be easily replicated in different environments or restored if needed. In this additional section, we’ll detail the steps to export the realm, user, client, and mapper configurations from the running Keycloak container using Docker commands. You will also see how to update your docker-compose.yml file to import these configurations automatically, making your setup more robust and easier to manage.

Exporting Keycloak Realm

1. Identify the Keycloak Container:

First, make sure that the Keycloak container is running by issuing the following command:

docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}"

Look for the container with the name claimsprovider-keycloak and image quay.io/keycloak/keycloak:25.0.0.

2. Export the Realm Configuration:

Use the following command to export the Keycloak realm configuration to a JSON file:

docker exec -u root claimsprovider-keycloak /opt/keycloak/bin/kc.sh export --file /tmp/sample-realm-export.json --realm sample-realm

3. Copy the Exported JSON File to Host:

Once the export is complete, copy the JSON file from the Keycloak container to your host machine.

Enter the working directory (keycloak-protocol-mapper-sample) & run the following command:

mkdir keycloak\data\import

docker cp claimsprovider-keycloak:/tmp/sample-realm-export.json ./keycloak/data/import/sample-realm.json

Updating docker-compose.yml for Import

To ensure the exported configuration is automatically imported when the Keycloak container starts & make your setup process repeatable and consistent, update the docker-compose.yml as follows:

1. Add Volume for Import:

Ensure there is a volume configuration to map the directory containing the exported JSON file:

volumes:
  - ./keycloak/data/import:/opt/keycloak/data/import

2. Update Keycloak Service Command:

Modify the claimsprovider-keycloak service command to include the import option:

command: start-dev --import-realm

docker-compose.yml Update:

version: '3.4'

name: keycloak-rest-protocol-mapper-sample
services:
  claimsprovider-keycloak:
    image: quay.io/keycloak/keycloak:25.0.0
    container_name: claimsprovider-keycloak
    restart: always
    ports:
      - 5000:8080
    environment:
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
    command: start-dev --import-realm
    volumes:
      - ./keycloak/data/import:/opt/keycloak/data/import
      - ./keycloak/providers:/opt/keycloak/providers
    networks:
      - default
      - internal-network

  claimsprovider.api:
    image: ${DOCKER_REGISTRY-}claimsproviderapi
    container_name: claimsprovider.api
    build:
      context: ClaimsProvider.Api
      dockerfile: Dockerfile
    depends_on:
      - claimsprovider-keycloak
    networks:
      - internal-network

networks:
  internal-network:
    internal: true

Conclusion

In this article, we've walked through the entire process of developing a custom protocol mapper for Keycloak, integrating it with an ASP.NET Core Minimal API, and managing the setup using Docker Compose. Here’s a quick recap of what we covered:

• Creating the ASP.NET Core Minimal API: We developed an ASP.NET Core Minimal API with a GET /api/v1/claims endpoint that dynamically fetches user claims.
• Building the Custom Protocol Mapper: Using Gradle, we set up our project, managed dependencies, and built the custom protocol mapper.
• Setting Up the Environment: We configured Docker Compose to run Keycloak and the ClaimsProvider API, ensuring Keycloak is externally accessible while the API remains internal.
• Deploying and Configuring Keycloak: We started our services, verified the setup, and configured Keycloak with a realm, user, client, and our custom protocol mapper.
• Exporting and Importing Configuration: For repeatability and consistency, we demonstrated how to export Keycloak configurations and update the Docker Compose file for automatic imports.

By following these steps, you can ensure that your Keycloak instance is capable of dynamically fetching and including custom claims in its tokens, tailored to your specific requirements. This setup not only enhances the security and flexibility of your authentication system but also integrates seamlessly with external data sources for real-time claim generation.

---

Add a comment