Installation using Kubernetes

This section describes how to deploy ASM using Kubernetes (K8s).

note

The instructions below refer to a fresh installation of ASM.

You can install K8s in one of the following ways:

🖥️ Local installation using minikube.

☁️ Cloud installation using EKS by Amazon Web Services (AWS).

Requirements

Local installation

To run ASM locally on a K8s minikube environment, install:

• Kubernetes and kubectl

  The open-source system for automating deployment, scaling, and management of containerized applications along with its command line tool.

• minikube

  Local K8s for macOS, Linux, and Windows.

• AWS CLI

  The official command line interface to manage AWS services.

• Helm

  The package manager for K8s.

• Docker and Docker Compose

  The open platform to package and run applications in isolated containers along with the tool for defining and running multi-container Docker applications.

Cloud installation

To run ASM on a K8s cloud AWS environment using Amazon EKS, install:

• kubectl

  The official command line tool for communicating with a K8s cluster.

• Helm

  The package manager for K8s.

• AWS CLI

  The official command line interface to manage AWS services.

• Docker and Docker Compose

  The open platform to package and run applications in isolated containers along with the tool for defining and running multi-container Docker applications.

• eksctl

  A CLI tool for creating and managing clusters on Amazon EKS.

Preparation

Axiomatics Services Manager is downloaded via AWS CLI. If you have not already done so, install AWS CLI following the instructions in AWS documentation.

1. Configure the Axiomatics AWS CLI account using the aws configure command, as explained in the AWS documentation. This procedure requires an Access key ID and a Secret access key, which will have been provided to you by Axiomatics.

   tip

   Optionally, you can create and use profiles to store these settings, as described here. This could come in handy in deployments using a remote registry which requires an additional AWS CLI account from your side.

2. The ASM distribution is downloaded as a ZIP file, containing a pre-configured Docker-based installation. Run the following command in the terminal to download ASM:

   aws s3api get-object --bucket axiomatics-customer-artifacts --key releases/com/axiomatics/axiomatics-services-manager/axiomatics-services-manager/7.3.0-0/axiomatics-services-manager-7.3.0-0.zip axiomatics-services-manager-7.3.0-0.zip

Docker registry

The Docker registry is a server side application that stores and lets you distribute Docker images. It is recommended to work with the remote registry provided by Axiomatics, or optionally your own remote registry. However, for local installations only, if a remote registry cannot be supported by your specific setup, then you can deploy a local registry server.

Follow the steps below according to your specific setup:

Axiomatics registry

Log in to the Amazon ECR registry provided by Axiomatics.

aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin 748131003707.dkr.ecr.eu-central-1.amazonaws.com

Remote registry

Important

In the snippet below replace

• <region_code> with the AWS Region that you want to create your cluster in
• <aws_account_id> with your AWS account ID

1. Configure your AWS CLI account using the aws configure command, as explained in the AWS documentation.

2. Log in to your Amazon ECR registry.

   aws ecr get-login-password --region <region_code> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com

3. Create the following repositories in your ECR registry, as described in the AWS documentation:

   axiomatics/adm
   axiomatics/api-gateway
   axiomatics/asm-core
   axiomatics/postgres
   axiomatics/influxdb
   axiomatics/pd-api
   axiomatics/pd-ui
   axiomatics/keycloak-mappers
   axiomatics/service-aggregator

Local registry

Important

In the snippets below replace <hostname> with your own hostname. In case you do not have a hostname and you are working with localhost, add the special domain host.docker.internal.

1. Create a directory.

   mkdir certs

2. Create a self-signed certificate.

   openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes -keyout certs/registry.key -out certs/registry.crt -subj "/CN=<hostname>" -addext "subjectAltName=DNS:<hostname>"

3. Add the certificate to Docker trusted certificates. Based on your OS:

   • Windows:

     Install the registry.crt certificate on your Local machine in the Trusted Root Certification Authorities.

   • macOS and Linux:

     1. Create the certificates directory, if it doesn't exist already.

        sudo mkdir -p /etc/docker/certs.d/<hostname>:5000

     2. Copy the generated certificate to the Docker certificates directory.

        sudo cp certs/registry.crt /etc/docker/certs.d/<hostname>:5000/ca.crt

4. Restart Docker and minikube.

5. Start the local registry.

   docker run -d -p 5000:5000 --restart=always --name registry -v "$(pwd)"/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.crt -e REGISTRY_HTTP_TLS_KEY=/certs/registry.key registry:2 

6. Create a PEM file and copy it to the Minikube configurations directory.

   openssl x509 -in certs/registry.crt -out certs/registry.pem -outform PEM 

   cp certs/registry.pem ~/.minikube/certs

Docker image

Important

If you are using the Axiomatics registry you should skip this section. However, if you are using your own remote or local registry, the steps below are required.

Build and push the Docker image

1. Extract the ASM distribution ZIP file to the folder from where you want to run ASM.

2. Navigate to the docker folder of the extracted distribution.

   1. Open the credentials.txt file and update the placeholder values for Access key ID and Secret access key with the values provided by Axiomatics.

      note

      These are the same values that were used in setting up AWS CLI.

   2. Open the .env file and add your registry.

      Remote registry

      REGISTRY=<aws_account_id>.dkr.ecr.<region_code>.amazonaws.com/axiomatics/

      Local registry

      REGISTRY=localhost:5000/

   3. Optionally, for deployments that include Access Decision Service (ADS):

      Add your hostname

      Open the .env file and update the values of HOST and CERTIFICATE_CN with your own hostname.

      note

      In case you are working with localhost, use the special domain host.docker.internal.

3. Enable the BuildKit feature of Docker:

   Windows

   $Env:DOCKER_BUILDKIT=1

   Linux/macOS

   export DOCKER_BUILDKIT=1

4. Run the following command to build ASM.

   docker-compose -f docker-compose.yml -f docker-compose.dashboard.yml build 

5. Run the following command to push the image to your registry.

   docker-compose -f docker-compose.yml -f docker-compose.dashboard.yml push

SSL Certificate

In production environments, you should import trusted certificates only, while in dev and test environments you have the ability to create a self-signed certificate.

Import

ASM requires a PKCS#12 (Public Key Cryptography Standard #12) certificate. In case you already have a certificate with different format such as .pem, you should convert it to PKCS#12. The certificate file must be named cert.keystore.p12. During the creation of the certificate, the name (or alias) must be set to server.

1. Navigate to the docker/api.gateway/certs/ folder of the extracted distribution.
2. Copy your certificate to this folder.

Create

Generate a self-signed certificate as follows:

1. Navigate to the docker folder of the extracted distribution.

2. Open the .env file and update the following entries:

   • CERTIFICATE_CN with your common name
   • CERTIFICATE_KEY_STORE_PASSWORD with your password

3. Generate the certificate:

Linux/macOS

1. From the docker/api.gateway/certs/ folder, run the following command.

sh generate_self_signed_certificate.sh

2. Make sure that the following files were created:

• cert.keystore.p12
• server.crt

Windows

1. From the docker\api.gateway\certs folder, run the following commands using PowerShell.

Important

In the first command, replace <your_cert_key_store_pw> and <your_cert_cn> with the same values that were previously used in the .env file.

docker build -t certificate -f ./Dockerfile.cert . --build-arg CERTIFICATE_KEY_STORE_PASSWORD=<your_cert_key_store_pw> --build-arg CERTIFICATE_CN=<your_cert_cn>

docker create --name cert certificate; docker cp cert:/cert.keystore.p12 ./cert.keystore.p12; docker cp cert:/server.crt ./server.crt; docker rm -f cert; docker image rm certificate

2. Make sure that the following files were created:
   • cert.keystore.p12
   • server.crt

ASM K8s installation

Local installation

1. Navigate to the kubernetes directory.

   Add the axiomatics_ASM.license license file and launch a terminal window in this directory.

2. Run the following command to start minikube.

   minikube start
   #In case the system is slow, use the following command instead:
   minikube start --memory 4096

3. After minikube has started, launch a new terminal window and create a tunnel.

   minikube tunnel
   Important

   This terminal window should remain open during the K8s installation.

4. On the initial terminal window, create the ASM namespace.

   kubectl create namespace axiomatics-asm

Create ConfigMaps

kubectl create configmap certs --from-file=../docker/api.gateway/certs -n axiomatics-asm

Create secrets

1. Create a Kubernetes database secret.

   kubectl create secret generic db-connection \
   --from-literal=dbuser=postgres \
   --from-literal=dbpassword=password \
   --from-literal=dbhost=db \
   --from-literal=dbport=5432 \
   --from-literal=adm_dbname=domain_manager \
   --from-literal=keycloak_dbname=keycloak \
   --from-literal=asm_dburl=postgresql://db:5432/asm \
   --namespace=axiomatics-asm

2. Create an ADM credentials secret.

   kubectl create secret generic adm-credentials \
   --from-literal=admApiAuthUsername=admin \
   --from-literal=admApiAuthPassword=secret -n axiomatics-asm

3. Create a Keycloak credentials secret.

   kubectl create secret generic keycloak-credentials \
   --from-literal=keycloakUser=admin \
   --from-literal=keycloakPassword=admin -n axiomatics-asm

4. Create a secret for pulling from ECR.

   Important

   Skip this step if you are using a Local registry.

   kubectl create secret docker-registry regcred \
   --docker-server=<aws_account_id>.dkr.ecr.<region_code>.amazonaws.com \
   --docker-username=AWS --docker-password=$(aws ecr get-login-password) \
   --namespace=axiomatics-asm

5. Create an ASM License secret.

   kubectl create secret generic asm-license \
   --from-file=axiomatics_ASM.license \
   --namespace axiomatics-asm

Final steps

You can deploy ASM with the Dashboard functionality enabled or disabled.

1. Follow the steps below according to your specific setup:

   Disabled

   Navigate to the kubernetes directory and install ASM using Helm.

   Important

   Replace the placeholders with your own values. For details, see the table below.

   helm install -n axiomatics-asm asm -f asmcharts/values.yaml asmcharts --set imageRegistry=<image_registry> --set asmHostname=<your_asm_hostname>

   tip

   Store this command in a TXT file for future reference, such as when upgrading to a later version of ASM or in case you want to enable cluster-level logging.

   Enabled

   The Dashboard provides visualization of key metrics for monitoring the authorization performance of the running instances of Access Decision Service (ADS).

   Navigate to the kubernetes directory and install ASM using Helm.

   Important

   Replace the placeholders with your own values. For details, see the table below.

   helm install -n axiomatics-asm asm -f asmcharts/values.yaml asmcharts --set imageRegistry=<image_registry> --set asmHostname=<your_asm_hostname> --set enableMetricsDashboard=true

   tip

   Store this command in a TXT file for future reference, such as when upgrading to a later version of ASM or in case you want to enable cluster-level logging.

   Placeholder	Value
   <image_registry>	The image registry: - For local registry: host.docker.internal:5000 - For remote or Axiomatics registry: <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com/axiomatics/
   <your_asm_hostname>	The hostname assigned to the instance running ASM. The default is localhost.

2. List all pods and make sure they are all running.

   kubectl get all -n axiomatics-asm

Cloud installation

Important

In the snippets below replace

• <region_code> with the AWS Region that you want to create your cluster in
• <my_cluster> with the name of your cluster
• <aws_account_id> with your AWS account ID
• <db_endpoint> with your own DB instance endpoint value such as mydb.123456789012.region-code.rds.amazonaws.com

1. Navigate to the kubernetes directory and add the axiomatics_ASM.license license file.

2. Launch a terminal window in this directory.

Create an Amazon EKS cluster

1. Create a cluster.

   eksctl create cluster --name <my_cluster> --region <region_code>

2. Enable kubectl to communicate with your cluster.

   aws eks update-kubeconfig --region <region_code> --name <my_cluster>

Create a load balancer using Helm

1. Creating an IAM OIDC provider for your cluster.

   eksctl utils associate-iam-oidc-provider --region <region-code> --cluster <my_cluster> --approve

2. Create an IAM policy.

   1. Download an IAM policy for the AWS Load Balancer Controller that allows it to make calls to AWS APIs on your behalf.

      curl -o iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json

   2. Create an IAM policy using the policy downloaded in the previous step.

      aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json

3. Create an IAM role and a Kubernetes service account.

   eksctl create iamserviceaccount \
   --cluster=<my_cluster> \
   --namespace=kube-system \
   --name=aws-load-balancer-controller \
   --role-name "AmazonEKSLoadBalancerControllerRole" \
   --attach-policy-arn=arn:aws:iam::<aws_account_id>:policy/AWSLoadBalancerControllerIAMPolicy \
   --approve

4. Install the AWS Load Balancer Controller using Helm.

   1. Add the eks-charts repository.

      helm repo add eks https://aws.github.io/eks-charts

   2. Update your local repo to make sure that you have the most recent charts.

      helm repo update

   3. Install the AWS Load Balancer Controller.

      Important

      Replace <vpc_id> with your own VPC ID which can be found on the EKS cluster's Networking tab.

      helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
       -n kube-system \
       --set clusterName=<my_cluster> \
       --set serviceAccount.create=false \
       --set serviceAccount.name=aws-load-balancer-controller \
       --set region=<region_code> \
       --set vpcId=<vpc_id> \

Create and configure the Axiomatics namespace

1. Create the namespace.

   kubectl create ns axiomatics-asm

2. Upload an SSL certificate to the AWS Certificate Manager (ACM).

   note

   In case you don't have a self-signed certificate, you can create one locally with the following command, and then upload it to AWS Certificate Manager (ACM).

   openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes -keyout certs/registry.key -out certs/registry.crt -subj "/CN=<hostname>" -addext "subjectAltName=DNS:<hostname>"

Create the databases

1. Create a PostgreSQL database v. 12.9 using Amazon RDS, in the same VPC as the EKS cluster.

2. Configure the security groups. Enable ingress on the RDS security group on port 5432 from VPC CIDR.

3. Create a temporary POD to log in to the database.

   kubectl run -n axiomatics-asm pgshell --pod-running-timeout=5m0s --rm -i --tty --image postgres -- /bin/bash

4. Log in to the RDS.

   psql -h <db_endpoint> -U postgres template1

5. Create the databases.

   create database asm;
   create database keycloak;
   create database domain_manager;
   create database pd;
   grant all privileges on database asm to postgres;
   grant all privileges on database domain_manager to postgres;
   grant all privileges on database keycloak to postgres;
   grant all privileges on database pd to postgres;

6. Exit.

   \q
   exit

Create ConfigMaps

kubectl create configmap certs --from-file=../docker/api.gateway/certs/cert.keystore.p12 -n axiomatics-asm

Create secrets

1. Create a Kubernetes database secret.

   Important

   Replace the placeholders for the database user, password, and host URL with your own.

   kubectl create secret generic db-connection \
   --from-literal=dbuser=<db_user> \
   --from-literal=dbpassword=<db_password> \
   --from-literal=dbhost=<db_endpoint> \
   --from-literal=dbport=5432 \
   --from-literal=adm_dbname=domain_manager \
   --from-literal=keycloak_dbname=keycloak \
   --from-literal=asm_dburl=postgresql://<db_endpoint>:5432/asm \
   --namespace=axiomatics-asm  

2. Create an ADM credentials secret.

   kubectl create secret generic adm-credentials \
   --from-literal=admApiAuthUsername=admin \
   --from-literal=admApiAuthPassword=secret -n axiomatics-asm

3. Create a Keycloak credentials secret.

   kubectl create secret generic keycloak-credentials \
   --from-literal=keycloakUser=admin \
   --from-literal=keycloakPassword=admin -n axiomatics-asm

4. Create a secret for pulling from ECR.

   kubectl create secret docker-registry regcred \
   --docker-server=<aws_account_id>.dkr.ecr.<region_code>.amazonaws.com \
   --docker-username=AWS --docker-password=$(aws ecr get-login-password) \
   --namespace=axiomatics-asm

5. Create an ASM License secret.

   kubectl create secret generic asm-license \
   --from-file=axiomatics_ASM.license -n axiomatics-asm

Final steps

You can deploy ASM with the Dashboard functionality enabled or disabled.

1. Follow the steps below according to your specific setup:

   Disabled

   Navigate to the kubernetes directory and install ASM using Helm.

   Important

   Replace the placeholders with your own values. For details, see the table below.

   helm install -n axiomatics-asm asm -f asmcharts/values.yaml asmcharts --set imageRegistry=<your_image_registry> --set asmHostname=<your_asm_hostname> --set useExternalDatabase=true --set gateway.service.type=ClusterIP --set isAwsInstallation=true --set certificateArn=<certificate_ARN>

   tip

   Store this command in a TXT file for future reference, such as when upgrading to a later version of ASM or in case you want to enable cluster-level logging.

   Enabled

   The Dashboard provides visualization of key metrics for monitoring the authorization performance of the running instances of Access Decision Service (ADS).

   1. Create an IAM policy and role using eksctl.

   2. Install the Amazon EFS driver using Helm.

   3. Create two Elastic File Systems (EFS) in the VPC that EKS is running named:

      • influx-data
      • influx-config

   4. Find the security group of the EKS cluster.

      Select EKS cluster > Networking > Cluster security group

   5. Use the same security group for the two EFS file systems you created previously.

      Select EFS > select the File System > Network > Manage

   6. Navigate to the kubernetes directory and install ASM using Helm.

      :::caution Important Replace the placeholders with your own values. For details, see the table below. :::

      helm install -n axiomatics-asm asm -f asmcharts/values.yaml asmcharts --set imageRegistry=<your_image_registry> --set asmHostname=<your_asm_hostname> --set useExternalDatabase=true --set gateway.service.type=ClusterIP --set isAwsInstallation=true --set certificateArn=<certificate_ARN> --set serverSslKeyStorePassword=<keystore_password> --set enableMetricsDashboard=true --set influx.efsPvInfluxDataVolumeId=<influx_data_EFS_ID> --set influx.efsPvInfluxConfigVolumeId=<influx_config_EFS_ID>
      tip

      Store this command in a TXT file for future reference, such as when upgrading to a later version of ASM or in case you want to enable cluster-level logging.

Placeholder	Value
<your_image_registry>	The image registry: - For local registry: host.docker.internal:5000 - For remote or Axiomatics registry: <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com/axiomatics/
<your_asm_hostname>	The hostname of the instance running ASM. TIP: In case you don't have a hostname, use localhost.
<certificate_ARN>	The Amazon Resource Name (ARN) specifying the server SSL certificate that you have configured on AWS ACM. This is going to be used by the load balancer.
<keystore_password>	The password that will be used to decrypt the KeyStore.
<influx_data_EFS_ID>	The Resource ID of the influx-data EFS. NOTE: Required only for deployments with Dashboard.
<influx_config_EFS_ID>	The Resource ID of the influx-config EFS. NOTE: Required only for deployments with Dashboard.

2. Upgrade the Helm charts replacing <ALB_DNS_name> with the DNS name assigned to the ALB created in front of the ASM instance.

   Important

   This step is required ONLY IF you installed ASM using Helm with localhost as the value of the asmHostname property.

   helm upgrade -n axiomatics-asm asm -f asmcharts/values.yaml asmcharts --set imageRegistry=<your_image_registry> --set asmHostname=<ALB_DNS_name> --set useExternalDatabase=true --set gateway.service.type=ClusterIP --set isAwsInstallation=true --set certificateArn=<certificate_ARN> --set serverSslKeyStorePassword=<keystore_password> --set enableMetricsDashboard=true --set influx.efsPvInfluxDataVolumeId=<influx_data_EFS_ID> --set influx.efsPvInfluxConfigVolumeId=<influx_config_EFS_ID>

3. List all pods and make sure they are all running.

   kubectl get all -n axiomatics-asm

Troubleshooting

Extra commands that could assist your troubleshooting effort.

Important

In the snippets below replace

• <region_code> with your cluster's AWS Region
• <aws_account_id> with your AWS account ID

Kubernetes statuses

ErrImagePull

If the pod status is ErrImagePull because of an expired registry credentials secret:

1. Delete secret.

   kubectl delete secret regcred -n axiomatics-asm
2. Recreate it.

   kubectl create secret docker-registry regcred --docker-server=<aws_account_id>.dkr.ecr.<region_code>.amazonaws.com --docker-username=AWS --docker-password=$(aws ecr get-login-password) --namespace=axiomatics-asm 

ImagePullBackOff

The status ImagePullBackOff means that the ECR Docker image is too large and cannot be downloaded. To resolve this issue, use the following commands to download the required image within minikube.

Important

Replace the placeholders with the appropriate values before running the commands.

minikube ssh
su - root # Default password is 'root'
apt-get update
apt-get install unzip  
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install 
aws configure # Provide AWS Access Key ID and AWS Secret Access Key
aws ecr get-login-password --region <region_code> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com
docker pull <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com/axiomatics/asm-core:<version>

What's next?

Add users

During the deployment, a Keycloak service was also installed, to serve as an authentication module for the users of ASM. Only a default administrative user is created automatically at this time. All other users must be created and assigned a role in Keycloak, before they can log in to ASM.

To add more users and map them to roles you must use Keycloak as described in Manage users in Keycloak. Then, you can assign them to projects following the instructions in the Users and projects topic.

Important

For deployments with Policy Designer, you should also configure Keycloak accordingly.

Additional configuration

Once the installation of ASM has been verified as successful, there are additional configurations you may want to consider.