Skip to main content
Version: 1.16

Deployment using Kubernetes

This section describes how to deploy ADS using Kubernetes (K8s).

You can install K8s in one of the following ways:

🖥️ Local installation using minikubeOpens in a new tab.

☁️ Cloud installation using EKSOpens in a new tab by Amazon Web Services (AWS).

note

The following describes a suggested approach for Kubernetes deployment. However, you are encouraged to customize your deployment to align with your specific system configuration taking into account potential security concerns and other relevant considerations.

Requirements​

To run ADS locally on a K8s minikube environment, install:

  • KubernetesOpens in a new tab and kubectlOpens in a new tab

    The open-source system for automating deployment, scaling, and management of containerized applications along with its command line tool.

  • minikubeOpens in a new tab

    Local K8s for macOS, Linux, and Windows.

  • AWS CLIOpens in a new tab

    The official command line interface to manage AWS services.

  • HelmOpens in a new tab

    The package manager for K8s.

  • DockerOpens in a new tab and Docker ComposeOpens in a new tab

    The open platform to package and run applications in isolated containers along with the tool for defining and running multi-container Docker applications.

Docker registry​

The Docker registry is a server side application that stores and lets you distribute Docker images. Axiomatics recommends working with a remote registry both for local and cloud installations. However, for local installations only, if a remote registry cannot be supported by your specific setup, then you can deploy a local registry server.

Follow the steps below according to your specific setup:

Login to your Docker registry​

Follow the steps below according to your specific setup:

Important

In the snippet below replace

  • <region_code> with the AWS Region that you want to create your cluster in
  • <aws_account_id> with your AWS account ID

  1. Log in to your Amazon ECR registry.

    aws ecr get-login-password --region <region_code> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region_code>.amazonaws.com

  2. Create the following repositories in your ECR registry:

    axiomatics/ads

Build the Docker images​

  1. Navigate to the ADS docker folder of the extracted distribution.

  2. Edit the credentials.txt file by updating the placeholder values for Access key ID and Secret access key with the values provided by Axiomatics.

    note

    These are the same values that were used in setting up AWS CLI.

  3. Copy a valid PDP license file to the docker directory.

    note

    This file is provided separately by Axiomatics.

  4. To add one or more Attribute Connectors download the respective jar files and navigate to resources/lib/. Place them under the lib folder as shown in the structure tree below:

    ├─ lib/
        ├─ <attribute-connector-1>.jar
        ├─ <attribute-connector-2>.jar

    Additional files, such as JDBC drivers (should they be necessary), must be also added under the lib folder, in the same way as Attribute Connectors.

    For updating one or more existing Attribute Connector(s), refer to section Update Attribute Connector(s) for Kubernetes deployment.

Run the following command to build ADS.

docker build --build-arg AWS_BUCKET=axiomatics-customer-artifacts --build-arg AWS_REPO=releases --build-arg ADS_VERSION=1.16.0 . -f Dockerfile -t <aws_account_id>.dkr.ecr.<region>.amazonaws.com/axiomatics/ads:1.16.0

Run the following command to push the image to your registry.

docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/axiomatics/ads:1.16.0

Setup​

Follow the steps below according to your specific setup:

  1. Navigate to the kubernetes directory.

  2. Add the axiomatics_PDP.license license file and launch a terminal window in this directory.

  3. Create the truststore.

    1. Get the SSL certificate of the gateway that was used for ASM to create the ADS truststore.jks.

    2. Create the truststore.jks from the .crt file.

      note

      For the creation of the truststore file, convert the certificate file that you used during the installation of ASM.

    3. Use the following command to convert the .crt file to .jks:

      keytool -importcert -deststorepass changeit -keystore truststore.jks -file server.crt -alias 1 -noprompt -storetype JKS

    4. Add the truststore.jks file to the ADS distribution folder under adscharts/configuration/.

  4. If minikube is not already running follow the commands below to start minikube.

    1. Start minikube:

      minikube start

    2. After minikube has started, launch a new terminal window and create a tunnel.

      minikube tunnel
      Important

      This terminal window should remain open during the K8s installation.

  5. On the initial terminal window, create the ADS namespace.

kubectl create namespace axiomatics-ads

Create secrets​

  1. Create a secret for the Keycloak client using your own client_id and secret, which can be found on the Keycloak dashboard. 

    kubectl create secret generic ads-adm-access-secret \
    --from-literal=client-id=<client_id> \
    --from-literal=client-secret=<secret>  \
    --from-literal=client-token-uri=https://<asm_hostname>/auth/realms/asm/protocol/openid-connect/token \
    --namespace=axiomatics-ads
    note

    In case of local installation: <asm_hostname> = host.docker.internal

    For more information on how to use Keycloak with ADS, please refer to Configure Keycloak for ADSOpens in a new tab.

  2. Create a secret for pulling from ECR.

    Important

    This applies only to installations using a remote Docker registry.

    kubectl create secret docker-registry regcred \
    --docker-server=<aws_account_id>.dkr.ecr.<region_code>.amazonaws.com \
    --docker-username=AWS --docker-password=$(aws ecr get-login-password --region=<region_code>) \
    --namespace=axiomatics-ads

  3. Create an ADS License secret.

    kubectl create secret generic ads-license \
    --from-file=axiomatics_PDP.license \
    --namespace axiomatics-ads

  4. Create a secret for the ADS PEP credentials:

    kubectl create secret generic ads-pep-client-login --from-literal=username=ads-user --from-literal=password=2bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25b --namespace=axiomatics-ads
    note

    In the snippet above, the password is encrypted using SHA-256 and decodes to secret.

    You can change your password with the following command:

    echo -n yourPassword | sha256sum

Final steps​

  1. Navigate to the kubernetes directory and install ADS using Helm.

    Important

    Replace the placeholders with your own values. For details, see the table below.

    helm install -n axiomatics-ads ads -f adscharts/values.yaml adscharts --set imageRegistry=<your_image_registry> --set adsHostname=<your_ads_hostname> --set asmHostname=<your_asm_hostname> --set admDomainName=<your_adm_domain_name> --set asmProjectName=<your_asm_project_name> --set ads.metrics.enabled=true --set ads.distributedCache.enabled=true
    tip

    Store this command in a TXT file for future reference, such as when upgrading to a later version of ADS.

    ParameterDescription
    imageRegistryThe image registry:
    - For local registry: host.docker.internal:5000/
    - For remote or Axiomatics registry: `<aws_account_id>.dkr.ecr.<region_code>.``amazonaws.com/axiomatics/
    adsHostnameThe hostname of your ADS.
    asmHostnameThe hostname of your ASM.
    For local installation, use host.docker.internal.
    asmProjectNameThe name of your ASM project.
    admDomainNameThe domain name of ADM.
    ads.metrics.enabledSet this parameter to true if you want to send metrics to the ASM Dashboard.
    The default value is false.
    ads.distributedCache.enabledSet this parameter to true if you want to use the distributed caching feature.
    The default value is false.

    NOTE: The distributed cache feature, uses DNS discovery which requires the use of a headless-service.
    Its name and port values, along with the Kubernetes namespace and the cluster name are used to form the dnsQuery property's value (necessary for cluster discovery) as follows:
    {{ headlessService.name }}.{{ k8s.namespace }}.svc.cluster.local
    Their default values are name: ads-headless-cache-service and port: 57800 respectively, but can be customized in the values.yaml file.

  2. List all pods and make sure they are all running.

    kubectl get all -n axiomatics-ads

Update configuration during runtime​

Upon successfully running the service in Kubernetes you can further customize the deployment.yaml file or any other files (such as values.yaml) located under adscharts/configuration and adscharts folders respectively.

To apply your changes to either of these files, follow this process:

  1. Run the following helm upgrade command:

    helm upgrade ads adscharts --namespace axiomatics-ads

For changes specifically affecting configuration files in the adscharts/configuration folder (such as truststore.jks), follow these steps:

  1. Execute the helm upgrade command mentioned above.

  2. Restart the deployment using the following kubectl command:

    kubectl rollout restart deployment/ads -n axiomatics-ads

Troubleshooting​

Extra commands that can assist your troubleshooting efforts.

note

In the snippets below replace

  • <region-code> with your cluster's AWS Region
  • <aws_account_id> with your AWS account ID

Minikube with -p/--profile​

The name of the minikube VM being used to allow having multiple instances of minikube independently.

minikube start -p axiocube
minikube tunnel -p axiocube
minikube stop -p axiocube

Image pull policy​

If the pod already exists with the IfNotPresent pull policy and has failed, restart it.

kubectl rollout restart deploy ads -n axiomatics-ads

Kubernetes statuses​

ImageErrPull​

If the job fails with ImageErrPull because of an expired registry credentials secret

  1. Delete secret.

    kubectl delete secret  regcred -n axiomatics-ads

  2. Recreate it.

    kubectl create secret docker-registry regcred --docker-server=<aws_account_id>.dkr.ecr.<region-code>.amazonaws.com --docker-username=AWS --docker-password=$(aws ecr get-login-password) --namespace=axiomatics-ads

Next steps​

  • The instructions above assume ADS is started with a basic configuration. See the section Deployment configuration for more information.
  • Consider using the ADS caching features to further improve the system's performance. See the Attribute caching section for more information.
  • There are also additional configurations you may want to consider for your implementation of ADS. See the section Additional configurations for more information.