I am working on Kubernetes for more than 1.5 years now. It is always important to consider the Security aspects as one of the major concerns in deployments. It is always best to avoid deploying the secrets keys in the configuration files. When it comes to AWS EC2 Roles does the job, for us where We can avoid putting AWS Secrets keys in our deployments. I would like to have a similar kind of configuration in my k8 deployments.
While I was setting up EC2 Roles in my production k8 cluster with the KIAM tool. It was very painful going through so many blogs to make my setup work in the right way. That’s when I thought of writing my own blog about the same to ease the process for my fellow developers.
- Working K8 Cluster
- Cert-manager Installed already.
- Master nodes IAM Role
- Worker nodes IAM Role
- helm (v3.0.0)
Step 1: Create IAM Role as “kiam_server” with following inline policy
Step 2: Update the trust relationship for “kiam_server” as following:
As per KIAM documentation, we have to deploy the KIAM server on master nodes and KIAM agent on worker nodes.
assumeRoleArn : "arn:aws:iam::XXXX:role/kiam_server"
- key: node-role.kubernetes.io/master
- name: ssl-certs
iptables : true
- name: ssl-certs
Install Kiam with above yaml configuration.
helm install stable/kiam -f values.yaml -n kube-system
Once the service is running you can verify it using the below commands.
kubectl get daemonsets --all-namespaces -l app=kiamNAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system kiam-agent 2 2 2 2 2 kubernetes.io/role=node 24d
kube-system kiam-server 3 3 3 3 3 kubernetes.io/role=master 24d
This part is about testing the above setup by deploying simple app.
In order to Pods to access the EC2 IAM roles with KIAM setup, It needs to annotated the namespaces as follows.
Apply above configuration
kubectl apply -f namespace.default.yaml
Deploy this following aws-iam-tester app. Please check the region in the configuration.
- key: kiam
- name: aws-iam-tester
- name: AWS_DEFAULT_REGION
Once application is running you can verify
kubectl exec -it POD_NAME /bin/sh
aws sts get-caller-identity #inside pod shell.
This should result as follows
Once you see these results, It is good to go ahead and configure for your applications and roll out for other application.
In high traffic environment, There are few issues I have faced especially with restarting the pods, Please consider to customize the resource limits to appropriate values.
Thank you for reading the blog, I hope I have provided the maximum guidelines for setting up KIAM on your Kubernetes. Happy Kubernetes. Please feel free to connect with me for any issues on kubernetes.slack.com at Sreedhar Bukya.
Special thanks to my friends who help me correcting the mistakes in this blog.