Fetching env vars from a running k8s pod in one step

Environment variables set in pods

Recently I joined a new organisation. And as usual I was expected to understand the functionalities of some of the components of the product that the team was working on. All these different components are nothing but a collection of microservices deployed as Kubernetes resources in cloud. So, when I was going through some of these components and trying to debug the source code, I found that all these components are using a bunch of environment variables which are defined in the respective Kubernetes manifests. And I know this is how Kubernetes pods work, nothing out of the ordinary. But, for me to debug the source code and understand the workflow, I had to set all the environment variables that the corresponding pod uses in my local terminal session, so that the component’s source code can run smoothly in my local.

Now for this I had to describe the pods running in the staging environment and find out the environment variables set in them. I could have fetched all the environment variable names from the Kubernetes manifests for the concerned components as well, but I needed the values. The values of most of the variables were either loaded from some config maps or mostly from some Kubernetes secrets. It takes time when we have a lot of values to pickup, because we have to first find out the secret/config map name and the secret/config key name from the pod description and then we have to fetch those secrets/configs separately.

Now, I know I can pickup the values of all the environment variables set in a pod from the Kubernetes dashboard. But I don’t like using the dashboard much, because first of all its not under our control, I mean using kubectl in the terminal gives us full control of what we are doing and how we are doing it, that’s not the case with the dashboard and more importantly if we have a very big cluster with many worker nodes running heavy workloads, the dashboard always takes too much time to load. Personally, I have encountered some embarrassing situations while using the dashboard, so don’t like it much myself. But, no doubt the dashboard is still a great tool for monitoring our clusters.

What I wanted was to somehow reduce the time it takes for me to fetch the values of all the environment variables that a pod was using, no matter whether the values were being loaded from a config map or a secret or from the metadata section of the pod manifest or whether the values were hardcoded in the manifest itself. This is mainly because it’s a common thing that I have to do very often. Now, since I didn’t want to use the dashboard so I thought of writing a script that can perform all the secondary steps for me and provide me with an interface where I can just pass the pod name and the namespace and then the script would return all the environment variables with their values set in the pod. Now, enough of the backstory, let’s talk about the script itself.

In order to fetch all the env variables from a running pod we have to perform the following :

Note: I have used ibmcloud CLI commands just because my cluster is running in IBM Cloud.

  • Get the kubectl context to point to the desired cluster
❯ ibmcloud ks clustersOKName        ID                     State    Created      Workers   Location   Version        Resource Group Name   Providermycluster   buh48ajd0sl5286r7nhg   normal   3 days ago   1         Dallas     1.18.10_1531   Default               classic
❯ ibmcloud ks cluster config -c mycluster
OKThe configuration for mycluster was downloaded successfully.Added context for mycluster to the current kubeconfig file.You can now execute ‘kubectl’ commands against your cluster. For example, run ‘kubectl get nodes’.
❯ kubectl config current-context
mycluster/buh48ajd0sl5286r7nhg
  • Describe the target pod
❯ kubectl describe pod samplewebapp-5854bf9b47-k5ft5
  • Collect the values of all the env vars that are hardcoded
Containers:   samplewebapp:      Environment:         PORT: 8080         PODNAME: samplewebapp-5854bf9b47-k5ft5 (v1:metadata.name)         DB_USER: <set to the key 'db.user' of config map 'db-config'>         DB_PASSWORD: <set to the key 'db.pswd' in secret 'db-secrets'>
...
...
...
  • For those env vars whose values are loaded from a config map, get the config map name and the config key name
DB_USER: <set to the key 'db.user' of config map 'db-config'>
  • Do the same for the env vars whose values are loaded from Kubernetes secrets
DB_PASSWORD: <set to the key 'db.pswd' in secret 'db-secrets'>
  • Get/Describe the config maps or secrets from where the final values need to be fetched
  • Collect the desired env var values and make sure we base64 decode the secrets we collect from the Kubernetes secret resources
kubectl -n mynamespace get secrets/db-secrets -o jsonpath=”{.db.pswd}” | base64 --decode

Now, what I did is encapsulate all these steps inside a script that provides an easy to use interface to the user, where first of all I need to make sure that my Kubectl context is set to the desired cluster and then I just need to pass the namespace and the pod name, that’s it. Additionally, I have also added the scope for other flags, which when set determines whether to fetch all the env vars or just the ones we desire. And I think its very important we utilise these flags to restrict ourselves from fetching all the env vars each and every time even when we don’t need all of them. Because, security should be given the utmost priority no matter what hacks we apply anywhere. So, now let’s see how to use this script. It’s basically a Python script that accepts command line arguments.

Just to make it a bit more concise, I created an alias for running this python script in my ~/.zshrc or ~/.bashrc file, so that I need not go to the folder where my script is each and every time I want to use it and run python getEnvVars.py. Instead I just use the alias name anywhere I like and provide the required flags and parameters.

alias penv=”python /<path_to_the_script>/getEnvVars.py” 

Note : Please don’t forget to source the ~/.zshrc or ~/.bashrc file after adding the alias.

Here are the different ways of using it :

Help

❯ penv --help
usage: getEnvVars.py [-h] -n NAMESPACE -po POD [-f VARIABLE_NAMES] [--fetchAll]

optional arguments:
-h, --help show this help message and exit
-n NAMESPACE, --ns NAMESPACE
Target namespace
-po POD, --pod POD Target pod
-f VARIABLE_NAMES, --fetch VARIABLE_NAMES
Comma separated list of env variables to be fetched
--fetchAll Fetch all env variables

List all env variables

❯ penv -n mynamespace -po samplewebapp-5854bf9b47-k5ft5
1. PORT
2. PODNAME
3. DB_USER
4. DB_PASSWORD
...
...
...

Fetch env variable by name

❯ penv -n mynamespace -po samplewebapp-5854bf9b47-k5ft5 -f DB_USER,PORT
DB_USER : sampleuser
PORT : 8080

Fetch all env variables with their set values

❯ penv -n mynamespace -po samplewebapp-5854bf9b47-k5ft5 --fetchAll
PORT : 8080
PODNAME : samplewebapp-5854bf9b47-k5ft5
DB_USER : sampleuser
DB_PASSWORD : samplepassword
...
...
...

Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store