Building a Kubernetes Project After a Rejection

Earlier this month, I was not picked for a role because I did not have hands on Kubernetes experience. While I was waiting to hear back on a different cybersecurity opportunity, I wanted to actually show I could work with Kubernetes and think like someone doing remediation work, not just talk about the concepts on a resume.

So I built a small Kubernetes security project. A Python script scans a running cluster for two common issues: containers running as root, and containers with no resource limits set. I set up two test pods, one insecure and one fixed, so the difference shows up in the same scan.

Starting Local

I started local with minikube, since that is a safe, free place to actually break things on purpose. The insecure pod had nothing protecting it at all, no security context, no resource limits, exactly the kind of setup that ships by default if nobody thinks to lock it down. Once the script caught both issues on it, I built a fixed version.

The first attempt used the standard nginx image, but that image is built to run as root internally, so it failed the moment I locked it down to run as non root. Switched to nginxinc/nginx-unprivileged, an image built specifically for that, and it started up clean. That was a good reminder that locking something down for security is not always a clean, one step fix. Sometimes fixing one thing breaks something else that was quietly depending on the old, less secure behavior.

Helm Chart

From there, I packaged both pods into a single Helm chart instead of keeping two separate YAML files around. One template with a setting that toggles the security fixes on or off. I went with a full Deployment, complete with liveness and readiness probes, rather than a stripped down version, since that is closer to what I would actually use on the job.

Deploying it hit another snag, a crash loop this time, caused by a mismatch between the health check port and the port the new image actually listens on. Checking the pod logs showed nginx starting up fine and then getting shut down by Kubernetes itself, not a code crash, a health check failure. Once I lined the probe's port up with the image's real port, it came up healthy.

Moving to EKS

Last step was taking the same setup out of minikube and onto an AWS EKS cluster, provisioned with CloudFormation. Same Helm chart, same script, same results, just on cloud infrastructure instead of my laptop.

Getting there was not friction free either. My first CloudFormation attempt failed because the Kubernetes version I had defaulted to had already been deprecated by AWS, a small reminder that infrastructure as code still needs to track what a cloud provider actually supports at any given moment.

I also wrote a small Lambda function that checks in on the cluster's status directly, keeping its permissions scoped to just that one cluster by ARN. Same least privilege habit I already use everywhere else in my AWS work.

I tore both the EKS cluster and the Lambda stack down once I confirmed everything worked, since they were only ever meant to exist for testing, not to sit there running and racking up cost.

What I Took Away

I did not get that role either. But I walked away from this month with a project I can actually point to: a script, a Helm chart, a cluster I stood up and tore down myself, and a couple of bugs I found and fixed along the way.

That is worth more to me than a line on a resume saying I know Kubernetes.

---

View the full project on GitHub or connect with me on LinkedIn.