Using Helm to deploy to Kubernetes
FROM : https://daemonza.github.io/2017/02/20/using-helm-to-deploy-to-kubernetes/
Introduction
In this post I will be showing how to use Helm (https://helm.sh) to build and deploy your own charts to a Kubernetes cluster.
Preface
What is Helm? Well, think of it as the apt-get / yum of Kubernetes, it’s a package manager for Kubernetes developed by the guys from Deis (https://deis.com/). If you deploy applications to Kubernetes, Helm makes it incredibly easy to version those deployments, package it, make a release of it, and deploy, delete, upgrade and even rollback those deployments as charts. Charts being the terminology thathelmuse for package of configured Kubernetes resources.
There is a second part to Helm and that isTiller. Tiller is the Helm server side that runs in Kubernetes and handles the Helm packages.
So before we can usehelmwith a kubernetes cluster, you need to installtilleron it. It’s as easy as running :
helm init
Building a Helm chart
Let’s see Helm in action, using a small little Go test api I created specifically for testing use cases like this, let’s build a helm chart of it.
git clone https://github.com/daemonza/testapi.git;
cd testapi
First create a skeleton structure chart
helm create testapi-chart
This will create atestapi-chartdirectory. Inside this directory the three files we are the most interested in for is Chart.yaml, values.yaml and NOTES.txt.
- Chart.yaml describes the chart, as in it’s name, description and version.
values.yaml is stores variables for the template files templates directory. If you have more complex deployment needs, that falls outside the default templates capability, edit the files in this directory. They are normal Go templates, Hugo ( https://gohugo.io ) which btw powers this blog, have a nice Go template primer ( https://gohugo.io/templates/go-templates/ ), if you need more information on how to work with Go templates.
NOTES.txt is used to give information after deployment to the user that deployed the chart. For example it might explain how to use the chart, or list default settings, etc. For this post I will keep the default message in it.
Open Chart.yaml and fill out the details of the application your deploying. Using thetestapias a example, this is how my Chart.yaml looks like :
apiVersion
:
v1
description
:
A simple api for testing and debugging
name
:
testapi-chart
version
:
0.0.1
Now openvalues.yamland edit it as needed. Again, using thetestapias a example this is how myvalues.yamlfile looks like.
replicaCount:
2
image:
repository: daemonza/testapi
tag: latest
pullPolicy: IfNotPresent
service:
name: testapi
type: ClusterIP
externalPort:
80
internalPort:
80
resources:
limits:
cpu:
100
m
memory:
128
Mi
requests:
cpu:
100
m
memory:
128
Mi
Run
helm
lint
in your testapi_chart directory to make sure everything is ok. If everything is good, you can package the chart as a release by running :
helm
package testapi-chart --
debug
I like to add the--debugflag to see the output of the packaged chart. Output should look similar to the following
Saved /Users/daemonza/testapi/testapi-chart/testapi-chart-
0.
0.1
.tgz
to
current
directory
Saved /
Users
/daemonza/testapi/testapi-chart/testapi-chart-
0.0
.1
.tgz
to
/
Users
/daemonza/.helm/repository/
local
From that we can see that the chart is placed in our current directory as well as in our local helm repository. To deploy this release, we can point helm directly to the chart file as follows :
helm
install
testapi-chart-0
.1
.0
.tgz
And your output should look similar to the following :
NAME: ordered-quoll
LAST DEPLOYED: Wed Mar
1
09
:
39
:
48
2017
NAMESPACE:
default
STATUS: DEPLOYED
RESOURCES:
==
>
v1/Service
NAME CLUSTER-IP EXTERNAL-
IP
PORT
(S)
AGE
ordered-quoll-testapi-ch 10.0.0.133
<
none
>
80/TCP 0s
==
>
extensions/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
ordered-quoll-testapi-ch
2
2
2
0
0
s
NOTES:
1.
Get the application URL by running these commands:
export
POD_NAME=$(kubectl get pods --
namespace
default
-l
"app=ordered-quoll-testapi-ch"
-o jsonpath=
"{.items[0].metadata.name}"
)
echo
"Visit http://127.0.0.1:8080 to use your application"
kubectl port-forward $POD_NAME
8080
:
80
From the above we can see that adeploymentwas created in kubernetes, the testapi got scaled to two pods and aservicegot created to expose thedeploymenton the cluster IP on port 80. And the NOTES.txt file tells us how to access the pod.
List the deployed packages with their release versions by running :
helm
ls
Which should return output similar to the following
NAME REVISION UPDATED STATUS CHART
ordered-quoll
1
Wed Mar
1
11
:
48
:
52
2017
DEPLOYED testapi-chart-
0.1
.0
Modify theChart.yamlfile and change the version from 0.1.0 to 0.1.1 package and deploy the 0.1.1 chart. Runninghelm lsagain now shows us that we have two packages of the testapi deployed
ordered-quoll
1
Wed Mar
1
11
:
48
:
52
2017
DEPLOYED testapi-chart-
0.1
.0
wishful-ibis
1
Wed Mar
1
12
:
03
:
31
2017
DEPLOYED testapi-chart-
0.1
.1
Let’s confirm that the testapi indeed deployed, and that there is two versions of it running :
kubectl
get
deployments
You should have output similar to :
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
ordered-quoll-testapi-cha
1
1
1
1
11
m
wishful-ibis-testapi-cha
1
1
1
1
3
m
Time to get rid of the older 0.1.0 deployment of the testapi chart. Using it’s package name fromhelm lsrun :
helm
delete
ordered-quoll
Confirm again withkubectl get deploymentsthat it really is removed. Expected output similar to :
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
wishful-ibis-testapi-cha
1
1
1
1
7
m
As we can see theordered-quoll0.1.0 version is removed. But what happens if we want to go back. Imagine this scenario, we deployed a new version of a application, and for some reason its got a problem and we need to rollback to the previous version. No worries,helmgot our back. Simply run
helm rollback ordered-quoll
1
Which means we want to rollback the testapi package ordered-quoll one revision back.
Expected output
Rollback
was a
success
! Happy Helming!
But what, if you cannot remember what the name was of a deleted package? Or just want to see all the packages that’s been deleted? No problem, run :
helm
ls --deleted
And if you want to see it ordered by date just add a-d
We have really gone a round about way of deploying packages so far with helm, normally you would only want to upgrade a package, instead of deploying a new version alongside it, unless off course your following a blue / green style deployment process.
Let’s test upgrading a release. Open the testapi project, edit the Chart.yaml as a example and change the description, and version number, package the release withhelm package ., but now instead of usinghelm installrun
helm
upgrade ordered-quoll .
This will upgrade ourordered-quollrelease to the changes we just made. Runninghelm lsnow, you should see the new version next toordered-quoll. And off course we can rollback this release as before.
For more information on usingHelmlook athttps://github.com/kubernetes/helm/blob/master/docs/using_helm.md
While this works well, it would be nicer to put ourchartin a Helm repository, as it’s then easy to share the chart or access it from other clusters, etc.
Setting up Helm repository
A Helm repository is nothing more than just a web server that’s able to serve a index.yaml file and chart files, which is really just tar.gz file containing the generated kubernetes resource manifest files from our helm chart templates. So almost any web server will do. For local testing you can also use the helm command itself. Here is a example of helm serving the charts from achartsdirectory.
helm
serve --repo-path ./charts
Problem is you need to get your charts onto the web server somehow, and there is a myriad amount of solutions on how to do it, it can as a example be as simple as just usingscpto get your chart to the web server, or setting up Caddy(https://caddyserver.com/) with the Upload plugin(https://caddyserver.com/docs/upload). You can also use AWS S3 or a Google GCS bucket as well to host a chart repository, which does make uploading easier, by using the google cloud command line utility or the UI or for S3 use one of the many S3 tools out there. I personally prefer s3cmd (http://s3tools.org/s3cmd) for uploading files to S3.
However I wanted something a little simpler, that generates the Helm index for me, without me having to do it by hand, and also be able to host it myself in a Kubernetes cluster, so I wrote a smallGoserver calledHelmetto act as my helm repository. It’s basically just a web server, to which you can upload chart files using something likecurland it then handles the repository indexing for you using helm in the backend.
You can find out more aboutHelmetathttps://github.com/daemonza/helmet
Using a Helm repository
UsingHelmetas our helm repository, let’s deploy it to our Kubernetes cluster and then add a chart. And off course we can deployHelmetwith Helm :D
git
clone
https://github.com/daemonza/helmet.git
;
cd
helmet/helmet-chart
helm package . --
debug
A helmet chart should be created, in my case it ishelmet-chart-0.0.1.tgz
Deploy the same way we deployed the testapi.
helm
install
helmet-chart-0
.0
.1
.tgz
--debug
For this blog post, I deployed everything to Kubernetes Minikube(https://github.com/kubernetes/minikube) So using minikube, let’s see how we can accesshelmet
HELMET=(`kubectl get services | awk
'/helmet/ {print $1}'
`)
minikube service
$HELMET
--url
Which gives me backhttp://192.168.99.100:31162. Using this URL, let’s add it as a helm repository.
helm
repo add helmet
http://192.168.99.100:31162/charts/
And confirm that ourhelmetrepo is there by runninghelm repo list. You should see the following :
NAME
URL
stable
https://kubernetes-charts.storage.googleapis.com/
helmet
http://192.168.99.100:31162/charts/
We can now add our testapi chart to thishelmetrepository with :
curl
-v
-T
testapi-chart-0
.1
.1
.tgz
-X
PUT
http
:
//
192.168
.
99.100
:
31162
/upload/
We can confirm that the chart is uploaded and the helm repo index got created by running
curl
http://192.168.99.100:31162/charts/index.yaml
Which will give us the following output :
apiVersion: v1
entries:
testapi-chart:
- apiVersion: v1
created:
2017
-
03
-
01
T12:
11
:
09.746867088
Z
description: A Helm chart
for
Kubernetes
digest: fe0c17d87b523c91cc59bd1e4d2f997defb2a215c4cc0fc02a1725922471e88a
name: testapi-chart
urls:
- http:
//masked-macaw-helmet-char:1323/charts/testapi-chart-0.1.1.tgz
version:
0.1
.1
generated:
2017
-
03
-
01
T12:
11
:
09.746407601
Z
We can now search for the testapi, across all our repositories.
helm
search testapi
Expected output :
NAME VERSION DESCRIPTION
helmet/testapi-chart
0.1
.1
A Helm chart
for
Kubernetes
Let’s try searching for something else. Searching for Jenkins gives us this
NAME VERSION DESCRIPTION
stable/jenkins
0.
1.14
Open
source
continuous integration
server
. It s...
Which we can see comes from thestablerepository. Very nice! Have I mentioned I love helm? :D Now, to continue let’s install the testapi from ourhelmetrepository :
helm
install helmet/testapi-chart
And that’s how simple it is to use helm repositories. We can now use thishelmethelm repository from anywhere upload charts to and deploy from.
CI / CD
Helm makes continuous integration and deployment a lot easier with Kubernetes. For example one strategy of doing it could be, matching up your git branches to helm repositories. Imagine you are busy building a application, and you have two branches (develop, master), master is your stable branch and anything there could be deployed to a production kubernetes cluster.
Example of your develop and stable helm repositories :
helm
develop add helmet
http://repositoryone:31162/charts/
helm master add helmet
http://repositorythree:31162/charts/
Write your code on a develop branch, and on a git push, Jenkins picks up that there was a change, build your code, run your tests, and create a helm chart and uploads that chart to yourdevelophelm repository. From there you can then easily deploy it into your development Kubernetes cluster (minikube for example). A push to master might trigger Jenkins to deploy the code as a stable release to your master repository and then deploy it to your production kubernetes cluster. Using a simple setup like this it becomes easy to deploy either develop level quality packages or stable packages to kubernetes clusters.
For Jenkins(https://jenkins.io/) you can use the this plugin(https://github.com/jenkinsci/kubernetes-ci-plugin) for easy helm usage.
Conclusion
Helm makes doing reliable reproducible deployments ridiculously easy to Kubernetes, and I cannot recommend enough to make it part of your standard way of working with Kubernetes.