Setup a Kubernetes Cluster on Azure

Photo by Joseph Barrientos on Unsplash

I finally received my copy of Kubernetes: Up and Running by Kelsey Hightower (put on pre-order more than a year ago!), and want to set up a Kubernetes cluster to experiment with.  I'm going to use Microsoft's Azure Container Service, but I want to set this up programmatically.  This will let me set up and tear down the cluster whenever I want to, and make sure that it's configuration doesn't drift over time.  This gives me confidence that I can recreate the cluster in exactly the same way later on, if I need to - if I want to move to a different region or resource group or whatever.  This also fits with the DevOps Infrastructure As Code philosophy, allowing you to check your resource scripts into source control and treat them the same as the code that's running on them.

There are a couple of prerequisites I need to set up before I get started.  Firstly - I've just talked about versioning the setup, so I need to set up source control.  Making a directory for this project and running

> git init

will create a local repo which I can push to a public server later on.

Next, OpenSSH so I can generate SSH keys.  If you're using Windows, the easiest way to install it is via Chocolatey.  Run this command in an Admin console:

choco install openssh

This will install OpenSSH but your path won't be set up - open a new console and generate an SSH key by running:

> ssh-keygen -t rsa -b 2048 -f k8s_id_rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in k8s_id_rsa.
Your public key has been saved in k8s_id_rsa.pub.
The key fingerprint is:
SHA256:oC6Wu+QP1C1YZg4jCOyjI0hlYtZApYrxvpaNaNLQ2gM adam@Minerva
The key's randomart image is:
+---[RSA 2048]----+
|+o+.             |
|o=.+             |
|B.* + .          |
|+* X o .         |
|*.= = . S        |
|Eo.o .           |
|.OB+.            |
|+=B=.            |
|o.=+.            |
+----[SHA256]-----+

You can press enter to set no password for the key - makes it easier later on.  This will create a public (k8s_id_rsa.pub) and private (k8s_id_rsa) key in the current directory - you'll need these shortly. 

We want to make sure the keys are not checked in to Git - we will use the keys to log in to Azure VMs soon, so use .gitignore to make sure they are not checked in accidentally:

 > ls > .gitignore

Run a git status command now and you'll see that the key files are ignored, and won't be checked in.

Finally, the Azure CLI.  This is an easy to use multi-platform Command Line Interface that can configure and manage Azure services.  It supports Linux, Mac and Windows and has a purpose-built syntax unlike PowerShell which is very powerful but can be quite confusing.  The installation instructions can be found here.

Once you've installed the CLI, you can log in to your account and get started:

> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code GENXXQH63 to authenticate.

This prompts you to go to a web page and enter a token, which is then used to associate your session with your Azure login.  The command line can now do everything that you can through the UI.  For example - you can create a resource group for your Kubernetes install:

> az group create --name k8s-rg --location australiaeast
{
  "id": "/subscriptions/XXX/resourceGroups/k8s-rg",
  "location": "australiaeast",
  "managedBy": null,
  "name": "k8s-rg",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null
}

The az commands give you back a handy JSON result of each operation that can be read to get the IDs of objects you have created or looked up.  This can be really useful when you're writing scripts, as we'll see later.

Now I want to register a service principal in my Azure account for Kubernetes.  This will create a "principal" or system account that can be granted permissions.  This is the way programmes or services in Azure are given rights to access databases, or to create more resources.

Then create the service principal:

> az ad sp create-for-rbac --name "Kubernetes" ↩
  --role Contributor ↩
  --scopes "/subscriptions/XXX/resourceGroups/k8s-rg"
Certificate expires 2018-10-23 07:03:48+00:00. Adjusting SP end date to match.
Retrying role assignment creation: 1/36
Retrying role assignment creation: 2/36
{
  "appId": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
  "displayName": "Kubernetes",
  "name": "http://Kubernetes",
  "password": "SomeRandomPassword",
  "tenant": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
}

Set the scope parameter to the ID of the resource group you created earlier - this means that the new service principal gets Contributor rights to the resource group but nothing else - so you can't use that account to add or delete anything outside the Kubernetes group.  You might get a number of "Retrying role assignment" messages - it seems like it takes some time to create the service principal and the role assignment fails until the create operation has completed.

Next you can create the Kubernetes cluster itself - this takes some time:

> az acs create --name k8s ↩
  --resource-group k8s-rg ↩
  --admin-username azureUser ↩
  --agent-count 3 ↩
  --dns-prefix fantail-k8s ↩
  --location australiaeast ↩
  --master-count 1 ↩
  --agent-vm-size Standard_D2_v2 ↩
  --ssh-key-value ./k8s_id_rsa.pub ↩
  --orchestrator-type Kubernetes ↩
  --service-principal http://Kubernetes ↩
  --client-secret SomeRandomPassword
{
  "properties": {
    "outputs": {
      "masterFQDN": {
        "type": "String",
        "value": "fantail-k8smgmt.australiaeast.cloudapp.azure.com"
      },
      "sshMaster0": {
        "type": "String",
        "value": "ssh azureUser@fantail-k8smgmt.australiaeast.cloudapp.azure.com -A -p 22"
      }
    },
    "provisioningState": "Succeeded",
    "timestamp": "2017-10-23T09:52:10.978859+00:00"
  },
  "resourceGroup": "k8s-rg"
}

I have removed a bunch of lines of the output - but kept the important ones.  The server's master domain name and the command I need to use to SSH to it.  You can try it out:

> ssh azureUser@fantail-k8smgmt.australiaeast.cloudapp.azure.com -A -p 22 -i ./k8s_id_rsa
The authenticity of host 'fantail-k8smgmt.australiaeast.cloudapp.azure.com (13.75.140.89)' can't be established.
ECDSA key fingerprint is SHA256:liSzrEk5++Dh2bXzDiS3Y9ZJPYxr/pPhKQd4zIwosQ0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'fantail-k8smgmt.australiaeast.cloudapp.azure.com,13.75.140.89' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-97-generic x86_64)
...
azureUser@k8s-master-313A5565-0:~$

You can also install the Kubernetes client - kubectl - and configure it for your new cluster:

> az acs kubernetes install-cli
Downloading client to /usr/local/bin/kubectl from https://storage.googleapis.com/kubernetes-release/release/v1.8.1/bin/darwin/amd64/kubectl 
> az acs kubernetes get-credentials ↩
  --ssh-key-file ./k8s_id_rsa ↩
  --resource-group k8s-rg ↩
  --name k8s

This last command has no output, but it copies the Kubernetes configuration - keys, certs and domain names into a .kube directory in your home (~/.kube on a Mac).  You can try out kubectl now:

> kubectl get nodes
NAME                    STATUS    ROLES     AGE       VERSION
k8s-agent-313a5565-0    Ready     agent     45m       v1.7.7
k8s-agent-313a5565-1    Ready     agent     45m       v1.7.7
k8s-agent-313a5565-2    Ready     agent     44m       v1.7.7
k8s-master-313a5565-0   Ready     master    44m       v1.7.7

Mission accomplished - Kubernetes is up and running! 

So why did I set up Git in the beginning?  The commands above can be put into scripts to make it easier to setup clusters in future - in a CI/CD pipeline, for example.  I don't always want to generate a new key, so I broke the setup into two steps, creating a key then creating the cluster:




Comments

Post a Comment

Popular posts from this blog

Feature Toggles on a .Net Core API

This is my second tutorial on feature toggling.  You can read the first here . There are many languages and frameworks to choose from when it comes to back ends - I wanted to make a REST service to deliver all of the Time Entries to my Angular application, and almost every language available has REST server capabilities.  I chose to use .Net Core (C#) and Java for my example app.  My background is in Java, and I was really interested to see how .Net Core works.  If you don't know, .Net Core is Microsoft's cross platform offering - it has the Windows-specific parts taken out and runs on Docker. Linux, MacOSX, and of course Windows too.  It can also be compiled on other platforms too, which is very handy. Today I'll be setting up LaunchDarkly using the .Net libraries - I'll do the Java version later on.  LaunchDarkly provides some samples  to help you get started, but I wanted to change a few things.  Just like I did in the Angular front end, I ...
Keep reading

Automatically Logging PowerShell in to Azure

Image
I've been working with Azure Resource Manager templates a lot lately, as an easy, repeatable way to create deployment environments for my code.  ARM templates are a JSON description of a resource or grouping of resources and can be applied to create or update an environment.  This means you can roll out a new environment quickly and easily - perfect for quickly setting up a Dev or Test system, and you know that if you use the same templates to configure Production you won't have any surprises, or "Well, it worked in Dev..." conversations. The best thing about ARM templates for me is how easy it is to generate them.  If you have a resource group, you can create a template by clicking on the Automation Script button: This will create a show you a JSON template and supporting files that can be downloaded as a Zip.  It contains all the information needed to reproduce your resource group's infrastructure.  It isn't a backup though - it doesn't have the we...
Keep reading