Setting up a Kubernetes cluster on Digital Ocean with SSL

Introduction

It is now possible to create/maintain Kubernetes clusters on Digital Ocean quite easily.
In this post I will show you how to create a cluster with a pod that has a public endpoint; this endpoint will be secured with a Let’s Encrypt SSL certificate.
Some basic Kubernetes knowledge is assumed so i won’t go into detail explaining what a deployment, service or pod is, or how to work with kubectl. N.B. Kubernetes support on Digital Ocean is still in beta, so it maybe not available to you.

Overview

The application we are going to deploy is called ‘whoami’ which has an endpoint that outputs some information about the request that is
sent to it.
We will expose this endpoint through https://whoami.example.com (N.B. example.com is a fake domain name and is used for illustration purposes, replace it with domain you actually own)

In the picture below is the end result is shown:
The parts that are of interest for external access are the ingress, ingress controller and the load balancer.

An ingress is a Kubernetes object defined in our application namespace that defines the rules to which service (or pod) external requests are routed.
In this case we define an ingress with the following rule:


“If an incoming request contains ‘whoami.example.com’ in the host header, the request is routed to the whoami service.”


The actual routing is done by an ingress controller which in most cases uses also a load balancer to route incoming requests.
The controller we will use is Traefik, which also handles SSL termination. This means also that the traffic between the controller and the service/pod is flowing through HTTP.
Traefik also handles the automatic acquiring and renewal of the Let’s Encrypt certificates

Prequisites:

  • You have a Digital Ocean account and created a project in the control panel.
  • You own the domain example.com and you are able to adminster its DNS records through Digital Ocean.
  • You have kubectl installed on your computer.
  • A Digital Ocean API token with read/write access, which is needed later for the configuration for the acquiring of the certificate.

Setting up a cluster in Digital Ocean

Through the Digital Ocean control panel create a cluster with 1 node to keep things simple (and cheap):

  • Choose ‘Kubernetes’ in the left navigation bar.
  • Click ‘Create’ and select ‘Clusters’.
  • Select datacenter AMS3.
  • Under ‘Add node pool’ select from the ‘Standard node pool’ the ‘$5 month plan’ and set the number of nodes to 1.
  • For the name type ‘k8s-test’.
  • Click ‘Create cluster’ and a new cluster will be provisioned.
  • When the cluster is provisioned go to the cluster control panel and click ‘Download’.
  • Copy this file to your ~/.kube directory and name the file ‘config’. This makes kubectl point to the just created cluster.
  • We need to create a default service account with full rights (needed for the Helm package manager and for the Traefik package be able to deploy its stuff to the cluster,
    you probably don’t want to do this in production (at least have a dedicated user). )
  • Paste the following to a file and execute ‘kubectl apply -f filename’ to deploy it to the cluster.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: kube-system

Traefik

To install the Traefik ingress controller:
  • Install Helm, this is a package manager for Kubernetes which makes installation of Traefik very easy.
  • Run ‘helm init’ to initialize Helm on the client and on the cluster.
  • Run ‘helm install –name traefik-thingie -f values.yaml stable/traefik –namespace kube-system’.
    The values.yaml file contain the configuration for Traefik. A more detailed explanation is provided below.
  • Wait until Traefik is deployed. (it creates also a L4 load balancer in Digital Ocean)
  • In the DO control panel look up the IP address of the load balancer just created and create an A record in the Domain section with the whoami.mydomain.com domain pointing to this IP address.

Configuration

Below is the configuration of the Traefik helm chart which is used in this example. For more information about all options see here.
ssl:
  enabled: true        # Enables SSL
  enforced: true       # Redirects HTTP to HTTPS
acme:
  enabled: true             # Enables Let's Encrypt certificates
  staging: true             # Use Lets Encrypt staging area for this example. For production purposes set this to false
  email: fill.in@your.email # Email address that Let's Encrypt uses to notify about certificate expiry etc.
  challengeType: "dns-01"   
  dnsProvider:              
    name:  digitalocean     # This is why you need your domain to be under Digital Ocean control
    digitalocean:
      DO_AUTH_TOKEN: "INSERT_API_TOKEN_HERE"
  domains:
    enabled: true
    domainsList:
      - main: "example.mydomain.com" # Name of the domain that belongs to this certificate
Save this file as ‘values.yaml’ and use it during installation of Traefik. (see above)

Application

The application is deployed on the cluster using a simple service and deployment.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: whoami-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami-container
        image: containous/whoami
----
apiVersion: v1
kind: Service
metadata:
  name: whoami-service
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: whoami
Paste this to a file and use (again) ‘kubectl apply -f filename.yaml’ to deploy it to the cluster.

Ingress

The important part however is the definition of the ingress.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: whoami-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: whoami.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: whoami-service
          servicePort: http
The interesting bits are here:

  • The annotation kubernetes.io/ingress.class should be set to ‘traefik’, so that the Traefik ingress controller is aware of our ingress (remember, an ingress is just a routing rule which the ingress controller should enforce)
  • The ‘host’ field should be set to our domain, and the ‘backend’ should be set to our service defined above.
Paste this to a file and use (again) ‘kubectl apply -f filename.yaml’ to deploy it to the cluster.

Showtime!

If all of the above went well, go to https://whoami.example.com and you should see a page with the content like below (note the X-Forwarded-Port and X-Forwarded-Proto which indicates that the request was coming through https. )
Hostname: whoami-deployment-84dfcf599c-5s9ll
IP: 127.0.0.1
IP: 10.244.16.3
GET / HTTP/1.1
Host: whoami.example.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.244.16.1
X-Forwarded-Host: whoami.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-thingie-74ff44595f-lflrm
X-Real-Ip: 10.244.16.1

Music theory part 1 – Basic Intervals and chords

Recently I started to take on my old hobby of music making and playing the keyboard, so i had also the need to brush up my knowledge on music theory.
These series of posts are mostly a summary for myself of information I found all over the internet. This first post concerns about basic intervals and chords.

Intervals

An interval is basically the distance between 2 notes. These distances or intervals have names and are used as building blocks for chords.
Some of the most common intervals are listed in the table below:

NameDescriptionExamples
Half stepBasic interval.
White key -> Black key
Black key -> White key
(except between B and C and E and F, which are 2 white keys)
C - Db
B - C
Whole step2 half steps
White key -> White key
(except between B and C and E and F)
D - E
E - F#
Major 3rd2 whole stepsF - A
C - E
Minor 3rd1 whole step
1 half step
(or Major 3rd - 1 half step)
F - Ab
C - Eb
Perfect 5th3 whole steps
1 half step
(or Major 3rd + Minor 3rd)
C - G
Db - Ab
Diminished 5th3 whole steps
(or Perfect 5th - 1 half step)
E - Bb
D - Ab
Augmented 5th4 whole steps
(or Perfect 5th + 1 half step)
F# - D
C - Ab
Major 7th5 whole steps
1 half step
(or Octave - 1 half step)
C - B
Ab - G
Minor 7th5 whole steps
(or Major 7th - 1 half step
or Octave - 1 whole step)
C - Bb
F - Eb
Diminshed 7th4 whole steps
1 half step
(or Minor 7th - 1 half step
or Octave - 1 whole step and 1 half step)
C - A
E - Db
Octave6 whole steps
C - C'
F# - F#'

Chords

A chord consists of 3 or more notes separated by particular intervals.

Basic triads

A triad is a chord that consists of 3 notes. See the following table for the basic triads. The composition column indicates which intervals make up the chord from a particular root note. The notation column indicates how the chord is commonly written in non staff notation.

TypeCompositionNotationExample
MajorRoot -> Major 3rd -> Perfect 5thROOTNOTEC
C - E - G

Ab
Ab -> C -> Eb
MinorRoot -> Minor 3rd -> Perfect 5th(ROOTNOTE)mCm
C - Eb - G

F#m
F# - A -Db
DiminishedRoot -> Min 3rd -> Diminished 5th(ROOTNOTE)dimCdim
C - Eb - F#

Edim
E - G - Bb
AugmentedRoot -> Major 3rd -> Augmented 5th(ROOTNOTE)augCaug
C - E - Ab

Bbaug
Bb - D - F#

7th chords

A 7th chord is a 4 note chord consisting of a basic triad plus an extra note which is a 7th interval from the root note.
Not all combinations of triads and 7th intervals are ‘allowed’. See the table below. An X indicates that this combination is not ‘allowed’, all other entries are the chords that represent this combination.

7th interval

Triad
major 7thminor 7thdiminished 7th
majorM77X
minormM7m7X
diminishedXØ7 (m7b5)O7
augmentedaugM7XX

The following table contains the composition of the 7th chords and the table is built up similar as the the one with the basic triads above.

TypeCompositionNotationExample
Major 7thRoot -> Major 3rd -> Perfect 5th - > Major 7th(ROOTNOTE)M7 CM7
C - E - G - B

Ab
Ab - C - Eb - A
Minor major 7thRoot -> Minor 3rd -> Perfect 5th -> Major 7th(ROOTNOTE)mM7CmM7
C - Eb - G - B

F#mM7
F# - A - Db - F
Augmented major 7thRoot -> Major 3rd -> Augmented 5th -> Major 7th(ROOTNOTE)augM7CaugM7
C - E - Ab - B
Major minor 7th
(aka Dominant 7th)
Root -> Major 3rd -> Perfect 5th - > Minor 7th(ROOTNOTE)7C7
C - E - G - Bb

Eb7
Eb - G - Bb - Db
Minor 7thRoot -> Minor 3rd -> Perfect 5th -> Minor 7th(ROOTNOTE)m7Cm7
C - Eb - G - Bb

Am7
A - C - E - G
Half diminished 7thRoot -> Min 3rd -> Diminished 5th -> Minor 7th(ROOTNOTE)Ø7
or
(ROOTNOTE)m7b5
CØ7
C - Eb - F# - Bb

Em7b5
E - G - Bb - D
Full diminished 7thRoot -> Major 3rd -> Diminished 5th -> Diminished 7th(ROOTNOTE)O7CO7
C - E - F# - A

Bb
Bb - D - E - G

 

New screenshots of the FruitHAP Android App

With the upcoming FruitHAP Android App you can control your FruitHAP setup from anywhere in your network.
Here are some screenshots:

 

In the dashboard you can control all switches etc. and see all sensor values in one screen.

 

In the Alerts screen you can see an overview of alerts which are defined in FruitHAP (for example when the temperature in a room is too high, or the doorbell has rang)

Stay tuned for more …

Upcoming new release of FruitHAP

It has been a while since the last version (see here) , but after some months of work a new version is coming soon!

Some of the new features:

  • The FruitHAP Notifier app, with which you can remote control your FruitHAP installation, receive alerts, turn on the lights etc.
  • Added support for my arduino temperature sensor (see here)
  • Added easy deployment of the engine to a target device with an Ansible script
  • More and better documentation (finally…)
  • Bug fixes, improvements

Keep an eye on my twitter account (@traveller1505) for updates on this..

Bike trip @ 12-03-2016

Temperature sensor with Arduino

I wanted to have a cheap temperature sensor to use with my home automation system (FruitHAP).
This sensor must be able to send its temperature through a 433 Mhz RF link since I have already a 433 Mhz RFXCom receiver.
Also a display would be nice to display the temperature so I could put this in a room of my house to monitor the temperature there.
I always wanted to do something with an Arduino, so i thought to create an Arduino project out of this. So I present you my first Arduino project.

Hardware

Parts list

For this project you need:

Since i am too lazy to solder (and i am more a software than a hardware guy) I used components from the Seeed Grove platform
This platform enables you to very quickly cobble together a circuit, it’s literally plug and play.
It consists of a shield which you just click on top of your Arduino and in this shield you can click in various other Grove components.

caption

Assembling the circuit

  • Plug the Base Shield onto the Arduino (check that the alignment is correct)
  • Plug the LCD with a grove cable into one of the I2C ports of the Base Shield
  • Plug the 433MHz transmitter with a grove cablee in port D2 of the Base Shield
  • Put the LED into the LED Socket kit (long leg is the anode/+)
  • Plug the LED socket kit with a grove cable in port D3 of the Base Shield
  • Plug the temperature sensor with a grove cable in port A0 (analog port 0) of the Base Shield

That’s all! The final result is in the pictures below.

OLYMPUS DIGITAL CAMERA OLYMPUS DIGITAL CAMERA OLYMPUS DIGITAL CAMERA

Software

Getting it

Get the code from here and upload it to the Arduino with the help of the IDE.
You need also some external libraries:

When all is correctly connected and set up, the current temperature should be displayed on the LCD (as a bonus the background color changes when the temperature changes, you can test it by putting you finger on the component labeled NTC on the temperature sensor)
The LED should blink every 10 seconds or so (can be altered, see source code), when this happens the current temperature is transmitted through the 433 Mhz transmitter.

Technical details

I won’t discuss all the code, you can view it yourself and in the comments there is also useful info, but i would like to point out some details.

X10

Like mentioned before, the library used to transmit the measurements is X10 since this can emulate a so called RFXSensor device, which is basically
a protocol based off X10.

This protocol can be understood by the RFXCom RFXtrx433 receiver (which i am using already in combination), but you are free to use another receiver/solution. It’s  described in the docs of the X10 library (look for the file x10_RF_formats.pdf and then the section RFXSensor and you can look in the source code of the lib itself).

However, if you use the RFXtrx433 as receiver, pay attention to the fact that it transforms the received information into its own protocol.
Basically the data will be stored in 2 bytes and the temperature is actually stored as temperature * 100 so the range of temperatures to be send is limited.

Possible improvements

Possible improvents could be:
– Put the thingie in a nice box
– Use a combined temperature/humidity sensor, so that you can monitor humidity too!

Hope you like it! Happy coding!

FruitHAP – Part 1

In this series of posts I will describe my own C# based Home Automation platform FruitHAP

What is FruitHAP

FruitHAP is my home-made home automation system, designed to run on a Raspberry Pi.
The name stands for Fruit Home Automation Platform and is a pun on the Raspberry Pi (since you all know that a raspberry is fruit). Also ‘Fruithap(je)’ in Dutch is the word for a plate of mashed fruit you give to a baby; something like the picture below.

Logo

Why did you make your own system, since there are already lot of other HA solutions

I have a house with an attic and since this my ‘man-cave’ i sit/work there a lot. The only problem is that I often can’t hear the doorbell ring, especially if the washing machine is also turned on (it’s on the attic as well and it’s noisy).

So I came up with an idea to make something that delivers a notification to my cellphone when someone rings the frontdoor bell.
Oh, and maybe it’s also convenient to put an image of the person standing there in the notification, so I can decide if it’s worthwhile to go downstairs and answer the door.

Then I bought a Raspberry PI, a wireless doorbell, a camera and some other stuff. Now I needed some software to make this setup work. I looked first at some existing HA platforms but decided that they didn’t put up to my expectations, so i decided to build my own. (it is a lot more fun too 🙂 )

Requirements

I came up with some requirements for my platform:

  • Extensible
    I heard that automating your house is addicting. It shoulde be easy to add different sensors and sensor types, since there are a lot of standards of sensors out there.
  • Accessibilty of sensor data
    It should be easy to access sensor data from other systems/apps. Maybe you want more than only notifications on your phone, maybe you want to have a dashboard with all your sensor data or log
    your energy consumption to a database to perform some analysis on it (BIG data is hot these days)
  • Should run on a Raspberry Pi
  • Open source

General overview of the platform

A high level overview of the platform is in the diagram below. I will explain it a bit in more detail.

fruithap_overview

Engine

The main part of the system is called the Engine. This is a daemon that runs on the Raspberry Pi and acts as the gateway between the actual physical sensors and the ‘outside’ world.
The engine communicates with the outside world by means of a message queue, so external applications can easily interact with the system by reading/writing into this queue, through a publish/subscribe mechanism or
request/response mechanism. (RPC calls)

For example, when the doorbell rings, the event goes from the doorbell through the engine and finally got published in the message queue. An app running on a phone can listen (subscribe) to this event on the queue and send a
notification when this event is fired.

An example of request/response is when an external application wants to read the temperature of a certain sensor. It sends a request to the message queue which the engine handles by retrieving the value from the sensor concerned
and sends it back (through the queue) to the external application.

The idea is that all the configuration of the engine and the other components will also be exposed in this way to make it easier to make a nice web interface for configuring everything, but that’s another story for another time.

Controllers

The ‘lowest’ layer is the controller layer. This layer handles all nitty-gritty low level hardware protocol stuff.
It communicates directly with the sensor hardware or through another hardware controller, such as a zWave or a 433 mhz controller.

Sensors

The sensor layer is the most important layer and is considered the core of the platform. These contain virtual representations of the physical sensors/actuators and use the controllers to communicate with the actual sensors
so that all hardware/protocol specifics are abstracted away (only the configuration for a sensor exposes some of the specifics).

For example one of the actuators supported is a KaKu button. KaKu stands for KlikAanKlikUit and consists a whole range of actuators/sensors/device with a specific communication protocol. Each device has an unique device ID and some more specific settings.
These have to be set only in the sensor configuration with an unique name and for the rest of the system and the outside world it is known as a button with that particular name, which can be ‘pressed’.

Sounds difficult? No worries, in the next post when I will describe my setup it will all come together.

Also it’s possible to have ‘aggregate’ sensors, a sensor that combines the input 2 or more separate physical sensors in a way and use this as one value. This is comparable with some of the Android ‘logical sensors’.
An example is a ButtonCamera, it combines a Button and a Camera, so when the button is pressed an image from the camera is retrieved and is sent along with the button pressed event as if it was one physical sensor.

Actions

An action is actually the interface between the sensor(s) and the message queue. Its input(s) is/are coming from either the sensors or from the outside (messages from the queue) and its
outputs are requests for sensors or other parts of the system (configuration!) or messages that are bound for the queue. Basically is this a rule engine where you can implement all kinds of scenarios for your home automation needs.
A standard action is for example to send a message to a queue every time a sensor updates. These actions can also have specific configuration.

Phew.. that was a lot of reading

This is just a short description of the system. If you have any remarks or comments about this post please let me know.
Next time I will show you an example of how this thing works. The example is actually my live doorbell setup.
When the doorbell rings FruitHAP is picking this up and a notification with a image from a camera is sent through Pushbullet to my phone.
More details about this in the next post..

See you next time!

Since FruitHap is open source, here is a link to the source.
If you can’t wait to try this thing (and you have the right equipment) you can also find here a quick setup with a complete Raspberry Pi image.

A blinking light with Raphaël.js

For my Freeboard based dashboard for my home automation system (that’s a thing i will talk about in another post 🙂 ) i was looking for a fancy blinking LED style light. Basically i wanted it to have the following requirements:

  • Possibility to turn on/turn off the light.
  • Possibility to start/stop blinking
  • Possibility to carry out some action when you click on the light
  • It has to look cool.

During my search for something like this i came along Raphael.js and i decided to build my own blinking led with this framework, just to get it to know better. I think it is a very cool framework with very cool UI possibilities.
So i created a a small javascript library, which i called Blinkenlight that implements all requirements. You can find the code and an example here.

Example usage

Preparation

Put the downloaded .js files somewhere on your file system, preferably some place from which you can reference to it from your html file.

HTML

In your html file add the following script tags in the head section:
<script type=text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/raphael/2.1.4/raphael-min.js">
<script type="text/javascript" src="http://code.jquery.com/jquery-2.1.4.js"></script>
<script type="text/javascript" src="path_to_js/blinkenlight.js"></script>

The first line adds the Raphaël.js library, the second line adds the JQuery library, which i explain later and the third line adds the Blinkenlight library.

Raphaël.js works with a canvas or paper on which we are going to draw our light. This canvas has to be put in some DOM element on your page.
So let’s add this element to the body section of the html page (you can use your own id here):
<div id="canvas_container"></div>

Initialization function

In a new script tag in the head section of the html file add the following:

<script type="text/javascript">
$(document).ready(function ()
{
});

We will initialize our canvas and draw our light and all other things in this function which comes with JQuery and this is the reason why we needed the JQuery library. It’s possible to use the window.onload() or the document.onload()  functions but i didn’t test it. (according to this it’s better to use the JQuery function because of possible browser incompatibilities)

Initialization of Raphael.js

To initialize Raphael.js add the two following lines in the $(document).ready function:

var element = document.getElementById('canvas_container');
var paper = new Raphael(element, 200, 80);

We retrieve our canvas DOM element (somehow this doesn’t work with a JQuery $("#canvas_container"), the page layout gets fucked up) and pass it to the Raphael.js constructor function. The two other parameters of this function are the width and height of the canvas in pixels, in the above case, 200 px wide and 80 px high.
This function returns a canvas which we use for the initialization of the blinkenlight.

Blinkenlight initialization

To initialize a Blinkenlight use the constructor function:

function BlinkenLight(paper, x, y, radius, colorOn, colorOff, glowColor, tooltipText, clickHandler)

Description of parameters:

paper
The Raphael canvas where the light is to be drawn.

x
X-coordinate on the canvas. (0 is left)

y
Y-coordinate on the canvas. (0 is top)

radius
Radius of the light in pixels

colorOn
Color of the light when it is in the ‘on’ state

colorOff
Color of the light when it is in the ‘off’ state

glowColor
Color of the glow around the light.

tooltipText
(Optional) Tooltip text which is displayed when a click handler is assigned

clickHandler
(Optional) Javascript that is executed when you click on the light

Example 1, without click handler:

var blinker1 = new BlinkenLight(paper, 100, 40, 20, "#00DDDD", "#004444", "#00FFFF");

Example 2, with click handler and tooltip text

var blinker2 = new BlinkenLight(paper, 40, 40, 20, "#00DD00", "#004400", "#00FF00", "Click me", function ()
{
alert('You clicked me!')
});

Initially the lights will be ‘off’, so the result of the above will be look like:

blinkenlights_off

Changing state of the light

To turn on the light, call the turnOn() function:

blinker1.turnOn();
blinker2.turnOn();

blinkenlight_on

To turn off the light, call the turnOff() function:

blinker1.turnOff();
blinker2.turnOff();

blinkenlights_off

Blinking

First call the setInterval() function with a parameters which sets the blinking interval in ms:

blinker1.setInterval(500);

Then call the start() function to start blinking:

blinker1.start();

To stop the blinking call the  stop() function:

blinker1.stop();

Conclusion

I hope you get an idea how this thing works. The example provided with the code shows all the concepts plus an example of a click handler that shows a image in an popup.
If you have any comments, tips or tricks please leave them in the comments below.What’s next

In a following post i will use this library to make an actual widget plugin for Freeboard.

 

Happy coding and until next time!

Bike trip @ 24-03-2012

Bike trip @ 08-03-2015