• Cloud provider
    • ➟ Terraform basics

Quote:

“Terraform is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently.”

Declarative schema Physical result 
CREATE SCHEMA ...
  CREATE TABLE ...

  CREATE VIEW ...

...

  • Database implementation using files or raw devices.

  • Creating Index / search trees.
Declarative schema Physical result 
resource "hcloud_server" "wwwSrv" {
  name         = "hello"
  image        =  "debian-13"
  server_type  =  "cx23"
}
 Creating IPs, networks, servers, message queues, ...

  • Access you cloud project by the Hetzner GUI interface.

  • Go to Security --> API Tokens --> Generate API token

  • Provide a name and hit “Generate API token”.

  • Copy the generated token's value and store it in a secure location e.g. a password manager.

    Caution

    The Hetzner GUI blocks subsequent read access after creation.

    In case of loss: Delete the existing and generate a new one.

  • Cloud provider
    • ➟ Terraform basics
      • ➟ Working with Terraform
        • ➟ Creating a server 
# Define Hetzner cloud provider
terraform {
  required_providers {
    hcloud = {
      source = "hetznercloud/hcloud"
    }
  }
  required_version = ">= 0.13"
}

# Configure the Hetzner Cloud API token
provider "hcloud" {
  token = "your_api_token_goes_here"
}

# Create a server
resource "hcloud_server" "mySrv" {
  name         = "hello"
  image        =  "debian-13"
  server_type  =  "cx23"   
}
$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hetznercloud/hcloud...
- Installing hetznercloud/hcloud v1.46.1...
- Installed hetznercloud/hcloud v1.46.1 (signed by a HashiCorp partner, key ID 5219EACB3A77198B)

...
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above.  ...
$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions ...
  + create

Terraform will perform the following actions:

  # hcloud_server.mySrv will be created
  + resource "hcloud_server" "mySrv" {
      + allow_deprecated_images    = false
      + backup_window              = (known after apply)
  ...
    }

Plan: 1 to add, 0 to change, 0 to destroy.
terraform apply
...
Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

hcloud_server.mySrv: Creating...
hcloud_server.mySrv: Still creating... [10s elapsed]
hcloud_server.mySrv: Creation complete after 14s [id=45822789]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Image layer 1
Image layer 2
Image layer 3
Image layer 4
Image layer 5
Image layer 6
Resource list hcloud_server.mySrv's resource properties 
$ terraform state list
hcloud_firewall.allowSsh
hcloud_server.mySrv
hcloud_ssh_key.authorizedUsers["joe"]    
hcloud_ssh_key.authorizedUsers["eve"]
 
$ terraform state show hcloud_server.mySrv
resource "hcloud_server" "mySrv" {
    firewall_ids               = [11889577,]
    id                         = "166290307"
    image                      = "debian-13"
    ipv4_address               = "2.29.10.236"
    ipv6_address               = "2a01:4f9:c015:5ae7::1"
    ipv6_network               = "2a01:4f9:c015:5ae7::/64"    
    location                   = "hel1"
    name                       = "hello"
    primary_disk_size          = 40
    server_type                = "cx23"
      ...
}
  1. Your first server by Terraform.
  • Cloud provider
    • ➟ Terraform basics 
Your server "hello" was created!

You can access your server with the following credentials:
 
IPv4	128.140.108.60
IPv6	2a01:4f8:1c1c:8e3a::/64
User	root
Password	rJ3pNvJXbqMp3XNTvFdq

You will be prompted to change your password on your first login.

To improve security, we recommend that you add an SSH key when creating a server.

  • No protection by any firewall

  • IP and (initial) credentials by email 😱

Solution:

  1. Add firewall with inbound ssh access rule avoiding: 

    $ ssh root@128.140.108.60
ssh: connect to host 128.140.108.60 port 22: Connection refused

  2. Replace password by public ssh key based authentication. 

# main.tf

resource "hcloud_firewall" "sshFw" {
  name = "ssh-firewall"
  rule {
    description = "Allow incoming ssh connections"
    direction   = "in"
    protocol    = "tcp"
    port        = "22"
    source_ips  = ["0.0.0.0/0", "::/0"]
  }
}
              ...
resource "hcloud_server" "mySrv" {         ...
  firewall_ids = [hcloud_firewall.sshFw.id]
}
# main.tf

resource "hcloud_ssh_key" "loginUserKey" {
  name       = "goik@hdm-stuttgart.de"
  public_key = file("~/.ssh/id_ed25519.pub")
}
              ...
resource "hcloud_server" "mySrv" {
              ...
  ssh_keys     = [hcloud_ssh_key.loginUserKey.id]

}

Note: Use the Hetzner Web GUI for removing any conflicting manually installed ssh keys beforehand. 

$ terraform apply

  # hcloud_firewall.sshFw will be created
  + resource "hcloud_firewall" "sshFw" {
       ...
  # hcloud_server.mySrv will be created
  + resource "hcloud_server" "mySrv" {
       ...
  # hcloud_ssh_key.goik will be created
  + resource "hcloud_ssh_key" "loginUserKey" {
       ...
Plan: 3 to add, 0 to change, 0 to destroy.
       ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
resource "hcloud_firewall" "sshFw" {...}
              ...
resource "hcloud_server" "mySrv" {     ...
  firewall_ids = [hcloud_firewall.sshFw.id]  }
"resources": [
    { "type": "hcloud_firewall",
      "name": "sshFw",
      "provider": "provider[\"registry.terraform.io/hetznercloud/hcloud\"]",
      "instances": [
        {"schema_version": 0,
         "attributes": {
            "id": "11941059",
            ...
  • Cloud provider
    • ➟ Terraform basics
      • ➟ Working with Terraform
        • ➟ Terraform state and Outputs

See terraform output documentation: 

# outputs.tf
output "helloIpAddress" {
  value       = hcloud_server.mySrv.ipv4_address
  description = "The server's IPv4 address"
}

output "helloLocation" {
  value       = hcloud_server.mySrv.location
  description = "The server's datacenter location"
}
 
$ terraform output
helloLocation  = "nbg1"
helloIpAddress = "159.69.152.37"
 
$ terraform output helloIpAddress
"159.69.152.37"
 
# outputs.tf

output "helloIpAddress" {
  value       = hcloud_server.mySrv.ipv4_address
  description = "The server's IPv4 address"
}

output "helloLocation" {
  value       = hcloud_server.mySrv.location
  description = "The server's datacenter location"
}

terraform output -json

{
  "helloLocation": {
    "sensitive": false,
    "type": "string",
    "value": "nbg1"
  },
  "helloIpAddress": {
    "sensitive": false,
    "type": "string",
    "value": "159.69.152.37"
  }
}
  1. Adding ssh public keys and firewall
  • Cloud provider
    • ➟ Terraform basics
      • ➟ Working with Terraform
        • ➟ Using variables 
# config.auto.tfvars
# Overriding default value "hello"
serverName = "testServer"
# variables.tf
variable "serverName" {
   description = "Project unique server name"     
   type        = string
   nullable    = false
   default     = "hello"
}
 
# main.tf
resource "hcloud_server" "mySrv" {
  name = var.serverName
  ...
}
terraform apply

...
serverProps = {
  "ip"   = "46.62.254.219"
  "name" = "testServer"
}

For security reasons, secrets should not be under version control: 

# provider.tf
provider "hcloud" { token = "xdaGfz9LmwO8SWkg ... "}
...
# variables.tf
variable "hcloud_token" {
   description = "Hetzner API token"
   type        = string
   nullable    = false
   sensitive   = true
}
# .env file
# Setting environment variable
export TF_VAR_hcloud_token="utRw..."
 
# main.tf
resource "hcloud_server" "mySrv" {
  name = var.serverName
      ...
}
. .env # Source environment file

echo $TF_VAR_hcloud_token
utRw...

terraform apply
...
  1. Using configuration variables
  • Cloud provider
    • ➟ Terraform basics
      • ➟ Loops 
# main.tf
resource "hcloud_server" "mySrv" {
  count     = 2
  name      = "www-${count.index}"
  ...
}
# outputs.tf
output "servers" {
  value = join(", ",
    [
      for srv in hcloud_server.mySrv :
         "(${srv.name}|${srv.ipv4_address})"     
    ])
}
 
terraform apply
...
Plan: 5 to add, 0 to change, 0 to destroy.

Outputs:                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                     
servers = "(www-0|46.62.254.219), (www-1|204.168.245.188)"
  1. Creating a set of servers
  • Cloud provider
    • ➟ Terraform basics
      • ➟ Using Terraform Maps 
# variables.tf
variable "ipByName" {
  type      = map(string)
  description = "IP's by server names"
}
# config.auto.tfvars
ipByName = { # key/value pairs
  mail = "192.178.1.3"
  database = "192.168.85.9"
}
 
#main.tf

terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "2.7.0"
    }
  }
}
resource "local_file" "serverConfigs" {
  for_each = var.ipByName
  filename = "Generate/${each.key}.cfg"
  content  = "ipv4=${each.value}"
  file_permission = 0744
}
 
> terraform apply
...
local_file.serverConfigs["mail"]: Creation complete ...
local_file.serverConfigs["database"]: Creation complete ...

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

> ls Generate
database.cfg  mail.cfg

cat Generate/database.cfg 
ipv4=192.168.85.9
  1. Multiple ssh keys access
  • Cloud provider
    • ➟ Terraform basics
      • ➟ Documentation 
...
├── .gitignore
├── env.template
├── Conf
│   └── config.tf.template
├── 030HelloSshPublicKey
│   ├── main.tf
│   ├── providers.tf
│   ├── Readme.md
│   └── variables.tf
├── 040...
...
└── Modules
    ├── HostMetaDat
   ...
    └── SshKnownHosts
   ...
export TF_VAR_hcloud_token=add_your_provider_token
export TF_VAR_dns_secret=add_your_ds_server_secret

  • env.template is to be copied to e.g. an env file.

  • Required credentials to be filled in.

  • Prior to executing e.g. terraform init the env file has to be sourced: 

    . ../env                  # Sourcing environment
echo $TF_VAR_hcloud_token # For testing purpose only
z0Tgf...
terraform init
...

  • In contrast to env.template the env file is being listed in .gitignore due to sensible content. 

# variables.tf
variable "hcloud_token" {  # See env file
  nullable = false
  sensitive = true
}
# env file
TF_VAR_hcloud_token="z0Tgf ... "
# providers.tf
...
provider "hcloud" { token = var.hcloud_token }
output "authorizedKeys" {
  description = "List of authorized users' public ssh keys"
  type        = map(string)

  value = {
    goik  = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKY24AeEibQGSMPtstaN4REByLCM3kzjM//apEZ9WyUB goik@hdm-stuttgart.de"
    xy001 = "KEY_OF_USER_xy001 xy001@hdm-stuttgart.de"
    xy002 = ...
  }
}

output "loginUserName" {
  description = "Login user's name with admin privileges"
  type        = string

  value = "devops"
}
env
Conf/config.tf

  • Problems:

    1. Multiple users and local Terraform state.

    2. Disjoint working environments, i.e. company and home office.

  • Solution: Move Terraform state to shared backend.

  • How to Manage Terraform State with GitLab

  • Cloud provider
    • ➟ Cloud-init

  • Initial system configuration

  • Standardized installation related tasks:

    packages, users, ssh keys, networks, hostnames, custom scripts

  • Individual CRUD file operations

  • Adding users

  • Supplying ssh user and host keys.

  • System hardening: Firewall rules, security policies, logging / monitoring

  • Installing packages

  • Enabling and starting services

  • System upgrade + reboot

  • Arbitrary command execution

 
# main.tf

resource "hcloud_server" "nginxServer" {           
  name         = "hello"
  image        = "debian-13"
  server_type  = "cx23"
  ssh_keys     = ...

  # Post server creation init script
  user_data    = file("myinit.sh")
}
#!/bin/bash

# myinit.sh

aptitude update && aptitude -y upgrade     
├── main.tf
├── ... 
└── myinit.sh
  1. Automatic Nginx installation

  • Server post-creation install steps

  • YAML based syntax 

#cloud-config
package_reboot_if_required: true
package_update: true
package_upgrade: true

packages:
  - nginx

runcmd:
  - systemctl enable nginx
  - systemctl start nginx
  - rm -rf /var/www/html/*
  - >
    echo "I'm Nginx @ $(curl -4 https://ifconfig.me) 
    created $(date -u)" > /var/www/html/index.html
  1. Cloud-init for starters
# main.tf
resource "hcloud_server" "web" {
  name         = var.server_name
       ...
  user_data = file("userData.yml")
}
#cloud-config

# userData.yml

packages:
  - nginx
runcmd:
  - systemctl enable nginx
  - rm /var/www/html/*
  - >
    echo "I'm Nginx @ $(curl -4 https://ifconfig.me) 
    created $(date -u)" > /var/www/html/index.html
 
├── main.tf
├── ... 
└── userData.yml
  1. Cloud-init by Terraform
# main.tf
resource "local_file" "user_data" {
  content         = templatefile("tpl/userData.yml", {
  loginUser              = "devops"
                    })
  filename        = "gen/userData.yml"
}

resource "hcloud_server" "mySrv" {
  ...
  # Using gen/userData.yml
  user_data    = local_file.user_data.content
}
 
# tpl/userData.yml
...
users:
  - name: ${loginUser}
    ...
 
# gen/userData.yml
...
users:
  - name: devops
     ...
 
resource "tls_private_key" "host" {
  algorithm   = "ED25519"
}
...
resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {
     host_ed25519_private =
       tls_private_key.host.private_key_openssh
     host_ed25519_public  = ...
                })
  filename = "gen/user_data.yml"
}
 
...
ssh_keys:
  ed25519_private: |
    -----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUA...
-----END OPENSSH PRIVATE KEY-----
 
# E1: File gen/user_data.yml is not valid YAML.
  in "<unicode string>", line 19, column 1:
    b3BlbnNzaC1rZXktdjEAAAAABG5vbmUA ...
 
# main.tf
resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {
     host_ed25519_private = 
       indent(4, tls....host.private_key_openssh)
     ...})
  filename = "gen/user_data.yml"
}
 
# gen/usr_data.yml
ssh_keys:
  ed25519_private: |
␣␣␣␣-----BEGIN OPENSSH PRIVATE KEY-----
␣␣␣␣b3BlbnNzaC1 ...
␣␣␣␣...
␣␣␣␣...
␣␣␣␣-----END OPENSSH PRIVATE KEY-----
 
# gen/usr_data.yml

ssh_keys:
  ed25519_private: |
␣␣␣␣-----BEGIN OPENSSH PRIVATE KEY-----
␣␣␣␣␣␣b3BlbnNzaC1 ...
␣␣␣␣␣␣...
␣␣␣␣␣␣...
␣␣␣␣␣␣-----END OPENSSH PRIVATE KEY-----
 
# /etc/ssh/ssh_host_ed25519_key
...
ssh_keys:
  ed25519_private: |
-----BEGIN OPENSSH PRIVATE KEY-----
␣␣b3BlbnNzaC1 ...
␣␣...
␣␣...
␣␣-----END OPENSSH PRIVATE KEY-----
 
resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {})
  filename = "gen/user_data.yml"
}
#> cloud-init schema --config-file  gen/user_data.yml
root@hello:~# journalctl -f
May 06 04:41:20 hello cloud-init[898]: Cloud-init v. 22.4.2 finished at Mon, 06 May 2024 04:41:20 +0000. Datasource DataSourceHetzner.  Up 11.78 seconds
   ...
May 06 04:46:16 hello sshd[927]: Invalid user abc from 43.163.218.130 port 33408
May 06 04:46:17 hello sshd[927]: Received disconnect from 43.163.218.130 port 33408:11: Bye Bye [preauth]
May 06 04:46:17 hello sshd[927]: Disconnected from invalid user abc 43.163.218.130 port 33408 [preauth]
   ...
May 06 04:50:54 hello sshd[930]: fatal: Timeout before authentication for 27.128.243.225 port 59866
   ...
May 06 04:52:45 hello sshd[933]: Invalid user cos from 43.163.218.130 port 59776
   ...
May 06 04:53:04 hello sshd[935]: Invalid user admin from 194.169.175.35 port 51128
May 06 04:53:49 hello sshd[937]: User root from 43.163.218.130 not allowed because not listed in AllowUsers
May 06 04:53:49 hello sshd[937]: Disconnected from invalid user root 43.163.218.130 port 50592 [preauth]
  1. Working on Cloud-init

Problem of repeated terraform apply: 

$ ssh root@128.140.108.60
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
resource "local_file" "known_hosts" {
  content         = join(" "
                        ,[ hcloud_server.mySrv.ipv4_address
                        , tls_private_key.host.public_key_openssh ]
                    )
  filename        = "gen/known_hosts"
  file_permission = "644"
}
157.180.68.60 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5...XXgAC3eaueHMQ8ePtbnw
resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {
     host_ed25519_private = tls_private_key.host.private_key_openssh
     host_ed25519_public  = tls_private_key.host.public_key_openssh
     devopsUsername       = hcloud_ssh_key.loginUser.name
               ...
  filename = "gen/user_data.yml" ...
#cloud-config
${yamlencode(
  { ...
    ssh_keys = {
      ed25519_private = host_ed25519_private
      ed25519_public  = host_ed25519_public
    } ...
  }
)}
users:
  - name: ${devopsUsername} ...
main.tf tpl/ssh.sh 
resource "local_file" "ssh_script" {
  content = templatefile("tpl/ssh.sh", {
    ip=hcloud_server.hello.ipv4_address
  })
  filename        = "bin/ssh"
  file_permission = "755"
  depends_on      = [local_file.known_hosts]
}
 
#!/usr/bin/env bash

GEN_DIR=$(dirname "$0")/../gen

ssh -o UserKnownHostsFile= \
  "$GEN_DIR/known_hosts" devops@${ip} "$@"
  1. Solving the ~/.ssh/known_hosts quirk
#cloud-config

chpasswd:
  list: |
    root:bingo9wingo
  expire: False
#cloud-config

write_files:
  - content:
        ${base64encode(private_key_pem)}
    encoding: base64
    path:  /etc/nginx/snippets/cert/private.pem

Note

Congrats to paultyng

  • Cloud provider
    • ➟ Selected Terraform topics
      • ➟ Terraform modules 
# main.tf
resource "hcloud_server" "myServer" { ... }

resource "local_file" "hostdata" {
  content = templatefile("Tpl/hostdata.json", {
    ip4      = hcloud_server.mySrv.ipv4_address
    ip6      = hcloud_server.mySrv.ipv6_address
    location = hcloud_server.mySrv.location
  })
  filename   = "Gen/hostdata.json"
}
{ "network": {
     "ipv4": "${ip4}","ipv6": "${ip6}"},
  "location": "${location}"}
 
{
"network": {
  "ipv4": "157.180.78.16",
  "ipv6": "2a01:4f9:c013:be69::1"
  },
  "location": "hel1"
}
 
├── CreateHostByModule
│   ├── main.tf
│   ├── variables.tf
│   └── Gen
|        └── hostdata.json ◀━━┓
└── Modules                    ┃
    └── HostMetaData           ┃
        ├── main.tf            ┃
        ├── outputs.tf         ┃
        ├── providers.tf       ┃
        ├── Tpl                ┃
        │   └── hostdata.json ━┛
        └── variables.tf
# variable.tf
variable "hcloud_token" { sensitive = true }
# main.tf
module "createHostAmongMetaData" {
  source       = "../Modules/HostMetaData"
  name         = "myServer"
  hcloud_token = var.hcloud_token
}
# variable.tf
variable "name" {
  type = string
  nullable = false
}

variable "hcloud_token" { 
  type = string
  nullable = false
}
 
# main.tf
resource "hcloud_server" "newServer" { 
  name         = var.name
  image        = "debian-13"
  server_type  = "cx23" }

resource "local_file" "hostdata" {
  content = templatefile("${path.module}/Tpl/hostdata.json", {
    ip4   = hcloud_server.newServer.ipv4_address
    ip6   = hcloud_server.newServer.ipv6_address
    location = hcloud_server.newServer.location
  })
  filename        = "Gen/${var.name}.json" }
 
{
  "network": {
    "ipv4": "${ip4}",
    "ipv6": "${ip6}"
  },
  "location": "${location}"
}
 
{
  "network": {
    "ipv4": "157.180.78.16",
    "ipv6": "2a01:4f9:c013:be69::1"
  },
  "location": "hel1"
}
 
resource "local_file" "hostdata" {
  content = templatefile(
     "${path.module}/Tpl/hostdata.json",{
    ip4 =
      hcloud_server.newServer.ipv4_address
    ip6 =
      hcloud_server.newServer.ipv6_address    
    location = 
      hcloud_server.newServer.location
  })
  filename        = "Gen/${var.name}.json" 
}
 
├── CreateHostByModule
│   ├── main.tf
│   ├── variables.tf
│   └── Gen
|        └── hostdata.json ◀━━┓
└── Modules                    ┃    
    └── HostMetaData           ┃
        ├── main.tf            ┃
        ├── outputs.tf         ┃
        ├── providers.tf       ┃
        ├── Tpl                ┃
        │   └── hostdata.json ━┛
        └── variables.tf
  1. Generating host meta data
  2. A module for ssh host key handling
  • Cloud provider
    • ➟ Selected Terraform topics
      • ➟ Volumes 
resource "hcloud_server" "mySrv" {
  server_type  =  "cx23"
...
}

resource "hcloud_volume" "volume01" {
  name      = "volume1"
  size      = 10
  server_id = hcloud_server.mySrv.id
  automount = true
  format    = "xfs"
}
 
df
...
/mnt/HC_Volume_100723816
Image layer 1
Image layer 2
Image layer 3
Image layer 4
  1. Partitions and mounting
output "volume_device" {
 value = hcloud_volume.volume01.linux_device
 description = "The volume's device"
}
 
terraform apply
...
volume_device = "/dev/disk/by-id/scsi-0HC_Volume_102626366"

Desired /etc/fstab entry: 

/dev/disk/by-id/scsi-0HC_Volume_102626366 /volume01 xfs rw,defaults 0 2
 
resource "hcloud_server" "mySrv" {
    ...
  user_data = templatefile("tpl/userData.yml", {     
     ...
    volume01Id = hcloud_volume.volume01.id
  })
}
resource "hcloud_volume" "volume01" {
  server_id = hcloud_server.mySrv.id
  ...
}

Problem: Cyclic dependency

mySrv <--> volume01
main.tf 
resource "hcloud_volume" "vol01" {
  name = "volume1"  ... }
resource "hcloud_server" "hello" {
  user_data = templatefile("userData.yml.tpl", {
       volumename = hcloud_volume.vol01.name
       device = hcloud_volume.vol01.linux_device
  })  ... }
resource "hcloud_volume_attachment" "main" {
  volume_id=hcloud_volume.vol01.id
  server_id=hcloud_server.hello.id ... }
 
# userData.yml.tpl
runcmd:
 - mkdir /${volumename}
 - echo ${device} /${volumename} xfs rw,defaults 0 2 >> /etc/fstab
  1. Mount point's name specification
  • Cloud provider

  • Dedicated course related DNS server ns1.hdm-stuttgart.cloud.

  • One subdomain per group e.g. g3.sdi.hdm-stuttgart.cloud corresponding to Group 3.

  • Zone edits require a subdomain specific hmac secret key being provided as dnsupdate.sec file in your personal group entry below the SDI course: 

    hmac-sha512:g3.key:I5sDDS3L1BU...

    Note

    The per zone secrets have been created using tsig-keygen -a hmac-sha512.

  • Edits become globally visible. Mind the TTL setting: A higher value means you'll have to wait longer until updates become visible.

Key file available in your working group below 113475 Software defined Infrastructure. 

$ export HMAC=hmac-sha512:g3.key:YXWSeh3l... 
$ dig @ns1.hdm-stuttgart.cloud -y $HMAC -t AXFR g3.sdi.hdm-stuttgart.cloud 
...
g3.sdi.hdm-stuttgart.cloud. 600	IN	SOA	ns1.hdm-stuttgart.cloud. ...
g3.sdi.hdm-stuttgart.cloud. 600	IN	NS	ns1.hdm-stuttgart.cloud.
g3.sdi.hdm-stuttgart.cloud. 600	IN	SOA	ns1.hdm-stuttgart.cloud. ...
g3.key.			0	ANY	TSIG	hmac-sha512. 1746433052 300 64 bak...
...

See AXFR for details. 

export HMAC=hmac-sha512:g3.key:YXWSeh3l... 

$ nsupdate -y $HMAC
> server ns1.hdm-stuttgart.cloud
> update add www.g3.sdi.hdm-stuttgart.cloud 10 A 141.62.75.114 
> send
> quit
$ dig +noall +answer @ns1.hdm-stuttgart.cloud www.g3.sdi.hdm-stuttgart.cloud
www.g3.sdi.hdm-stuttgart.cloud. 9 IN  A       141.62.75.114
$ dig  +noall +answer @8.8.8.8  www.g3.sdi.hdm-stuttgart.cloud
www.g3.sdi.hdm-stuttgart.cloud. 3 IN  A       141.62.75.114
$ nsupdate -y $HMAC
> server ns1.hdm-stuttgart.cloud
> update delete www.g3.sdi.hdm-stuttgart.cloud. 10 IN  A       141.62.75.114
> send
> quit
>
$ dig  +noall +answer @8.8.8.8  www.g3.sdi.hdm-stuttgart.cloud
$

Note

Examples at DNS Updates with nsupdate

goik>dig +noall +answer  www.g3.sdi.hdm-stuttgart.cloud
www.g3.sdi.hdm-stuttgart.cloud. 9 IN A	141.62.75.114

  • DNS record caching.

  • Another 9 seconds to go before DNS cache invalidation.

  1. Enhancing your web server.

  • Providing DNS info for sdi.hdm-stuttgart.cloud and sub-zones:

    • g1.sdi.hdm-stuttgart.cloud

    • g2.sdi.hdm-stuttgart.cloud

    • ...

  • Remote API for per-zone editing

terraform {
  required_providers {
    dns = {
      source = "hashicorp/dns"
    }
    ...
}
provider "dns" {
  update {
    server        = "ns1.hdm-stuttgart.cloud"
    key_name      = "g12.key."  # Corresponding to your group e.g., Group 12
    key_algorithm = "hmac-sha512"
    key_secret    = var.dns_secret 
  }
}

This requires:

  • A corresponding variable "dns_secret" {...} declaration.

  • An e.g., export TF_VAR_dns_secret = "sVfw2a...vAUqw==" (non - versioned!) environment variable. 

resource "dns_a_record_set" "helloRecord" {
  zone = "${var.dnsDomainName}." # The dot matters!
  name = hcloud_server.mySrv.name
  addresses = [hcloud_server.mySrv.ipv4_address]
  ttl = 10
}

Likewise with dns_cname_record and other record types.

  1. Creating DNS records
  2. Creating a host with corresponding DNS entries
  3. Creating a fixed number of servers
  • Cloud provider
    • ➟ Selected Terraform topics
      • ➟ Generating web SSL certificates

https://www.youtube.com/watch?v=T4Df5_cojAs

Domain Validated

Fully automated process solely based on DNS / infrastructure challenges.

Organization Validated

Checking organization in question.

Extended Validation

Additional checks i.e. telephone based verification. 

provider "acme" {
  server_url = "https://acme-staging-v02.api.letsencrypt.org/directory"

  # Production:
  # server_url = "https://acme-v02.api.letsencrypt.org/directory"
}
resource "tls_private_key" "private_key" { algorithm = "RSA" }
resource "acme_registration" "reg" {
  account_key_pem = tls_private_key.private_key.private_key_pem
  email_address   = "nobody@example.com"
}
resource "acme_certificate" "certificate" {
      ...
  dns_challenge { ... }
}
resource "acme_certificate" "certificate" {
  ...
  dns_challenge {
    provider = "route53"
    ...
  }
}

acme DNS provider list:

  • acme-dns

  • alidns

  • ...

  • rfc2136

  • ...

  • zonomi

 
dns_challenge {
  provider = "rfc2136"

  config = {
    RFC2136_NAMESERVER     = "ns1.hdm-stuttgart.cloud"
    RFC2136_TSIG_ALGORITHM = "hmac-sha512"
    RFC2136_TSIG_KEY       = "gxy.key."
    RFC2136_TSIG_SECRET    = var.dns_secret # Relates to environment variable TF_VAR_dns_secret
  }
}
... updating zone 'goik.sdi.hdm-stuttgart.cloud/IN': 
  deleting rrset at '_acme-challenge.goik.sdi.hdm-stuttgart.cloud' TXT
... updating zone 'goik.sdi.hdm-stuttgart.cloud/IN': 
    adding an RR at '_acme-challenge.goik.sdi.hdm-stuttgart.cloud' TXT 
      "GtJZJZjCZLWoGsQDODCFnY37TmMjRiy8Hw9M1eDGhkQ"
... deleting rrset at ... TXT
... adding an RR ... TXT "eJckWl2F43nsf27bzVOjcrTGp_VFeCj2qTVM5Uodg-4"
... deleting an RR at _acme-challenge.goik.sdi.hdm-stuttgart.cloud TXT
... updating zone ... deleting an RR ... TXT
  1. Creating a web certificate
  2. Testing your web certificate
  3. Combining certificate generation and server creation
  • Cloud provider
    • ➟ Selected Terraform topics
      • ➟ Private networks
Image layer 1
Image layer 2
Image layer 3
Image layer 4
Image layer 5
resource "hcloud_network" "privateNet" {
  name     = "Private network"
  ip_range = "10.0.0.0/8"
}
resource "hcloud_network_subnet" "privateSubnet" {
  network_id   = hcloud_network.privateNet.id
  type         = "cloud"
  network_zone = "eu-central"
  ip_range     = "10.0.1.0/24"
}
resource "hcloud_network_route" "gateway"{
  network_id = hcloud_network.privateNet.id
  destination = "0.0.0.0/0"
  gateway = "10.0.1.20"
}
resource "hcloud_server" "gateway" {
    ....
  public_net {
    ipv4 = hcloud_primary_ip.gatwewayIp.id
  }
  network {
    network_id = hcloud_network.privateNet.id
    ip         = "10.0.1.20"
  }
}
resource "hcloud_server" "intern" {
    ....
  public_net {
    ipv4_enabled = false
    ipv6_enabled = false
  }
  network {
    network_id = hcloud_network.privateNet.id
    ip         = "10.0.1.30"
  }
}
  1. Creating a subnet

  • Host “intern” does not have Internet access.

  • Consequences:

    1. No package updates.

    2. No package installs

    3. ...

    • Allow IP forwarding on gateway host

    • Configure NAT enabling gateway host as router

  2. Use an application level gateway:

    1. apt-cacher-ng

    2. Squid

Image layer 1
Image layer 2

  • Problem: apt-cacher-ng installation requires time for service to become available.

  • Consequence: Package installs / updates on host “intern” must be deferred.

  • Problem: No standard Terraform “service ready” dependency management hook.

tpl/waitForAptProxy.sh main.tf 
#!/bin/bash
echo "Waiting for apt-cacher-ng ..."
while ! nc -z ${interface} 3142; 
  do
     sleep 8 # wait 8 second for next poll
     echo apt-cacher-ng not yet ready ...
  done

echo "apt-cacher-ng service ready"
 
resource "local_file" "waitForAptProxy" {
  content = templatefile(
               "tpl/waitForAptProxy.sh", {
    interface = var.frontendServer.privateIp
  })
  filename        = "gen/waitForAptProxy"
  file_permission = "755"
}
 
resource "null_resource" "waitForProxy" {
  connection {
    type     = "ssh"
    user     = "devops"
    host_key = ...public_key_openssh
    agent    = "true"
    host     = ...web.ipv4_address
  }
  provisioner "remote-exec" {
    script = local_file.waitForAptProxy.filename
  }
}
 
resource "hcloud_server" "intern" {
...
  depends_on = [
    hcloud_network_subnet.pSubnet
   ,null_resource.waitForProxy
  ]
}
  1. Adding an application level gateway
  • Cloud provider
    • ➟ Pulumi

  1. Infrastructure as code (IAC)

  2. Infrastructure state model like Terraform

  3. Looks imperative but IS actually declarative.

  4. Various language bindings: TypeScript, Python, Go, C#, Java ...

Pulumi / Java Terraform 
import com.pulumi.hcloud.Server;       
...
var server = new Server (
   "mySrv",
   ServerArgs.builder()
     .name("hello")
     .image("debian-13")
     .serverType("cx23")
     .location("nbg1")
     .build()
);
 
resource "hcloud_server" "mySrv" {          
  name         = "hello"
  image        =  "debian-13"
  server_type  =  "cx23"
  location     =  "nbg1"
}
Architecture
public static void stack(Context ctx) {
   Server node1 = new Server ("Server1", ServerArgs.builder()        
                .name("node1") ... .build()
   );

   Output<String> ipv4Node1 =  node1.ipv4Address();
   ctx.export("Node1 ipv4", ipv4Node1);

   Server node2 = new Server ("Server2", ServerArgs.builder()
                        .name("node2") ... .build()
   );
   ctx.export("Node2 ipv4", node2.ipv4Address());
}
$ pulumi up
Previewing update (java):
     Type                    Name        Plan       
 +   pulumi:pulumi:Stack     Hello-java  create     
 +   ├─ hcloud:index:Server  Server2     create     
 +   └─ hcloud:index:Server  Server1     create     

Outputs:
    Node1 ipv4: [unknown]
    Node2 ipv4: [unknown]

Resources:
    + 3 to create

Do you want to perform this update?  [Use arrows to move, type to filter]
> yes
  no
  details

Updating (java):
     Type                    Name        Status            
 +   pulumi:pulumi:Stack     Hello-java  created (10s)     
 +   ├─ hcloud:index:Server  Server2     created (8s)      
 +   └─ hcloud:index:Server  Server1     created (9s)      

Outputs:
    Node1 ipv4: "88.99.224.53"
    Node2 ipv4: "91.99.143.208"

Resources:
    + 3 created

Duration: 12s
  1. Hello server creation
static private final String cloudInitCode = """
  #cloud-config
  packages:
    - nginx
  runcmd:
    - systemctl enable nginx
    - rm /var/www/html/*
    - echo I am Nginx @ $(dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com)
      created $(date -u) >> /var/www/html/index.html""";

var server = new Server ("mySrv", ServerArgs.builder()
     .name("hello")
        ...
     .userData(cloudInitCode)
     .build()
);
static private final String sshPublicKeyIdentifier = "userSshPublicKey";
...
final String content = Files.readString(Path.of("/home/goik/.ssh/id_ed25519.pub"));

new SshKey(sshPublicKeyIdentifier, SshKeyArgs.builder()
    .name(sshPublicKeyIdentifier).publicKey(content).build()
);
...
var server = new Server ("mySrv", ServerArgs.builder()
     .name("hello")
        ...
     .sshKeys(sshPublicKeyIdentifier)
     .build()
);
  1. ssh keys and cloud-init script

Requirements:

  • Firewall

  • Host(s) carrying web server

  • DNS

var firewall = new Firewall("stdFirewall", FirewallArgs.builder()
        .name("my-firewall")
        .rules( FirewallRuleArgs.builder()
                        .direction("in")
                        .protocol("icmp")
                        .sourceIps(
                                "0.0.0.0/0",
                                "::/0")
                        .build(),
                ...
                FirewallRuleArgs.builder()
                        .direction("in")
                        .protocol("tcp")
                        .port("22")
                        .sourceIps(
                                "0.0.0.0/0",
                                "::/0")
                        .build())
        .build());
> pulumi package add terraform-provider hashicorp/dns
src
└── main
    ├── java
    │   ├── com
    │   │   └── pulumi
    │   │       └── dns
    │   │           ├── AaaaRecordSetArgs.java
    │   │           ├── AaaaRecordSet.java
    │   │           ├── ARecordSetArgs.java
    │   │           ├── ARecordSet.java
    │   │           ├── CnameRecordArgs.java
    │   │           ├── CnameRecord.java
    │   │           ├── config
    │   │           │   ├── inputs
    │   │           │   │   └── Updates.java
    │   │           │   └── outputs
    │   │           │       └── UpdatesGssapi.java
    │   │           ├── Config.java
    │   │           ├── DnsFunctions.java
    │   │           ├── inputs
    │   │           │   ├── AaaaRecordSetState.java
    │   │           │   ├── ARecordSetState.java
     ...             ...
var dnsProvider = new Provider("dnsProvider", ProviderArgs.builder()
               .updates(ProviderUpdateArgs.builder()
                .server("ns1.sdi.hdm-stuttgart.cloud")
                 .keyName("gxy.key.")
                 .keyAlgorithm("hmac-sha512")
                 .keySecret(DNS_SECRET)
                 .build() )
               .build()
);
var ipv4 = new ARecordSet("www_server_A_record", ARecordSetArgs.builder()
                .zone("gxy.sdi.hdm-stuttgart.cloud.")
                .name("www")
                .addresses (server.ipv4Address() //** Ugly!
                  .applyValue(id -> List.of(id))
                )
                .ttl(100)
                .build()
                , CustomResourceOptions.builder()
                    .provider(dnsProvider)
                    .build()
);

var ipv6 = ... // HdM, dream on ...

**: See Erroneous API Docs samples

final var server = new Server("www-server", ServerArgs.builder()
                    .name("www")
                    .image("debian-13")
                       ...
                    .firewallIds(firewall.id()
                       .applyValue(id -> List.of(Integer.parseInt(id)) // Ugly!
                    )
                    .build());
  1. A set of servers