OrGY: My Personal Technology Stack

May 10th, 2021 on ols.wtf

I’ve seen a few different alternatives to the traditional LAMP stack in my career. Often times these serve a valid purpose, such as WAMP which is near enough the same except running on Windows. Other times they seem over the top, at least to my eyes, as is the case with MEAN, the full JavaScript stack.

I have a set way of doing things for personal projects that allows me to develop my applications quickly and get them deployed with relative ease. I hadn’t specifically labelled it a stack as such, but once I saw the acronym it had I knew I had to write about it.

Put simply, the OrGY stack contains the following software elements:

  • OpenBSD as the underlying Operating System
  • relayd(8) as the TLS-terminating proxy
  • Golang as the development language
  • YAML as the data store

In this article, I will go through each of my choices, including an introduction to the technologies and the reasons they have been chosen. I will also provide an example application and associated proxy configuration to enable you to try out the stack with a relatively-simple project. It is important to note that this is not something I have sat down beforehand to shoehorn into a meme blog post, this is a natural progression of my experiences with software development and systems administration culminating into what I consider to be an optimal stack for personal projects.

OpenBSD

OpenBSD is a proactively secure operating system which puts an emphasis on portability, standardisation, correctness, and integrated crypto. If you aren’t already aware, the OpenBSD team are responsible for a number of other projects you no doubt use including OpenSSH, OpenBGPD, mandoc, and LibreSSL.

As a result of the work of the OpenBSD project, there are two extremely attractive system calls available to developers called pledge(2) and unveil(2). The system calls allow for greater control over what system calls a process can make, and what areas of the file system it is able to access.

For more details on these two fantastic syscalls, check out these slides from Bob Beck which explain further, as well as this talk from Theo de Raadt.

I started tinkering with OpenBSD roughly five years ago on an old laptop that I used as a test bed for technologies I wanted to learn more about. It has since made it on to all of my personal machines, and all of the servers I run.

Each application running on any server has its own user account with appropriate permissions set and the necessary limits changed in /etc/login.conf to afford it more resource if needed.

relayd(8)

This is a daemon which relays and redirects connections to a target host, whether this is localhost or a pool of servers with the relaying host acting as a load balancer. It monitors target hosts for availability and can forward at either layer 3 (via pf(4)) or layer 7.

Like all most OpenBSD daemons and services, the configuration file is simple but gives enough options to do exactly what one needs to accomplish.

I have relayd(8) set to listen on port 80 and 443, and then httpd(8) listens on 8080 and 8081 for plain text and TLS connections respectively.

Golang

Go was first published in 2009 and is based on C syntax. It offers features to help with safety and performance including safe memory usage, strict typing, super concurrency, and compilation to a static binary.

I started writing Go in June 20181, and since then I use it as regularly as Bash, if not even more so now. I especially love it for its simple way to implement networking through the official packages. I know that Rust is gaining popularity, and I have even attempted writing things in Rust before. Anything more than a simple Hello, World! application though and it feels like every line I write is followed by someone punching me in the side of the fucking head. It is really difficult to do, especially for someone like myself who has made an entire career out of being able to do just enough without knowing the full fundamentals, in this case the fundamentals of CS.

What is awesome about Golang is that it has support for the pledge(2) and unveil(2) system calls mentioned above, as well as a third-party package which wraps the system calls in a safe manner that treats the call as a noop on non-OpenBSD systems.

Finally, Golang has excellent support for reading and parsing YAML files. There are two main packages that I use:

  • github.com/gernest/front: This allows you to split an input into a map with YAML key/value pairs and a string containing the body, as split by a user-defined delimeter. I usually choose “---”.
  • gopkg.in/yaml.v2: This allows you greater control over parsing YAML, I tend to use it for projects that are YAML only, and don’t make use of front matter.

YAML

Speaking of YAML, people hate it. I work in “DevOps”2 where a hatred of YAML usually comes as standard. I quite like it though. It is clear, you can comment it, it is parseable by humans and machines, what is not to like?

When writing a small scale personal project, why bother with the headache of setting up yet another MySQL database, or running MongoDB in a docker container? Yes you could use SQLite, and I do find myself doing this on occasion. But SQLite is just a file, as is YAML, so where are the gains?

Using YAML files gives you fine grain control over permissions of who can read or write your data. If you want your orgy application data to be writable only by its user, but readable by a select few other users, ou can simply chmod the file to 640, and chown it to orgy:orgyread and add users to the orgyread group as required.

An Example Application

By way of demonstration, I have produced an example application that is running as a live demo. This demo is running the below code and configuration.

Application Code

The entirety of the application code is laid out below. It exists in source control if you want to check it out there, but we will go through each section here with a degree of implied knowledge.

package main

import (
    "fmt"
    "html/template"
    "io/ioutil"
    "log"    
    "net/http"
    "time"

    "github.com/gorilla/mux"
    "gopkg.in/yaml.v2"
    "suah.dev/protect"
)

Here we declare the package as main, i.e. not a module, and import the standard and third-party libraries as required. The standard libraries are pretty self explanatory. In terms of the third-party libraries, mux is a router, yaml.v2 is one of the YAML libraries mentioned above, and protect is the wrapper library for the pledge(2) and unveil(2) syscalls also noted earlier.

var countCount uint64
var countUpdated string

type CountData struct {
    Updated string `yaml:"updated"`
    Count   uint64 `yaml:"count"`
}

We define countCount and countUpdated—I hate naming things can you tell?—as a 64-bit unsigned integer and a string respectively. There is no inbuilt protection of overflows, so we must handle that elsewhere. In this section we also define a struct of CountType which will contain the data which we have previously defined. These are also tagged with the corresponding key that will be in the YAML file.

func initialiseCount() {
    countData := CountData{}
    file, err := ioutil.ReadFile("count.yaml")
    if err != nil {
        log.Fatal(err)
    }
    err = yaml.Unmarshal([]byte(file), &countData)
    if err != nil {
        log.Fatal(err)
    }
    countCount = countData.Count
    countUpdated = countData.Updated
}

This is our first function to be called (outside of main). It reads the contents of the count.yaml file and unmarshals it using the struct we defined earlier. It then writes the value of each key in the YAML file to the right variable which we defined earlier.

func writeCount() {
    dataForFile := fmt.Sprintf("updated: %s\ncount: %v", countUpdated, countCount)
    err := ioutil.WriteFile("count.yaml", []byte(dataForFile), 0644)
    if err != nil {
        log.Fatal(err)
    }
}

This function writes the values of countCount and countUpdated back to the file in order to persist the data.

{% raw %}
var displayCount = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    data := CountData{
        Count:   countCount,
        Updated: countUpdated,
    }
    html := `<!doctype html>
    <html lang="en">
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <head>
    <title>orgy stack example (counter)</title>
    <link rel="stylesheet" href="https://ols.wtf/boilerplate.css">
    </head>
    <body>
    <h1>orgy stack example (counter)</h1>
    <p>The counter is at {{ .Count }}, it was updated at {{ .Updated }}</p>
    <form action="/increase" method="post">
    <button>Increase the counter</button>
    </form>
    </body>
    </html>`
    tmpl := template.Must(template.New("display").Parse(html))
    err := tmpl.ExecuteTemplate(w, "display", data)
    if err != nil {
        log.Fatal(err)
    }

})
{% endraw %}

In this function we are passed the HTTP response writer, and a pointer to the request. We assign the current values of countCount and countUpdated to a variable which is of type CountData, the struct we defined earlier. We then have a small amount of HTML which contains placeholders for the values. This is then executed and written to the HTTP response writer.

var increaseCount = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    if countCount < 18446744073709551615 {
        t := time.Now()
        countUpdated = t.Format("2006-01-02 15:04:05")
        countCount++
        redirPath := "/"
        r.Method = "GET"
        http.Redirect(w, r, redirPath, 302)
    } else {
        fmt.Fprintf(w, "Cannot increase counter any further, sorry.")
    }
})

This function handles the post request on /increase which increments the countCount by one and updates countUpdated with the current date and time. It does this only if the current value of countCount is less than the number at which it would overflow to 0.

func main() {
    protect.Pledge("stdio rpath wpath cpath inetunix unveil error")
    protect.Unveil("./count.yaml", "rwc")
    protect.UnveilBlock()
    initialiseCount()
    ticker := time.NewTicker(5 * time.Second)
    quit := make(chan struct{})
    go func() {
        for {
            select {
            case <-ticker.C:
                writeCount()
            case <-quit:
                ticker.Stop()
                return
            }
        }
    }()
    r := mux.NewRouter()
    r.Handle("/", displayCount).Methods("GET")
    r.Handle("/increase", increaseCount).Methods("POST")
    http.ListenAndServe(":1337", r)
}

Finally we have our main function, which is called first. At the top we use protect to specify which syscalls we intend to make (and we are not allowed to make any others) as well as the path we want access to, and the permissions we require on that path. After this we initialise the count variables from the YAML file, wrap the function to write back to the file in a loop that runs every 5 seconds, and initialise our router.

This isn’t optimal code, remember earlier how I said I have gotten by in life by hacking things together, but it is elegant enough for me and does the job in a way that I feel happy with.

Proxy configuration

log connection
log state changes

external_ipv4 = "45.63.99.239"

table <web> { 127.0.0.1 }
table <acme> { 127.0.0.1 }
table <avgen> { 127.0.0.1 }
table <orgy> { 127.0.0.1 }

http protocol "www" {
  match request header "Host" value "ols.wtf" forward to <web>
  match request path "/.well-known/acme*" forward to <acme>
  match request path "/avatar" path strip 1 forward to <avgen>
  match request path "/orgy-example" path strip 1 forward to <orgy>

}

relay "www4" {
  listen on $external_ipv4 port 80
  protocol www
  forward to <orgy> port 1337
  forward to <avgen> port 1338
  forward to <acme> port 8081
  forward to <web> port 8081
}

The above is a pretty consolidated view of an /etc/relayd.conf which hosts a couple of plain text application alongside a web server, and also has a route configured for acme just in case we decide to generate some certs and go secure any time soon.

And there we have it, the OrGY stack. I would encourage you to look at this without the cynicism that you undoubtedly have by this point in the article. I am under no illusions that this stack would not fly in any sort of production or large-scale environment. It doesn’t scale well, it can be made distributed with NFS, sure, but would actively choose such a life?

I hope you have enjoyed reading this insight into what goes on in my head with regard to application development, and I look forward to reading the responses and outrage that people may feel. Again I would like to assure you this is 100% real.

Go forth and OrGY


  1. I needed to contribute to an internal project at work, which we use to raise tickets in Jira from Slack. We wanted a quick way to add story points to the ticket without needing to open Jira, which is what my first even Golang commit was. 

  2. You might find this post about job titles entertaining. 


Do you have a comment to make on this content? Start a discussion in my public inbox by emailing ~ols/public-inbox@lists.sr.ht. You can see the inbox here.