Moritz Clasmeier

Rustberry Jukebox

2019-11-12T00:00:00+00:00

So, I am one of those tech-affine parents, who wanted to build some kind of Jukebox device for their child. In this post I provide an overview of my specific jukebox implementation.

There are commercial products and existing hobbyist projects, but for several reasons I decided to work on my own project. It is called Rustberry Jukebox and the core software is written in Rust. Even though the name might suggest otherwise, please note that this is not related to an existing Rust crate with a similar name.

The feature set I had in mind for the jukebox device:

Rustic/old aesthetics (as opposed to the polished modern interface of commercial products)
Stream music from Spotify via WLAN
Playback controllable via RFID tags
Hardware button for switching the jukebox on and off
Status LEDs (jukebox running and jukebox playing)
The code should be conveniently cross-compilable for a Raspberry Pi.

This is the resulting jukebox:

Let us now have a closer look.

Components

The Case

I am pretty unexperienced when it comes to physical manufacturing. Fortunately I found a potential enclosing, which seemed promising: an old looking suitcase / treasure chest. I thought it might be feasible to build a double bottom into this suitcase, having enough hidden space for all the tech stuff inside (Raspberry Pi, speakers, power, circuits, cables / adapters) and providing enough space above the double bottom for user controls and RFID-equipped records.

The Records

What should the RFID tags be attached to, if one does not want to use plain RFID cards or transponders? Again, I was lucky to find a product, which matches the intended rustic aesthetics and allows for cheap extensibility: wooden discs. The idea is to stick the RFID tag to the bottom side of a disk and use the top side for artwork.

The Tech

I had an old Raspberry Pi 2 laying around, which I intended to use as a technological foundation for the jukebox. The playback should be controllable via RFID tags – a technology, which I am not familiar with at all. After some research I had identified the MIFARE RC522 as a common and well-supported RFID reader/writerfor suitable for use with a Raspberry Pi. This device is compatible with RFID tags such as MIFARE Classic 1K Chip, 13.56mhz.

For enabling audio output, I went with the Trust Leto 2.0 USB Speakers. I was somewhat worried about the energy consumption (6W) of these speakers, since I prefer to not use additional power sources besides the Raspberry Pi’s own USB connectors for compactness reasons. But they seem to work fine at medium volume. The USB speakers connect to the computer via a standard stereo jack, but as is well-known the stereo jack output of the Raspberry Pi offers poor quality. Therefore I have decided to extract the audio signal from the Raspberry HDMI output using a simple HDMI-to-VGA adapter capable of extracting the HDMI audio signal.

The Software

As mentioned above there are already software solutions for an RFID-controllable jukebox. But after a quick look at the Phoniebox software RPi-Jukebox-RFID I decided to build my own project. The primary motives for this decision include the following:

RPi-Jukebox-RFID seems like a rather huge Python project and I am neither particularly familiar with Python tooling, nor with the Python ecosystem. Also, I have not had the best experiences with Python codebases in the past, though I cannot judge about the quality of this particular project.
Currently I am primarily interested in one particular use-case: Spotify integration – which is labelled as “experimental” for RPi-Jukebox-RFID. I wanted something more compact and simple.
From the introductory video of the Phoniebox it seems that the RFID-control logic is such that an RFID tag is used only for triggering playback. What I would prefer is that an RFID tag is used for controlling the playback, which means starting and stopping it, similar to the commercial Toniebox: Playback is active as long as the RFID tag is in range of the RFID reader.

After evaluation of a few options I decided to build the software with Rust, since I learned to like that language, it performs well, has a great package ecosystem, can be used for lower-level hardware access, comes with a low memory footprint and it has a pretty good cross-compilation story. Besides, I have never worked on a similar project and it seemed like an interesting learning opportunity.

Spotify Playback

Regarding Spotify Playback, my initial plan was to run Firefox on the Raspberry Pi and use the Spotify Web SDK for providing the Spotify streaming capabilities. This worked pretty well on my development machine. But once I tried it out on the Raspberry I had learn the hard way that the Spotify Web SDK requires Widevine DRM Support, which the non-official Firefox builds do not contain (and for ARM there are no official Firefox builds). So this was my daily lesson in the category “Integrate early”. So, how do I stream from Spotify? Well, Librespot comes to the rescue:

librespot is an open source client library for Spotify. It enables applications to use Spotify’s service, without using the official but closed-source libspotify. Additionally, it will provide extra features which are not available in the official library.

Librespot is used and packaged by the Raspotify project. They provide easy to install Debian packages for ARM Raspbian. With Raspotify installed and configured to use a specific Spotify Premium account, the Raspberry is ready to be used as Spotify client through the Spotify Web API.

OS

Actually I would have preferred being able to use NixOS on the Raspberry Pi, but unfortunately the ARM port of NixOS was way to rough around the edges for my use-case. The issues I have had with NixOS even on my Raspberry Pi 3, which comes with an AARCH64 CPU somewhat supported by upstream NixOS, included:

Missing and/or incomplete documentation, in particular when it comes to configuring the Raspberry Pi firmware and the boot process (after having written a first configuration.nix according to the documentation, the Raspberry Pi was unable to boot).
The boot process is slower than Raspbian’s.
After about 2h work I was still not able to get audio working – something that just works on Raspbian.

With NixOS I like being able to declaratively configure the complete operating system with all required services and deploy a system configuration to a remote NixOS with complete rollback functionality built-in. Maybe somewhen in the future I can write the NixOS derivations for my Jukebox Software and deploy it to a Raspberry Pi running NixOS. But this is not today.

Given this unsatisfying situation I have decided to build on Raspbian.

RFID IO

A quick search on crates.io revealed the following list of potentially useful crates:

It seems that mfrc522 is currently very limited in functionality (see https://docs.rs/mfrc522/0.2.0/mfrc522/struct.Mfrc522.html) and does not yet support reading and writing the data blocks on an RFID tag. rfid-rs on the other side does support reading from and writing to the RFID tag’s data blocks, but the upstream code base had some issues, which is why I have created my personal fork for the purpose of this project. I will see if my changes can be integrated into the upstream version of the crate.

GPIO

For GPIO access there are multiple crates available, for example:

I have decided to go with gpio-cdev, since – according to my understanding – using the character device API for GPIO is recommended for new applications. I have been missing some built-in functionality for listening for events on mutliple GPIO lines at the same time, but that was easy to implement using threads and channels.

At some point I had confused the different GPIO line labeling systems. For debugging GPIO, the following command turned out to be very helpful:

watch -d -n 0.5 gpio -g readall

(the gpio program comes from the wiringpi package)

Circuits

For the first vesion of the Jukebox the following hardware related functionality should be be supported:

A single physical button for switching the box on and shutting it down.
Stable RFID tag reading via the RC522 reader.
A status LED indicating that the box is running.
A status LED for indicating that it is in playback mode (i.e. an RFID tag is near the RFID reader).

Power Switch

I found an article, which describes the wake-up functionality:

Simply put, shorting pins 5 and 6 (GPIO3 and GND) together will wake the Pi up from a halt state.

As indicated above, there should be a single physical button for switching the device on and shutting it down. Therefore, it is already clear that the power button needs to be connected to GPIO3, which needs to be configured as an input line. Since pressing the button is required to connect GPIO3 with GND, the GPIO3 line needs to be set to high when the button is not pressed. In other words, we need a Pull-up resistor connecting GPIO3 through a resistor with a voltage source when the button is not pressed. When the button is pressed GPIO3 will be connected with GND.

Power Status LED

There is a nice hack for building a status LED: Connect a LED to the Raspberry Pi’s serial console as described in the article Build a Simple Raspberry Pi LED Power/Status Indicator.

Playback Status LED

This is nothing fancy, just another LED conneted to a regular GPIO input line, which is configured as output and controlled in software.

Cross Compilation

I am currently developing on Darwin/x86_64, but the Raspberry Pi runs Linux/ARMv7. I have created a Docker image description, for a build container. The container contains the cross-compilation toolchain and is reachable via SSH. A build script is used for producing binaries of the Rustberry Jukebox software for the Raspberry Pi.

Nixified Jekyll setup

2019-01-29T00:00:00+00:00

What I particularly like about Nix is its ability to encapsulate environments. No need to remember how to set up a certain environment, just encode it in a Nix expression, check it into version control and use nix-shell. This is especially useful when having to occasionally deal with certain environments, without being too familiar with the technical details. For me one such example is Jekyll, a Ruby application (and I am not familiar with Ruby).

I have created a new Jekyll blog and used bundix to convert a Gemfile into a Nix expression (gemset.nix). To this, add the following default.nix:

with (import <nixpkgs> {});
let
  ghPagesEnv = bundlerEnv {
    name = "mtesseract-github-pages-env";
    inherit ruby;
    gemdir = ./.;
  };
in
{
  shell = stdenv.mkDerivation {
    name = "mtesseract-github-pages";
    buildInputs = [ ghPagesEnv ];
  };
  serve = stdenv.mkDerivation {
    name = "mtesseract-github-pages-serve";
    buildInputs = [ ghPagesEnv ];
    shellHook = ''
      exec ${ghPagesEnv}/bin/jekyll serve --watch
    '';
  };
}

This expression contains two attributes: shell and serve. Thus, nix-shell -A shell spawns an interactive shell having Jekyll in path, while nix-shell -A serve spawn Jekyll to serve the page at http://localhost:4000. For details, check out the repository of this blog.

Deploying a Reflex Application to NixOS

2019-01-28T00:00:00+00:00

In the following post I describe how a simple Haskell web application built on top of Reflex FRP can be deployed to a remote NixOS server. If a remote NixOS server is available using it for deployment can be cheaper alternative to commercial cloud providers in case the features of those (e.g. scaling, hosted services, availability guarantees, etc.) are not strictly required.

The sample Reflex application can be found on GitHub.

Our setting is as follows:

The application, consisting of a frontend and a backend, is built with Nix
The development system is a Mac (i.e., not x86_64-linux)
The application supports some trivial database access
The application is to be deployed to a remote server running NixOS (x86_64-linux)

The Application Skeleton

The application is based on the project skeleton as documented as part of the Reflex Platform. It contains three Cabal projects (common, frontend, backend) and the reflex-platform as a Git submodule. Nix expressions are used for building the applications.

In addition, there is a top-level directory deployment containing the deployment relevant code.

Building on macOS for Linux

For deploying we want to transfer a complete build artifact, a Nix closure, to the remote server and activate it. Since the development computer is assumed to be running macOS and the application needs to be built for Linux (x86_64-linux), we use Docker and the nix-docker project. The included script start-docker-nix-build-slave assists in setting up a Nix remote builder.

The application skeletons needs a tweak to the default.nix expression in order to respect the system parameter.

Database Access

Our application server (the system on which the application frontend and backend will be deployed) runs in its own virtual machine managed via libvirt. In order to separate the stateless and stateful aspects of the deployment, we set up a dedicated database server VM, i.e. a NixOS server running PostgreSQL. The database server is not directly connected to the internet, instead the database server and the application server are connected via an internal bridge.

The NixOS configuration of the database server looks as follows:

{ config, pkgs, ... }:

{
  imports =
    [
      ./hardware-configuration.nix
    ];

  boot.loader.grub.enable = true;
  boot.loader.grub.version = 2;
  boot.loader.grub.device = "/dev/vda";

  networking = {
    hostName = "db";
    interfaces.ens3.ipv4.addresses = [ { address = "192.168.122.2"; prefixLength = 24; } ];
    nameservers = [ "8.8.8.8" ];
    defaultGateway = "192.168.122.1";
    firewall = {
      allowedTCPPorts = [ 22 5432 ];
      enable = true;
    };
  };

  time.timeZone = "Europe/Berlin";

  environment.systemPackages = with pkgs; [
    vim postgresql100
  ];

  services = {
    ntp = {
      enable = true;
    };
    openssh = {
      enable = true;
    };
    postgresql = {
      enable = true;
      package = pkgs.postgresql100;
      enableTCPIP = true;
      authentication = pkgs.lib.mkOverride 10 ''
        local all all trust
        host all all 192.168.122.0/24 trust
        '';
      initialScript = pkgs.writeText "backend-initScript" ''
        CREATE ROLE app WITH LOGIN PASSWORD 'app' CREATEDB;
        CREATE DATABASE app;
        GRANT ALL PRIVILEGES ON DATABASE app TO app;
      '';
    };
  };
  system.stateVersion = "18.09";
  users.users.root.openssh.authorizedKeys.keys = [ "ssh-rsa ..." ];
  environment.noXlibs = true;
  system.autoUpgrade.enable = true;
}

This configuration makes sure that a PostgreSQL role app with the same password and database name is initially created.

Deploying

The Nix expressions needed for deployment together with a small shell script are stored in the directory deployment/:

pkgs.nix: Expression pinning a specific version of nixpkgs for reproducibility of builds.
system.nix: A complete NixOS system configuration including:
- a systemd configuration for running the backend application
- an nginx configuration for exposing the frontend and the backend via HTTPS using Let’s Encrypt certificates
deploy.nix: Wrapper for the system configuration using the pinned version of pkgs.nix. This expression can be used for building complete system configurations, which can be deployed to a remote server.

In addition the following Nix expressions are expected by this deployment setup:

dns.nix: Containing the DNS entries for the backend and the frontend application. For example:

{
  frontend = "app.example";
  backend = "api.example";
}

network.nix: The NixOS network configuration of the application server. For example

{
  hostName = "dev";
  interfaces = {
    ens3.ipv4.addresses = [ { address = "a.b.c.d"; prefixLength = 28; } ];
    ens9.ipv4.addresses = [ { address = "192.168.122.3"; prefixLength = 24; } ];
  };
  defaultGateway = "a.b.c.d";
  nameservers = [ "8.8.8.8" ];
  firewall.allowedTCPPorts = [ 22 443 ];
  extraHosts = ''
    192.168.122.2 db
  '';
}

Of course, the public IP address of the application server and the default gateway need to be filled in here. This configures two network interfaces, one public interface and one internal interface for talking to the database server (db).

ssh.nix: List of public SSH keys which can be used for logging into the application server:

[ "ssh-rsa ..." ]

Having this in place the actual deployment procedure is straight-forward and described in the shell script deploy.sh:

#!/bin/sh

set -e

if [ $# -ne 1 ]; then
    echo "Usage: $0 <remote ssh>" >&2
    exit 1
fi

REMOTE="$1"
echo "**** Deploying to $1 ****"

nix-build --attr system deploy.nix
artifact=$(readlink result)
nix-copy-closure $REMOTE $artifact
ssh $REMOTE "${artifact}/bin/switch-to-configuration" switch

As can be seen, we build the system configuration from deploy.nix (using nix-build --attr system), then we transfer the closure of the built artifact to the remote system and finally activate it. The shell script expects a remote ssh login as parameter, e.g. root@app.example. Credits to Gabriel Gonzales, as this procedure is heavily inspired by his post NixOS in production.

Example: Conduit Streaming with Servant

2017-05-25T00:00:00+00:00

Servant is an awesome set of Haskell packages for Web development. Under the hood the Servant server uses WAI and Warp. Forunately, the package wai-conduit provides support for creating HTTP responses from Conduit sources, which also allow for streaming responses from within a Servant server.

Edsko de Vries has described the basic mechanism for doing this in a Github comment.

This post describes this in more detail, giving a complete example (which can be cloned from its repository).

Let us start! The server API type for a trivial non-streaming example looks like this:

type ServiceAPI = "one" :> Get '[JSON] Int

A handler implementing this endpoint is expected to return a value of type Int and Servant takes care of serializing this value before sending it to the client. An implementation for this API can look like this:

serveAPI :: Server ServiceAPI
serveAPI = serveOne

serveOne :: Handler Int
serveOne = return 1

Now, imagine we want to implement an endpoint /ones, which streams infinitely many ones. Say we would like to use Conduit, then a reasonable handler type for the endpoint /ones would look like this:

serveOnes :: Handler (Source (ResourceT IO) (Flush ByteString))

Note: See the Conduit documentation for information on the Flush type.

Given the above type, an implementation for serveOnes could look as follows:

serveOnes = return go
  where go = do
          yield (Chunk "1")
          yield Flush
          go

Now, in order to be able to use this handler for implementing a Servant API, we need to extend Servant’s Web API DSL. Say we would like to extend the above API with a streaming endpoint as follows:

type ServiceAPI =
   "one" :> Get '[JSON] Int
   :<|> "ones" :> GetStream ByteString

We can begin by definin a new type

data GetStream a

In order to be able to use GetStream for defining Servant endpoints, we need to implement a HasServer instance for it. Edsko has demonstrated a basic class implementation in his comment on Github. I would like to make this a bit more configurable by defining a new type class Streamable:

class Streamable a where
  streamableToBuilder :: a -> Builder
  streamableCT :: Proxy a -> Maybe MediaType
  streamableCT _ = Nothing
  streamableDelimiter :: Proxy a -> Maybe Builder
  streamableDelimiter _ = Nothing

The method streamableToBuilder is obviously the most important method here as it allows us to convert a value of the type implementing this class into a ByteString Builder. The method streamableCT can be used for defining the response content type while streamableDelimiter can be used for defining a Builder delimiter which will be inserted in the response stream whenever the handler produces a Flush value. While we are at it, let us implement the following helper function:

toBuilderDelimited :: forall a. Streamable a => Flush a -> [Flush Builder]

This function will take care of inserting delimiters on Flush values. Implementing is is straight-forward:

toBuilderDelimited (Chunk a) = [Chunk (streamableToBuilder a)]
toBuilderDelimited Flush = case streamableDelimiter (Proxy :: Proxy a) of
                             Just delim -> [Chunk delim, Flush]
                             Nothing    -> [Flush]

Having this in place, we can now implement HasServer for GetStream a:

instance Streamable a => HasServer (GetStream a) ctxt where
  type ServerT (GetStream a) m = m (Source (ResourceT IO) (Flush a))

  route Proxy _ctxt sub = leafRouter $
    \env req k ->
      bracket createInternalState
              closeInternalState
              (runAction sub env req k . mkResponse)
    where

      mkResponse :: InternalState
                 -> Source (ResourceT IO) (Flush a)
                 -> RouteResult Response
      mkResponse st =
        Route
        . responseSource ok200 headers
        . (.| CL.map toBuilderDelimited .| C.concat)
        . transPipe (`runInternalState` st)

      headers :: ResponseHeaders
      headers =
        let maybeMediaTypeBS = Media.renderHeader <$> streamableCT (Proxy :: Proxy a)
        in maybeToList $ ("Content-Type",) <$> maybeMediaTypeBS

The function mkResponse is responsible for converting a Conduit source into a HTTP Response. It uses the previously defined helper function toBuilderDelimited and responseSource from the wai-conduit package. The function headers takes care of producing the desired Content-Type header.

So far we don’t have any instances for Streamable. A naive implementation for ByteString might look like this:

instance Streamable ByteString where
  streamableToBuilder = lazyByteString . ByteString.Lazy.fromStrict

Here we convert a (strict) ByteString to a ByteString Builder without setting any content or delimiter.

For a different use case one might want to stream newline-delimited JSON values using a content type of application/x-json-stream. The Streamable instance implementing this would be

instance Streamable Value where
  streamableToBuilder = lazyByteString . encodingToLazyByteString . toEncoding
  streamableCT _ = Just ("application" Media.// "x-json-stream")
  streamableDelimiter _ = Just (lazyByteString "\n")

Given this instance, we can extend our service API by writing

type ServiceAPI = "one" :> Get '[JSON] Int
   :<|> "ones" :> GetStream ByteString
   :<|> "hello" :> GetStream Value

and implement a handler for the /hello endpoint as follows:

serveHello :: Handler (Source (ResourceT IO) (Flush Value))
serveHello = return go
  where go = do
          yield (Chunk (Array (Vector.replicate 10 (String "hello"))))
          yield Flush
          go

This produces a stream of newline-delimited JSON arrays each containing 10 strings. That’s it for now, happy streaming!

PureScript Binding for Pebble

2016-05-28T00:00:00+00:00

I hacked together a very rudimentary PureScript binding for the PebbleJS API, which allows for the writing of Pebble applications using Javascript.

The project is named FPebble and its repository can be found on Github..

To get an impression of how a simple Pebble application can look like using PureScript, have a look at the demo application.