Approach

I want to imagine that I'm building this project in terms of it being a product which will be manufactured and shipped. This raises interesting questions such as "what is the manufacturing process?" and "how would I tell a manufacturer to implement the process?".

These same two questions could be rephrased as "how is the ModulOSC software, and everything needed to build and run it built?" and "what do I need to do to go from source code to a running project?".

In asking these questions, I'm asking whether I can automate the process of checking out the source code and building an SD card image for the RPi. Also, I want to ask whether if I run this process more than once with the same inputs (source code), that the output is actually the same. This has practical consequences for the development of this project, since there are a lot of complex components and I want to do as little work as possible in order to get my latest code running. I also want to ensure that if I have a known-good set of sources that I will be able to produce a working system image from it at any time.

This is distinctly different from the prevailing RPi hacking community philosophy of grab an OS image, run it and apt-get install a bunch of stuff in an unmanaged and ad-hoc manner until you have everything that you need (plus loads more that you don't) to run a poorly written python script.

Solution

I've chosen to use the yocto project for my linux and development tools needs. The website will explain in depth what it is, what it does and how it does it. To summarise though, it can do the following:

  • Take as input a bunch of "recipe" and configuration files which describe how to build software packages and which machine(s) to build them for.
  • Build a native compiler tool chain (meaning, build a compiler which runs on your computer which produces binaries for your computer)
  • Using the native tools, build a cross compiler tool chain (meaning, build a compiler which runs on your computer which produces binaries for the RPi)
  • Build all the libraries and packages needed for your image for the RPi
  • Install all the packages from an image recipe into an SD card image

It actually can do a ton more than this, but these are the essentials which I require. All I need to do is provide the configuration and recipe files which specify which software I want built and packaged for the RPi. Particularly useful are the toolchain builds, these provide everything I need to compile the software on my desktop for the RPi, rather than resorting to installing and using compilers on the RPi itself, which not only takes up a lot of space on the device, but which would also be painfully slow.

The recipes for this system are arranged in layers; for software I need which isn't in any of the default layers, I can simply create my own layer with them in and tell the configuration to use my layer. My layer is available on my bitbucket account and is called meta-bel. So far, I've included recipes for nginx and uwsgi (to run the web UI), a few necessary libraries, and the image recipes which build the SD cards I want to boot.

Using this system, I can simply issue a command such as bitbake modulosc and some time (in fact many hours, if starting from scratch) later have a file which I can copy to an SD card for the RPi to boot. To me, this is about the best thing ever for producing a software platform (or "manufacturing the product"). I start with some versions of source code and can reliably and repeatedly produce images which I know will work on the RPi.