systemd plan: Factory Reset, Stateless Systems, Reproducible Systems & Verifiable Systems

Factory Reset, Stateless Systems, Reproducible Systems & Verifiable Systems.

Taking benefit of the /usr merge that a number of distributions have completed we want to bring runtime behaviour of Linux systems to the next level. With the /usr merge completed most static vendor-supplied OS data is found exclusively in /usr, only a few additional bits in /var and /etc are necessary to make a system boot. On this we can build to enable a couple of new features:

1. A mechanism we call Factory Reset shall flush out /etc and /var, but keep the vendor-supplied /usr, bringing the system back into a well-defined, pristine vendor state with no local state or configuration. This functionality is useful across the board from servers, to desktops, to embedded devices.

2. A Stateless System goes one step further: a system like this never stores /etc or /var on persistent storage, but always comes up with pristine vendor state. On systems like this every reboot acts as factor reset. This functionality is particularly useful for simple containers or systems that boot off the network or read-only media, and receive all configuration they need during runtime from vendor packages or protocols like DHCP or are capable of discovering their parameters automatically from the available hardware or periphery.

3. Reproducible Systems multiply a vendor image into many containers or systems. Only local configuration or state is stored per-system, while the vendor operating system is pulled in from the same, immutable, shared snapshot. Each system hence has its private /etc and /var for receiving local configuration, however the OS tree in /usr is pulled in via bind mounts (in case of containers) or technologies like NFS (in case of physical systems), or btrfs snapshots from a golden master image. This is particular interesting for containers where the goal is to run thousands of container images from the same OS tree. However, it also has a number of other usecases, for example thin client systems, which can boot the same NFS share a number of times. Furthermore this mechanism is useful to implement very simple OS installers, that simply unserialize a /usr snapshot into a file system, install a boot loader, and reboot.

4. Verifiable Systems are closely related to stateless systems: if the underlying storage technology can cryptographically ensure that the vendor-supplied OS is trusted and in a consistent state, then it must be made sure that /etc or /var are either included in the OS image, or simply unnecessary for booting.

Followed by Concepts, Problems, What's Next and Conclusions.

Note: To follow further systemd developments see Lennart Pottering's Google+[/ur] page.

-- Tom

 

(1) can already be done, with shell scripts no less. If you want to use it as a security feature you're barking up the wrong tree anyway, you should be using VM provisioning and snapshots; once a Linux system is rooted, you can never guarantee that a restore done by root will be clean.

(2) again, astonishingly easy to do with shell scripts. If you want to get fancy, throw in symlinks and tmpfs and cron jobs. Really fancy, use Puppet or something.

(3) so, like LXC built into the init system? Don't we already have... LXC initscripts?

(4) like a live CD that uses filesystem images with signed checksums. Only... Fancy.

Hear that grinding sound? I think that's wheels being reinvented.

Edit: to be clear I'm not much of a systemd fan. From what I've seen so far, it has added a lot of complexity and not delivered on most of its promises. Likewise I've only gotten pain and misery from Pulseaudio, and from polkit/consolekit, etc. etc.

Believe me, I appreciate very much that simpler is not always better. And I recognize that Poettering is a better programmer than I will ever be. But more complex is not necessarily better either, and even highly expert programmers can go barking up the wrong tree.

 

Hi Gullible,

While I understand where you are coming from (slow and steady - if it works don't fix it), systemd adds parallelism to the init process which has been sorely needed since multi-cores have more or less (forever hence) replaced the notion of single core processsors in one computer - and systemd is the only way to speed up the boot process with multi-cores.

All new ideas like systemd need fleshing out in order to work better. The systemd criticism you have read, has been for the most part refuted by Pottering and you might need to read his rebuttal to it all. Here is a comparison of init systems by one of the creators of systemd.

The shell scripts you refer to (I bet) have not been parallelized. The reason I state that is that I once worked in parallel processing and our parallel OS group (both sysV and Mach microkernel) was very good at doing that sort of thing. We had a brilliant parallel init process for all the hardware written by one of the hardware engineers - the code was very elegant and simple to understand.

Yes, the wheel has been redesigned - with parallelism in mind, sort of the way we do everything on computers nowadays (speaking from a systems point of view).

-- Tom

 

Maybe I'm not understanding this, because I really don't see the need; and also because, on the desktop, systemd has utterly failed to deliver.

For multi-socket servers, my experience (which could be biased, mind) has been that the most time intensive bootup tasks are network services, which are either
a) very large single-threaded programs that do not benefit from parallelism
or
b) very large multithreaded programs that already make use of parallelism

In any case we're migrating away from multi-socket servers being used for one task; many high-end x86-64 machines from what I've seen run Xen/VMware/etc. on the bare metal, and have small VMs provisioned per task.

On desktops OTOH, boot time is pretty trivial, and from what I have seen does not benefit a lot from parallelism. If anything most distros' boot speeds have hit rock bottom since the introduction of systemd, due to the addition of new services. Basically I think the boot process is something that will benefit more from simplicity and sane defaults than parallelism. My 2c anyway.

I have seen that chart many times. I can't evaluate (or, in some cases, fully understand) the utility of every feature mentioned. I can only say what I've personally seen; which is lots of software churn, some impressively slow boot times, and nothing significant changing in userspace (aside from the inevitable churn).

I'm wondering what the benefit of parallism is in this case? If you're wiping stuff from disk storage, even SSD, parallelism should not help much with performance.

I missed the part about restoring /etc to a known configuration on boot, though. That would be useful, and more difficult to do cleanly; it would have to be done very early in the boot process, I think. But again I don't see why it demands systemd's huge changes, instead of some modifications of existing initscripts.