Delta support

A tale of two firmware versions

Binary deltas of firmware via fwup has been a supported feature for a while. There has also been a kind of rudimentary support for it in NervesHub. We can now remove the word rudimentary. There is now proper delta support in NervesHub. It is not in a tagged release quite yet but NervesCloud runs off of main and so can you if you would benefit from this.

One thing that prompted finalizing this work was that Frank Hunleth shipped the support for disk encryption combined with deltas for fwup. And since they use encryption this enabled us to make deltas work for SmartRent. I am very excited to see what the results are there.

The PRs that enabled this:

Enable firmware delta delivery
This included making a new library called confuse to parse the libconfuse style of config file that fwup uses. We needed to extract some information from the meta.conf files provided in .fw archives. This let us determine which resources have deltas enabled and which do not and only calculate deltas for the right ones.

This also includes a bag of tests that took a minute to get right to make sure that we produced the expected results with the deltas we generate.

This made deltas work.

If you created a deployment group on a product with deltas enabled it would immediately start making deltas for the firmwares that are in use within that deployment group. And with a bit of luck it all works out. But the control is very coarse-grained.

Coded up and worked through by me with some good review by Josh Kalderimis and Elin Olsson. Lots of help on fwup-related know-how from Frank.

Move delta setting to deployment group
This lets you try deltas with a subset of devices. Thanks Elin for that one and Josh for suggesting it. And subsequently we fixed a bug in that update, thanks Nate Shoemaker.

Add information about available deltas to deployment summary
A first step towards making deltas a first-class citizen is to make it visible that they exist. This makes them show up on the deployment group page.

There is more work to be done.

In progress: Safer delta updates
This is a large-ish PR because it also generalizes a few fwup-specific things and tries to make for a maintainable future.

The important parts are that the Confuse library is now updated to pull out more and better information and to understand which fwup versions allow which features. This means we can disqualify a device from delta updates because it has a fwup version on device that is too old to allow the update to succeed. Then we deliver the full update.

fwup first got support for deltas for "raw" image writes. Then for FAT filesystems. And very recently for encrypted filesystems. When this lands it should mean deltas become an option for pretty much any user of Nerves.

It also add the choice between delta and full update when pushing an update manually. Which is nice.

In progress: Log transfer savings from deltas
Knowing if deltas matter for you and how much they matter is kind of important. And this PR will introduce us tracking that information in and surfacing it in the UI. Showing the savings per transfer and in total.

This can be exciting for a couple of reasons. Maybe you run NervesHub and need smaller updates. Or maybe you don't run NervesHub but want to shrink your updates using deltas. You can use this as a reference implementation for generating valid delta firmware images which is a bit finicky.

There is also a quirk we need to look at in fwup. A delta update causes more reads than writes and this makes the progress bar kind of weird. We know :)

Project updates

The first update is the RPi 5 Hailo AI support. So this is the Raspberry Pi AI Kit hardware. This has been a relay race where I encouraged Gus Workman to start it and then connected him with Cocoa Xu when he hit trouble and then Paulo Valente and Vittoria expressed interest so we pulled them in. Me and Cocoa got it working. Paulo and Vittoria reproduced it and put it to use with Nx and more in their Code BEAM Stockholm talk. And now. Frank has integrated it into the main system.

nerves_system_rpi5, v0.6.1

This is a security/bug fix release that also includes Hailo AI support. Hailo support is currently experimental and won’t be loaded if you don’t have it.

• Package updates
  • Erlang/OTP 27.3.4.1
  • Buildroot 2025.02.3 (fixed 2025.02.2)
  • Raspberry Pi WiFi firmware 1:20240709-2~bpo12+1+rpt3
  • rpi-libcamera v0.5.0+rpt20250429
  • rpicam-apps 1.7.0
  • erlinit 1.14.3
  • fwup 1.13.0

ssh_subsystem_fwup, v0.6.3

This release adds REUSE compliance to improve copyright and licensing accuracy for compliance tools.

• Improvements
  • Fixed a minor Elixir 1.19 warning
  • Removed support for Elixir 1.13 and earlier since they’re no longer regularly tested.

uboot_env, v1.0.2

• Changes
  • Fix Elixir 1.19 warning
  • Update licensing and copyright for REUSE compliance

vintage_net_wifi, v0.12.7

This release makes VintageNet compliant with the REUSE Specification .

• Fixes

  • Removed dependency on GenStateMachine and use :gen_statem directly. There’s no change in functionality, but it will remove the compilation warnings when building the GenStateMachine dependency.
  • Fix Elixir 1.19 warning
    

• Changed

  • Only support Elixir 1.14 and later. It should work with Elixir 1.13 still, but this may change in the future without us noticing. Other libraries are requiring Elixir 1.14 and later, so please upgrade if this affects you.

nerves_system_br, v1.31.3

This is a bug fix update for 1.31.2.

• Package updates
  • erlinit 1.14.3
  • rpi-libcamera v0.5.0+rpt20250429
  • rpicam-apps 1.7.0

Trellis is giving IEx!

I hope all of you are following Gus Workman's efforts with making Nerves-first hardware. He did Soleil which is a cool power-management HAT for Raspberry Pi.

This is the Trellis which is an entire board, an Allwinner T113-S4, a neat industrially capable little chip. Affordable at low volume. And he is making everything open. Frank helped get him booted when there were mystery DRAM quirks and now we are rolling.

Follow the effort on his Blusky .

He is giving a training at ElixirConf US in Orlando. He will be speaking at NervesConf EU... and we also have ... plans.

Participating in the community

The Nerves community is found wherever Elixirists gather. Try any of the following:

Questions are best asked on the Elixir Forum .

Social conversation and banter:

• Elixir Slack , #nerves channel
• Elixir Discord , #nerves channel

How you can help Nerves

Contribute in the way that works for you:

• Send corrections or improvements for documentation wherever it fails to help you.
• Write about Nerves, give talks about Nerves. Make videos about Nerves. It is all good.
• Write or port a new hardware library and include it in the Elixir Circuits collection .
• Get in touch about taking over maintenance duties for some libraries, we might be able to provide you hardware.
• Apply for an EEF stipend on something Nerves-related and build it. We can help if you have questions about this.

Finally, if you have questions about the newsletter or want to suggest something you can simply respond to this email.

- Lars