Request for Comment about Shipping Multiple Devicetrees: Identifying Devicetree Blobs for your board

If we gave you a brand-new,  device powered by Snapdragon technology booting Linux, would you want to spend time figuring out the right hardware configuration to boot it? Or would you rather just select the right configuration and start using the device?

That's why being able to ship multiple devicetrees is important. Device manufacturers frequently create a single software package diverse enough to ship with multiple boards or SKUs. Included in the software packages are multiple devicetree blobs (DTBs), each describing a different hardware configuration so that the operating system knows what to load.

This post explores our quest for the ideal way of picking the correct DTB (software) for the board (hardware). We want to introduce a common definition for adding board identifiers to devicetrees. My goal in the post is to stimulate further discussion of board-id, a vendor- and OEM-agnostic mechanism by which bootloaders can select the appropriate DTB.

(This is a summary of my presentation “Shipping Multiple Devicetrees: How to Identify Which DTB Is for My Board” at Embedded Open Source Summit. See below for details and links.)

Picking the right devicetree – Background

Devicetrees – or DTBs; the terms are interchangeable – are a community solution to describing hardware. You could embed hardware descriptions into the Linux kernel itself, but they would have to cover all hardware options (sensors, displays, coprocessors, etc.), which would occupy a lot of space. That’s why operating system authors create devicetrees for the hardware they support.

Our Snapdragon X Elite platform has multiple SKUs, each of which has a different devicetree. We want to package all hardware descriptions in the software such that, when you boot a device powered by the Snapdragon X Elite, the right devicetree gets loaded.

But there's a disconnect between choosing which devicetree to use and passing its hardware description to the kernel. Usually, the bootloader does that, but the kernel doesn't own the bootloader, so we have to solve the process of picking.

In short, either you, as OEM or developer, pick the DTB on your own, or else you have some unique, downstream way of picking it.

The current state of the art is that you pick which devicetree to use.  That way, the software package contains only the correct DTB and the bootloader doesn't have to pick. For example, at build time, you run a system command to include the correct devicetree. Or, in the case of a boot image, you provide an argument for the appropriate devicetree to include. Or, you concatenate the kernel image to its DTB.

All other solutions are downstream-based. They involve packaging up multiple DTBs so that the bootloader can pick.

In short, we want to ship multiple devicetrees in a single software package that supports multiple SKUs and reference designs. That will reduce friction when your (and our) customers receive a new board.

Our approaches at Qualcomm Technologies, Inc. (QTI)

How do we at QTI identify different devices? Frankly, after 10 years we still haven’t solved the problem. But here’s what we’ve tried.

Hardware identifier register

In our systems on chip, every SoC has a hardware identifier register that identifies it as Snapdragon X Elite, Snapdragon 8 Gen 3, etc.

Our reference boards are another type. All of our business units – Automotive, IoT, eXtended Reality, etc. – have different reference board platforms; for example, the Qualcomm Robotics RB3 Gen 2 development kit. A partition in persistent storage contains additional identifiers for our reference boards.

Our reference devices are still another type. For our mobile chipsets, for example, we have a reference device that looks like a phone, and it has an identifier.

Those identifiers add up, such that a single software package for QTI could contain about 20 DTBs, with a hundred or more overlays.

Downstream code

We have a solution in our downstream code. In our devicetree we plug in properties like qcom,msm-id, qcom,board-id, qcom,pmic-id and qcom,oem-id to describe how fields map into the configuration data table (CDT) and SoC hardware identifiers:

We tried to upstream that approach many years ago. But the community rejected it because it involves Qualcommspecific properties that the kernel never uses.

Encoding into DT image

There is metadata associated with the packaging format of the DTB, so another option is to encode it into the header of the DT image and identify the hardware that way. But it's non-trivial to manage that metadata if it's not embedded in the file itself.

Upstream approaches

Here are the options that have already been upstreamed.

Use compatible strings

Devicetrees have the concept of compatible strings, which are simple strings of text. So, we tried to describe our hardware following a format like this:

Following that format, you can end up with a string that looks like this:

The problem is that it can take 80 or more characters merely to describe QTI hardware with compatible strings. That complexity makes it difficult to add support for our hardware in popular bootloaders, each of which requires custom matching and maintenance of algorithms. Nevertheless, there’s evidence that this technique is used by vendors like Chromebook, Tegra and maybe others.

Compatible strings often include the SoC. But a DTB that works with, say, one model of Chromebook, may not work with another model of Chromebook, even if they’re powered by the same SoC, say, a Qualcomm SM6115. So relying on the fallback of compatible strings quickly becomes fraught if you don't find a match.

Let the bootloader figure it out

Another approach would be to put all of your nodes into the DT, then let the bootloader figure out what to enable or disable. The problem is that that creates a contract between the bootloader and devicetrees. So, if you're not using, say, U-Boot or an upstream bootloader that relies on Linux, it's not going to work as soon as you want to up-state the devicetree. If the bootloader is expecting the devicetree to have a certain node, but you want the devicetree to not have a node, or you want to add a new node, then fix-ups will start failing. And then what?

Downstream approaches

Most of these are already described above:

• At QTI we use our MSM ID and board ID. It's a bit field containing some magic constants from a region in ROM.
• Google Pixel takes a similar approach, using a field called soc_ID.
• The Android Open Source Project (AOSP) has dtbo.img metadata. It adds an optional ID, revision and u32 or some custom fields for each DTB. As I described above, this is like using the DT image, where you encode the hardware identifiers into the metadata of the DTB image.
• U-Boot uses the filename to match the DTB. It has a property that tests the DTB for a match, similarly to compatible strings.
• coreboot uses a configuration based on a combination of vendor, main board, board id and sku id. We see some promise in that approach, although it’s not yet generic enough to work in every situation, and it's certainly not upstream-standardized.

DTB selection feature – A wish list

First of all, we want to try to lower the effort involved in supporting new boards. We don't want to add custom string parsing that needs to get reviewed every time you have to support a new bootloader. The ideal is to simply lay out the ways to read the board’s properties, then match them.

Next, we should expect that hardware will provide much more information – for example, SoC revision numbers – than the software cares about.

Then, we want the feature to be translatable to different DTB packaging formats. At the moment, U-Boot, Android Bootloader (ABL) and coreboot all use different formats. It would be a sign of progress to make these formats generic and let the kernel be the source of truth.

Finally, we want a documented mechanism for matching these properties so that any conforming DTB will just work, whether on U-Boot, coreboot, grub or ABL.

Next steps

As mentioned in the outset, the goal of the post is to stimulate further discussion on a current RFC called “Adding board-id support for multiple DT selection.” We’d like the community to take a close look at it, then sign up or provide input.

We know we’re not the only vendor affected by the lack of an elegant solution. We also know that the hivemind sees things we may be overlooking. Let us know what you think by contributing on Qualcomm Developer Discord.

And, for more details, see my presentation on this topic from Embedded Open Source Summit 2024, with links to slide decks and video:

• Shipping Multiple Devicetrees: How to Identify Which DTB Is for My Board