SCMI: abstracting platform resources using power and performance domains in Linux
We continue to talk about power management in Linux. The traditional System Control and Management Interface (SCMI) approach generates a lot of request traffic between the agent and the platform to configure platform resources, such as clocks, regulators, PHYs, and interconnects every time the system boots. This traffic adds to boot time, which is critical for automotive applications that need to be ready within two seconds after power-up. Reducing traffic can improve boot KPIs (key performance indicators).
Another problem that SCMI can help solve is the effort and complexity of creating and maintaining Linux drivers for peripherals. A single peripheral may require multiple drivers for clocks, regulators, interconnects, GPIO/Pin-control, PHYs, and other frameworks. Developing, testing, troubleshooting, upstreaming and maintaining these drivers is a lot of work, and it has to be repeated for each platform.
A possible solution is to leverage the firmware, which already knows the peripheral devices on its platform, and send configuration requests to it instead of working through Linux drivers.
At the Qualcomm Innovation Center, Inc. (QUIC), we are exploring power management in Linux and this idea by changing the Linux kernel to abstract resources in firmware and use SCMI power and performance protocols to configure peripherals. These changes can reduce traffic and shorten boot time, as well as eliminate complexity and cut the amount of time spent developing peripheral device drivers.
In this post, you will see how our approach moves complexity from the Linux driver (which requires development effort to be programmed differently for each platform) to the firmware. Since the firmware already knows the platform, it knows what it needs to do to configure the peripheral. You will learn how we define logical performance and power domains instead of dealing with each platform resource individually. You will also see how we are upstreaming our efforts to achieve this and how you can try them out and contribute.
(This is a summary of my presentation “Using logical performance and power domains to achieve resource abstraction over SCMI” at Linaro Connect. See below for details and links.)
The complexity of a typical driver
The diagram below depicts the complexity of a peripheral device driver:
A single device may interact with various resources, necessitating a unique driver for each interaction type. The device might be part of a power domain and could experience a reset. It might need to communicate with the clock driver via the clock framework or require interaction with GPIO pin control and regulator drivers. Additionally, it might utilize the PHY driver that also has its resources such as clocks. The peripheral device driver may also engage in bandwidth voting and interact with the interconnect driver through the interconnect framework.
Platform resource drivers further increase the complexity introduced by each peripheral device driver. Our goal is to streamline this process to expedite product launches by reducing the amount of development work involved with resource drivers.
Using SCMI for abstraction
SCMI serves as a system control management interface not bound by any specific OS, featuring the following architecture:
Contained within the orange box, there are two distinct levels:
1. The term protocol refers to the various services offered, including clock management, voltage regulation, and power domain control.
2. Transport represents the method through which two entities interact. One of these entities is the agent that requests services such as Mailbox, SMC/HVC calls, and VirtIO, while the other entity is the platform that provides these services.
The communication between the Platform Controller (also referred to as firmware in this context) and the agent occurs through SCMI protocols using this transport mechanism.
Here is a brief outline of how SCMI is utilized to abstract resources:
The highlighted blue boxes represent the upstream drivers and their related controllers currently in use, meaning these drivers can abstract all resources within the firmware. Consequently, there's no necessity for writing and maintaining a multitude of drivers in Linux. Peripheral drivers can simply issue a request to the firmware through these SCMI drivers for managing resource operations.
It should be noted, though, that the absence of a PHY driver implies that abstraction using SCMI is not feasible for it (and similarly, there is no provision for interconnect yet). Nevertheless, the PHY driver may utilize the same drivers to manage its resources by abstracting them into the firmware.
Hence, utilizing SCMI enables us to reduce the amount of development work required. Despite this, each peripheral must still integrate with a variety of kernel frameworks and subsystems.
What could be the approach to refine this process? We could look to streamline through focusing on power domain, performance domain and reset.
The SCMI performance and power domains
Our approach is to define logical (or modelled) performance and power domains.
Performance domains
The following example is based on this diagram:
Consider that that peripheral has two configurations:
Level-L1: The clock is running at @f1 frequency, interconnect bandwidth is voting for @b1 and the voltage regulator is set to @m1 microvolts.
Level-L2: The clock is running at @f2 frequency, interconnect voting @b1 and voltage regulator set to @m1.
The SCMI perf domain0 comprises those two levels. The driver can send a request to firmware over the performance protocol to set the device to, say, Level-L1. The firmware already knows what Level-L1 means, so it performs resource voting accordingly.
By consolidating resources, this method provides improved abstraction and reduces the quantity of requests sent to the firmware. It broadens the traditional application of performance domains, which usually pertains to computational devices, to include peripherals as well. Performance domains can be specified through functions such as:
- Dynamic clock frequency adjustment (for instance, set rate operations)
- Dynamic allocation of interconnect bandwidth
- Modifications to PHY configurations on-the-fly
- Adjustments to baud rate dynamically (such as in UART)
As outlined, this strategy diverts complexity from the driver—which requires significant development effort to code for each platform—to the firmware. Given that the firmware is already programmed with platform-specific knowledge, it can efficiently manage configurations for various clock speeds, baud rates, or PHY modes.
Power domains
The approach also applies to grouping on- and off-operations in power domains, as shown below:
The power domain could have two operations:
power-on: The clock is set to enable, the regulator is on, the physical power domain is on and PHY is initialized/on.
power-off: All resources are set to the opposite of power-on.
This method clusters binary operations within the SCMI power domain, similar to the performance domain, leading to significantly less firmware communication. Consider a peripheral such as a camera with over 20 clocks; grouping requests rather than submitting 20 individual clock requests saves considerable time.
Code flow in Linux
The flow of our code in Linux looks as follows, with the performance domain on the left and power domain on the right:
For the performance domain, the peripheral device driver uses the Operating Performance Points (OPP) Framework APIs to set the performance level. They call the SCMI performance domain provider driver, scmi_perf_domain.c, which then uses the performance protocol layer to go over the transport to the firmware.
For the power domain, the peripheral device driver uses runtime Power Management (PM) framework APIs. They call the SCMI PM domain provider driver, scmi_pm_domain.c, which uses the power protocol layer and goes over the transport to the firmware.
What’s posted in the kernel so far?
- The SCMI performance driver is available to work with peripheral devices by extending genpd framework (kernel 6.7).
- The OPP framework is extended to work with the SCMI performance driver (kernel 6.7).
- The approach uses both power and performance domains, and there could be several at once. So we need be able to attach and detach manually in the driver instead of using the framework. Helper APIs are now available so that there’s no need to attach/detach consumer drivers manually in case of multiple power-domains (kernel 6.9).
An architecture of firmware-managed resources
The high-level diagram below depicts our architecture on the Qualcomm SA8775P system-on-chip (SoC):
?$QC_Responsive$&fmt=png-alpha)
The SA8775P is a virtualized platform running our Gunyah™ hypervisor software.
To the left, we find the Firmware virtual machine (VM), equipped with a multi-threaded RTOS capable of symmetric multi-processing (SMP). This VM boasts several virtual CPUs and hosts an SCMI server that communicates with back-end resource drivers.
On the right side, the Linux VM functions as agents.
We establish individual SCMI channels for every device within Linux. For instance, UFS has its dedicated shared memory along with a pair of doorbells for receiving (RX) and transmitting (TX). The RX doorbell is converted into interrupt requests (IRQ), while TX is handled by HVC. This setup allows devices to simultaneously send requests to the firmware. With the capability of parallel processing and very fine-grained locking—even when dealing with shared resources—the firmware efficiently manages concurrent requests.
Next steps
Our current progress indicates that the development of the Firmware VM within our architecture is nearing completion. Regarding the Linux VM, modifications to most peripheral device drivers are undergoing internal review.
We have revised or enhanced an SCMI transport driver to be compatible with our hypervisor ABI. These updates have been implemented in kernel 6.6, details of which are available for your review.
For more details, watch my presentation “Using logical performance and power domains to achieve resource abstraction over SCMI” at Linaro Connect 2024. At the end of the video is the Q&A session that followed.
Read other articles about power management in Linux and Qualcomm® Linux® Software Stack:
