1.. SPDX-License-Identifier: GPL-2.0 2.. include:: <isonum.txt> 3 4============================== 5Intel Uncore Frequency Scaling 6============================== 7 8:Copyright: |copy| 2022-2023 Intel Corporation 9 10:Author: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> 11 12Introduction 13------------ 14 15The uncore can consume significant amount of power in Intel's Xeon servers based 16on the workload characteristics. To optimize the total power and improve overall 17performance, SoCs have internal algorithms for scaling uncore frequency. These 18algorithms monitor workload usage of uncore and set a desirable frequency. 19 20It is possible that users have different expectations of uncore performance and 21want to have control over it. The objective is similar to allowing users to set 22the scaling min/max frequencies via cpufreq sysfs to improve CPU performance. 23Users may have some latency sensitive workloads where they do not want any 24change to uncore frequency. Also, users may have workloads which require 25different core and uncore performance at distinct phases and they may want to 26use both cpufreq and the uncore scaling interface to distribute power and 27improve overall performance. 28 29Sysfs Interface 30--------------- 31 32To control uncore frequency, a sysfs interface is provided in the directory: 33`/sys/devices/system/cpu/intel_uncore_frequency/`. 34 35There is one directory for each package and die combination as the scope of 36uncore scaling control is per die in multiple die/package SoCs or per 37package for single die per package SoCs. The name represents the 38scope of control. For example: 'package_00_die_00' is for package id 0 and 39die 0. 40 41Each package_*_die_* contains the following attributes: 42 43``initial_max_freq_khz`` 44 Out of reset, this attribute represent the maximum possible frequency. 45 This is a read-only attribute. If users adjust max_freq_khz, 46 they can always go back to maximum using the value from this attribute. 47 48``initial_min_freq_khz`` 49 Out of reset, this attribute represent the minimum possible frequency. 50 This is a read-only attribute. If users adjust min_freq_khz, 51 they can always go back to minimum using the value from this attribute. 52 53``max_freq_khz`` 54 This attribute is used to set the maximum uncore frequency. 55 56``min_freq_khz`` 57 This attribute is used to set the minimum uncore frequency. 58 59``current_freq_khz`` 60 This attribute is used to get the current uncore frequency. 61 62SoCs with TPMI (Topology Aware Register and PM Capsule Interface) 63----------------------------------------------------------------- 64 65An SoC can contain multiple power domains with individual or collection 66of mesh partitions. This partition is called fabric cluster. 67 68Certain type of meshes will need to run at the same frequency, they will 69be placed in the same fabric cluster. Benefit of fabric cluster is that it 70offers a scalable mechanism to deal with partitioned fabrics in a SoC. 71 72The current sysfs interface supports controls at package and die level. 73This interface is not enough to support more granular control at 74fabric cluster level. 75 76SoCs with the support of TPMI (Topology Aware Register and PM Capsule 77Interface), can have multiple power domains. Each power domain can 78contain one or more fabric clusters. 79 80To represent controls at fabric cluster level in addition to the 81controls at package and die level (like systems without TPMI 82support), sysfs is enhanced. This granular interface is presented in the 83sysfs with directories names prefixed with "uncore". For example: 84uncore00, uncore01 etc. 85 86The scope of control is specified by attributes "package_id", "domain_id" 87and "fabric_cluster_id" in the directory. 88 89Attributes in each directory: 90 91``domain_id`` 92 This attribute is used to get the power domain id of this instance. 93 94``fabric_cluster_id`` 95 This attribute is used to get the fabric cluster id of this instance. 96 97``package_id`` 98 This attribute is used to get the package id of this instance. 99 100The other attributes are same as presented at package_*_die_* level. 101 102In most of current use cases, the "max_freq_khz" and "min_freq_khz" 103is updated at "package_*_die_*" level. This model will be still supported 104with the following approach: 105 106When user uses controls at "package_*_die_*" level, then every fabric 107cluster is affected in that package and die. For example: user changes 108"max_freq_khz" in the package_00_die_00, then "max_freq_khz" for uncore* 109directory with the same package id will be updated. In this case user can 110still update "max_freq_khz" at each uncore* level, which is more restrictive. 111Similarly, user can update "min_freq_khz" at "package_*_die_*" level 112to apply at each uncore* level. 113 114Support for "current_freq_khz" is available only at each fabric cluster 115level (i.e., in uncore* directory). 116 117Efficiency vs. Latency Tradeoff 118------------------------------- 119 120The Efficiency Latency Control (ELC) feature improves performance 121per watt. With this feature hardware power management algorithms 122optimize trade-off between latency and power consumption. For some 123latency sensitive workloads further tuning can be done by SW to 124get desired performance. 125 126The hardware monitors the average CPU utilization across all cores 127in a power domain at regular intervals and decides an uncore frequency. 128While this may result in the best performance per watt, workload may be 129expecting higher performance at the expense of power. Consider an 130application that intermittently wakes up to perform memory reads on an 131otherwise idle system. In such cases, if hardware lowers uncore 132frequency, then there may be delay in ramp up of frequency to meet 133target performance. 134 135The ELC control defines some parameters which can be changed from SW. 136If the average CPU utilization is below a user-defined threshold 137(elc_low_threshold_percent attribute below), the user-defined uncore 138floor frequency will be used (elc_floor_freq_khz attribute below) 139instead of hardware calculated minimum. 140 141Similarly in high load scenario where the CPU utilization goes above 142the high threshold value (elc_high_threshold_percent attribute below) 143instead of jumping to maximum uncore frequency, frequency is increased 144in 100MHz steps. This avoids consuming unnecessarily high power 145immediately with CPU utilization spikes. 146 147Attributes for efficiency latency control: 148 149``elc_floor_freq_khz`` 150 This attribute is used to get/set the efficiency latency floor frequency. 151 If this variable is lower than the 'min_freq_khz', it is ignored by 152 the firmware. 153 154``elc_low_threshold_percent`` 155 This attribute is used to get/set the efficiency latency control low 156 threshold. This attribute is in percentages of CPU utilization. 157 158``elc_high_threshold_percent`` 159 This attribute is used to get/set the efficiency latency control high 160 threshold. This attribute is in percentages of CPU utilization. 161 162``elc_high_threshold_enable`` 163 This attribute is used to enable/disable the efficiency latency control 164 high threshold. Write '1' to enable, '0' to disable. 165 166Example system configuration below, which does following: 167 * when CPU utilization is less than 10%: sets uncore frequency to 800MHz 168 * when CPU utilization is higher than 95%: increases uncore frequency in 169 100MHz steps, until power limit is reached 170 171 elc_floor_freq_khz:800000 172 elc_high_threshold_percent:95 173 elc_high_threshold_enable:1 174 elc_low_threshold_percent:10 175