1PSCI OS-initiated mode 2====================== 3 4:Author: Maulik Shah & Wing Li 5:Organization: Qualcomm Innovation Center, Inc. & Google LLC 6:Contact: Maulik Shah <quic_mkshah@quicinc.com> & Wing Li <wingers@google.com> 7:Status: RFC 8 9.. contents:: Table of Contents 10 11Introduction 12------------ 13 14Power state coordination 15^^^^^^^^^^^^^^^^^^^^^^^^ 16 17A power domain topology is a logical hierarchy of power domains in a system that 18arises from the physical dependencies between power domains. 19 20Local power states describe power states for an individual node, and composite 21power states describe the combined power states for an individual node and its 22parent node(s). 23 24Entry into low-power states for a topology node above the core level requires 25coordinating its children nodes. For example, in a system with a power domain 26that encompasses a shared cache, and a separate power domain for each core that 27uses the shared cache, the core power domains must be powered down before the 28shared cache power domain can be powered down. 29 30PSCI supports two modes of power state coordination: platform-coordinated and 31OS-initiated. 32 33Platform-coordinated 34~~~~~~~~~~~~~~~~~~~~ 35 36Platform-coordinated mode is the default mode of power state coordination, and 37is currently the only supported mode in TF-A. 38 39In platform-coordinated mode, the platform is responsible for coordinating power 40states, and chooses the deepest power state for a topology node that can be 41tolerated by its children. 42 43OS-initiated 44~~~~~~~~~~~~ 45 46OS-initiated mode is optional. 47 48In OS-initiated mode, the calling OS is responsible for coordinating power 49states, and may request for a topology node to enter a low-power state when 50its last child enters the low-power state. 51 52Motivation 53---------- 54 55There are two reasons why OS-initiated mode might be a more suitable option than 56platform-coordinated mode for a platform. 57 58Scalability 59^^^^^^^^^^^ 60 61In platform-coordinated mode, each core independently selects their own local 62power states, and doesn't account for composite power states that are shared 63between cores. 64 65In OS-initiated mode, the OS has knowledge of the next wakeup event for each 66core, and can have more precise control over the entry, exit, and wakeup 67latencies when deciding if a composite power state (e.g. for a cluster) is 68appropriate. This is especially important for multi-cluster SMP systems and 69heterogeneous systems like big.LITTLE, where different processor types can have 70different power efficiencies. 71 72Simplicity 73^^^^^^^^^^ 74 75In platform-coordinated mode, the OS doesn't have visibility when the last core 76at a power level enters a low-power state. If the OS wants to perform last man 77activity (e.g. powering off a shared resource when it is no longer needed), it 78would have to communicate with an API side channel to know when it can do so. 79This could result in a design smell where the platform is using 80platform-coordinated mode when it should be using OS-initiated mode instead. 81 82In OS-initiated mode, the OS can perform last man activity if it selects a 83composite power state when the last core enters a low-power state. This 84eliminates the need for a side channel, and uses the well documented API between 85the OS and the platform. 86 87Current vendor implementations and workarounds 88^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 89 90* STMicroelectronics 91 92 * For their ARM32 platforms, they're using OS-initiated mode implemented in 93 OP-TEE. 94 * For their future ARM64 platforms, they are interested in using OS-initiated 95 mode in TF-A. 96 97* Qualcomm 98 99 * For their mobile platforms, they're using OS-initiated mode implemented in 100 their own custom secure monitor firmware. 101 * For their Chrome OS platforms, they're using platform-coordinated mode in 102 TF-A with custom driver logic to perform last man activity. 103 104* Google 105 106 * They're using platform-coordinated mode in TF-A with custom driver logic to 107 perform last man activity. 108 109Both Qualcomm and Google would like to be able to use OS-initiated mode in TF-A 110in order to simplify custom driver logic. 111 112Requirements 113------------ 114 115PSCI_FEATURES 116^^^^^^^^^^^^^ 117 118PSCI_FEATURES is for checking whether or not a PSCI function is implemented and 119what its properties are. 120 121.. c:macro:: PSCI_FEATURES 122 123 :param func_id: 0x8400_000A. 124 :param psci_func_id: the function ID of a PSCI function. 125 :retval NOT_SUPPORTED: if the function is not implemented. 126 :retval feature flags associated with the function: if the function is 127 implemented. 128 129CPU_SUSPEND feature flags 130~~~~~~~~~~~~~~~~~~~~~~~~~ 131 132* Reserved, bits[31:2] 133* Power state parameter format, bit[1] 134 135 * A value of 0 indicates the original format is used. 136 * A value of 1 indicates the extended format is used. 137 138* OS-initiated mode, bit[0] 139 140 * A value of 0 indicates OS-initiated mode is not supported. 141 * A value of 1 indicates OS-initiated mode is supported. 142 143See sections 5.1.14 and 5.15 of the PSCI spec (DEN0022D.b) for more details. 144 145PSCI_SET_SUSPEND_MODE 146^^^^^^^^^^^^^^^^^^^^^ 147 148PSCI_SET_SUSPEND_MODE is for switching between the two different modes of power 149state coordination. 150 151.. c:macro:: PSCI_SET_SUSPEND_MODE 152 153 :param func_id: 0x8400_000F. 154 :param mode: 0 indicates platform-coordinated mode, 1 indicates OS-initiated 155 mode. 156 :retval SUCCESS: if the request is successful. 157 :retval NOT_SUPPORTED: if OS-initiated mode is not supported. 158 :retval INVALID_PARAMETERS: if the requested mode is not a valid value (0 or 159 1). 160 :retval DENIED: if the cores are not in the correct state. 161 162Switching from platform-coordinated to OS-initiated is only allowed if the 163following conditions are met: 164 165* All cores are in one of the following states: 166 167 * Running. 168 * Off, through a call to CPU_OFF or not yet booted. 169 * Suspended, through a call to CPU_DEFAULT_SUSPEND. 170 171* None of the cores has called CPU_SUSPEND since the last change of mode or 172 boot. 173 174Switching from OS-initiated to platform-coordinated is only allowed if all cores 175other than the calling core are off, either through a call to CPU_OFF or not yet 176booted. 177 178If these conditions are not met, the PSCI implementation must return DENIED. 179 180See sections 5.1.19 and 5.20 of the PSCI spec (DEN0022D.b) for more details. 181 182CPU_SUSPEND 183^^^^^^^^^^^ 184 185CPU_SUSPEND is for moving a topology node into a low-power state. 186 187.. c:macro:: CPU_SUSPEND 188 189 :param func_id: 0xC400_0001. 190 :param power_state: the requested low-power state to enter. 191 :param entry_point_address: the address at which the core must resume 192 execution following wakeup from a powerdown state. 193 :param context_id: this field specifies a pointer to the saved context that 194 must be restored on a core following wakeup from a powerdown state. 195 :retval SUCCESS: if the request is successful. 196 :retval INVALID_PARAMETERS: in OS-initiated mode, this error is returned when 197 a low-power state is requested for a topology node above the core level, 198 and at least one of the node's children is in a local low-power state 199 that is incompatible with the request. 200 :retval INVALID_ADDRESS: if the entry_point_address argument is invalid. 201 :retval DENIED: only in OS-initiated mode; this error is returned when a 202 low-power state is requested for a topology node above the core level, 203 and at least one of the node's children is running, i.e. not in a 204 low-power state. 205 206In platform-coordinated mode, the PSCI implementation coordinates requests from 207all cores to determine the deepest power state to enter. 208 209In OS-initiated mode, the calling OS is making an explicit request for a 210specific power state, as opposed to expressing a vote. The PSCI implementation 211must comply with the request, unless the request is not consistent with the 212implementation's view of the system's state, in which case, the implementation 213must return INVALID_PARAMETERS or DENIED. 214 215See sections 5.1.2 and 5.4 of the PSCI spec (DEN0022D.b) for more details. 216 217Power state formats 218~~~~~~~~~~~~~~~~~~~ 219 220Original format 221 222* Power Level, bits[25:24] 223 224 * The requested level in the power domain topology to enter a low-power 225 state. 226 227* State Type, bit[16] 228 229 * A value of 0 indicates a standby or retention state. 230 * A value of 1 indicates a powerdown state. 231 232* State ID, bits[15:0] 233 234 * Field to specify the requested composite power state. 235 * The state ID encodings must uniquely describe every possible composite 236 power state. 237 * In OS-initiated mode, the state ID encoding must allow expressing the 238 power level at which the calling core is the last to enter a powerdown 239 state. 240 241Extended format 242 243* State Type, bit[30] 244* State ID, bits[27:0] 245 246Races in OS-initiated mode 247~~~~~~~~~~~~~~~~~~~~~~~~~~ 248 249In OS-initiated mode, there are race windows where the OS's view and 250implementation's view of the system's state differ. It is possible for the OS to 251make requests that are invalid given the implementation's view of the system's 252state. For example, the OS might request a powerdown state for a node from one 253core, while at the same time, the implementation observes that another core in 254that node is powering up. 255 256To address potential race conditions in power state requests: 257 258* The calling OS must specify in each CPU_SUSPEND request the deepest power 259 level for which it sees the calling core as the last running core (last man). 260 This is required even if the OS doesn't want the node at that power level to 261 enter a low-power state. 262* The implementation must validate that the requested power states in the 263 CPU_SUSPEND request are consistent with the system's state, and that the 264 calling core is the last core running at the requested power level, or deny 265 the request otherwise. 266 267See sections 4.2.3.2, 6.2, and 6.3 of the PSCI spec (DEN0022D.b) for more 268details. 269 270Caveats 271------- 272 273CPU_OFF 274^^^^^^^ 275 276CPU_OFF is always platform-coordinated, regardless of whether the power state 277coordination mode for suspend is platform-coordinated or OS-initiated. If all 278cores in a topology node call CPU_OFF, the last core will power down the node. 279 280In OS-initiated mode, if a subset of the cores in a topology node has called 281CPU_OFF, the last running core may call CPU_SUSPEND to request a powerdown state 282at or above that node's power level. 283 284See section 5.5.2 of the PSCI spec (DEN0022D.b) for more details. 285 286Implementation 287-------------- 288 289Current implementation of platform-coordinated mode 290^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 291 292Platform-coordinated is currently the only supported power state coordination 293mode in TF-A. 294 295The functions of interest in the ``psci_cpu_suspend`` call stack are as follows: 296 297* ``psci_validate_power_state`` 298 299 * This function calls a platform specific ``validate_power_state`` handler, 300 which takes the ``power_state`` parameter, and updates the ``state_info`` 301 object with the requested states for each power level. 302 303* ``psci_find_target_suspend_lvl`` 304 305 * This function takes the ``state_info`` object containing the requested power 306 states for each power level, and returns the deepest power level that was 307 requested to enter a low power state, i.e. the target power level. 308 309* ``psci_do_state_coordination`` 310 311 * This function takes the target power level and the ``state_info`` object 312 containing the requested power states for each power level, and updates the 313 ``state_info`` object with the coordinated target power state for each 314 level. 315 316* ``pwr_domain_suspend`` 317 318 * This is a platform specific handler that takes the ``state_info`` object 319 containing the target power states for each power level, and transitions 320 each power level to the specified power state. 321 322Proposed implementation of OS-initiated mode 323^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 324 325To add support for OS-initiated mode, the following changes are proposed: 326 327* Add a boolean build option ``PSCI_OS_INIT_MODE`` for a platform to enable 328 optional support for PSCI OS-initiated mode. This build option defaults to 0. 329 330.. note:: 331 332 If ``PSCI_OS_INIT_MODE=0``, the following changes will not be compiled into 333 the build. 334 335* Update ``psci_features`` to return 1 in bit[0] to indicate support for 336 OS-initiated mode for CPU_SUSPEND. 337* Define a ``suspend_mode`` enum: ``PLAT_COORD`` and ``OS_INIT``. 338* Define a ``psci_suspend_mode`` global variable with a default value of 339 ``PLAT_COORD``. 340* Implement a new function handler ``psci_set_suspend_mode`` for 341 PSCI_SET_SUSPEND_MODE. 342* Since ``psci_validate_power_state`` calls a platform specific 343 ``validate_power_state`` handler, the platform implementation should populate 344 the ``state_info`` object based on the state ID from the given ``power_state`` 345 parameter. 346* ``psci_find_target_suspend_lvl`` remains unchanged. 347* Implement a new function ``psci_validate_state_coordination`` that ensures the 348 request satisfies the following conditions, and denies any requests 349 that don't: 350 351 * The requested power states for each power level are consistent with the 352 system's state 353 * The calling core is the last core running at the requested power level 354 355 This function differs from ``psci_do_state_coordination`` in that: 356 357 * The ``psci_req_local_pwr_states`` map is not modified if the request were to 358 be denied 359 * The ``state_info`` argument is never modified since it contains the power 360 states requested by the calling OS 361 362* Update ``psci_cpu_suspend_start`` to do the following: 363 364 * If ``PSCI_SUSPEND_MODE`` is ``PLAT_COORD``, call 365 ``psci_do_state_coordination``. 366 * If ``PSCI_SUSPEND_MODE`` is ``OS_INIT``, call 367 ``psci_validate_state_coordination``. If validation fails, propagate the 368 error up the call stack. 369 370* Update the return type of the platform specific ``pwr_domain_suspend`` 371 handler from ``void`` to ``int``, to allow the platform to optionally perform 372 validations based on hardware states. 373 374.. image:: ../resources/diagrams/psci-osi-mode.png 375 376Testing 377------- 378 379The proposed patches can be found at 380https://review.trustedfirmware.org/q/topic:psci-osi. 381 382Testing on FVP and Google platforms 383^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 384 385The proposed patches add a new CPU Suspend in OSI mode test suite to TF-A Tests. 386This has been enabled and verified on the FVP_Base_RevC-2xAEMvA platform and 387Google platforms, and excluded from all other platforms via the build option 388``PLAT_TESTS_SKIP_LIST``. 389 390Testing on STM32MP15 391^^^^^^^^^^^^^^^^^^^^ 392 393The proposed patches have been tested and verified on the STM32MP15 platform, 394which has a single cluster with 2 CPUs, by Gabriel Fernandez 395<gabriel.fernandez@st.com> from STMicroelectronics with this device tree 396configuration: 397 398.. code-block:: devicetree 399 400 cpus { 401 #address-cells = <1>; 402 #size-cells = <0>; 403 404 cpu0: cpu@0 { 405 device_type = "cpu"; 406 compatible = "arm,cortex-a7"; 407 reg = <0>; 408 enable-method = "psci"; 409 power-domains = <&CPU_PD0>; 410 power-domain-names = "psci"; 411 }; 412 cpu1: cpu@1 { 413 device_type = "cpu"; 414 compatible = "arm,cortex-a7"; 415 reg = <1>; 416 enable-method = "psci"; 417 power-domains = <&CPU_PD1>; 418 power-domain-names = "psci"; 419 }; 420 421 idle-states { 422 cpu_retention: cpu-retention { 423 compatible = "arm,idle-state"; 424 arm,psci-suspend-param = <0x00000001>; 425 entry-latency-us = <130>; 426 exit-latency-us = <620>; 427 min-residency-us = <700>; 428 local-timer-stop; 429 }; 430 }; 431 432 domain-idle-states { 433 CLUSTER_STOP: core-power-domain { 434 compatible = "domain-idle-state"; 435 arm,psci-suspend-param = <0x01000001>; 436 entry-latency-us = <230>; 437 exit-latency-us = <720>; 438 min-residency-us = <2000>; 439 local-timer-stop; 440 }; 441 }; 442 }; 443 444 psci { 445 compatible = "arm,psci-1.0"; 446 method = "smc"; 447 448 CPU_PD0: power-domain-cpu0 { 449 #power-domain-cells = <0>; 450 power-domains = <&pd_core>; 451 domain-idle-states = <&cpu_retention>; 452 }; 453 454 CPU_PD1: power-domain-cpu1 { 455 #power-domain-cells = <0>; 456 power-domains = <&pd_core>; 457 domain-idle-states = <&cpu_retention>; 458 }; 459 460 pd_core: power-domain-cluster { 461 #power-domain-cells = <0>; 462 domain-idle-states = <&CLUSTER_STOP>; 463 }; 464 }; 465 466Testing on Qualcomm SC7280 467^^^^^^^^^^^^^^^^^^^^^^^^^^ 468 469The proposed patches have been tested and verified on the SC7280 platform by 470Maulik Shah <quic_mkshah@quicinc.com> from Qualcomm with this device tree 471configuration: 472 473.. code-block:: devicetree 474 475 cpus { 476 #address-cells = <2>; 477 #size-cells = <0>; 478 479 CPU0: cpu@0 { 480 device_type = "cpu"; 481 compatible = "arm,kryo"; 482 reg = <0x0 0x0>; 483 enable-method = "psci"; 484 power-domains = <&CPU_PD0>; 485 power-domain-names = "psci"; 486 }; 487 488 CPU1: cpu@100 { 489 device_type = "cpu"; 490 compatible = "arm,kryo"; 491 reg = <0x0 0x100>; 492 enable-method = "psci"; 493 power-domains = <&CPU_PD1>; 494 power-domain-names = "psci"; 495 }; 496 497 CPU2: cpu@200 { 498 device_type = "cpu"; 499 compatible = "arm,kryo"; 500 reg = <0x0 0x200>; 501 enable-method = "psci"; 502 power-domains = <&CPU_PD2>; 503 power-domain-names = "psci"; 504 }; 505 506 CPU3: cpu@300 { 507 device_type = "cpu"; 508 compatible = "arm,kryo"; 509 reg = <0x0 0x300>; 510 enable-method = "psci"; 511 power-domains = <&CPU_PD3>; 512 power-domain-names = "psci"; 513 } 514 515 CPU4: cpu@400 { 516 device_type = "cpu"; 517 compatible = "arm,kryo"; 518 reg = <0x0 0x400>; 519 enable-method = "psci"; 520 power-domains = <&CPU_PD4>; 521 power-domain-names = "psci"; 522 }; 523 524 CPU5: cpu@500 { 525 device_type = "cpu"; 526 compatible = "arm,kryo"; 527 reg = <0x0 0x500>; 528 enable-method = "psci"; 529 power-domains = <&CPU_PD5>; 530 power-domain-names = "psci"; 531 }; 532 533 CPU6: cpu@600 { 534 device_type = "cpu"; 535 compatible = "arm,kryo"; 536 reg = <0x0 0x600>; 537 enable-method = "psci"; 538 power-domains = <&CPU_PD6>; 539 power-domain-names = "psci"; 540 }; 541 542 CPU7: cpu@700 { 543 device_type = "cpu"; 544 compatible = "arm,kryo"; 545 reg = <0x0 0x700>; 546 enable-method = "psci"; 547 power-domains = <&CPU_PD7>; 548 power-domain-names = "psci"; 549 }; 550 551 idle-states { 552 entry-method = "psci"; 553 554 LITTLE_CPU_SLEEP_0: cpu-sleep-0-0 { 555 compatible = "arm,idle-state"; 556 idle-state-name = "little-power-down"; 557 arm,psci-suspend-param = <0x40000003>; 558 entry-latency-us = <549>; 559 exit-latency-us = <901>; 560 min-residency-us = <1774>; 561 local-timer-stop; 562 }; 563 564 LITTLE_CPU_SLEEP_1: cpu-sleep-0-1 { 565 compatible = "arm,idle-state"; 566 idle-state-name = "little-rail-power-down"; 567 arm,psci-suspend-param = <0x40000004>; 568 entry-latency-us = <702>; 569 exit-latency-us = <915>; 570 min-residency-us = <4001>; 571 local-timer-stop; 572 }; 573 574 BIG_CPU_SLEEP_0: cpu-sleep-1-0 { 575 compatible = "arm,idle-state"; 576 idle-state-name = "big-power-down"; 577 arm,psci-suspend-param = <0x40000003>; 578 entry-latency-us = <523>; 579 exit-latency-us = <1244>; 580 min-residency-us = <2207>; 581 local-timer-stop; 582 }; 583 584 BIG_CPU_SLEEP_1: cpu-sleep-1-1 { 585 compatible = "arm,idle-state"; 586 idle-state-name = "big-rail-power-down"; 587 arm,psci-suspend-param = <0x40000004>; 588 entry-latency-us = <526>; 589 exit-latency-us = <1854>; 590 min-residency-us = <5555>; 591 local-timer-stop; 592 }; 593 }; 594 595 domain-idle-states { 596 CLUSTER_SLEEP_0: cluster-sleep-0 { 597 compatible = "arm,idle-state"; 598 idle-state-name = "cluster-power-down"; 599 arm,psci-suspend-param = <0x40003444>; 600 entry-latency-us = <3263>; 601 exit-latency-us = <6562>; 602 min-residency-us = <9926>; 603 local-timer-stop; 604 }; 605 }; 606 }; 607 608 psci { 609 compatible = "arm,psci-1.0"; 610 method = "smc"; 611 612 CPU_PD0: cpu0 { 613 #power-domain-cells = <0>; 614 power-domains = <&CLUSTER_PD>; 615 domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>; 616 }; 617 618 CPU_PD1: cpu1 { 619 #power-domain-cells = <0>; 620 power-domains = <&CLUSTER_PD>; 621 domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>; 622 }; 623 624 CPU_PD2: cpu2 { 625 #power-domain-cells = <0>; 626 power-domains = <&CLUSTER_PD>; 627 domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>; 628 }; 629 630 CPU_PD3: cpu3 { 631 #power-domain-cells = <0>; 632 power-domains = <&CLUSTER_PD>; 633 domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>; 634 }; 635 636 CPU_PD4: cpu4 { 637 #power-domain-cells = <0>; 638 power-domains = <&CLUSTER_PD>; 639 domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>; 640 }; 641 642 CPU_PD5: cpu5 { 643 #power-domain-cells = <0>; 644 power-domains = <&CLUSTER_PD>; 645 domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>; 646 }; 647 648 CPU_PD6: cpu6 { 649 #power-domain-cells = <0>; 650 power-domains = <&CLUSTER_PD>; 651 domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>; 652 }; 653 654 CPU_PD7: cpu7 { 655 #power-domain-cells = <0>; 656 power-domains = <&CLUSTER_PD>; 657 domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>; 658 }; 659 660 CLUSTER_PD: cpu-cluster0 { 661 #power-domain-cells = <0>; 662 domain-idle-states = <&CLUSTER_SLEEP_0>; 663 }; 664 }; 665 666Comparisons on Qualcomm SC7280 667^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 668 669CPUIdle states 670~~~~~~~~~~~~~~ 671 672* 8 CPUs, 1 L3 cache 673* Platform-coordinated mode 674 675 * CPUIdle states 676 677 * State0 - WFI 678 * State1 - Core collapse 679 * State2 - Rail collapse 680 * State3 - L3 cache off and system resources voted off 681 682* OS-initiated mode 683 684 * CPUIdle states 685 686 * State0 - WFI 687 * State1 - Core collapse 688 * State2 - Rail collapse 689 690 * Cluster domain idle state 691 692 * State3 - L3 cache off and system resources voted off 693 694.. image:: ../resources/diagrams/psci-flattened-vs-hierarchical-idle-states.png 695 696Results 697~~~~~~~ 698 699* The following stats have been captured with fixed CPU frequencies from the use 700 case of 10 seconds of device idle with the display turned on and Wi-Fi and 701 modem turned off. 702* Count refers to the number of times a CPU or cluster entered power collapse. 703* Residency refers to the time in seconds a CPU or cluster stayed in power 704 collapse. 705* The results are an average of 3 iterations of actual counts and residencies. 706 707.. image:: ../resources/diagrams/psci-pc-mode-vs-osi-mode.png 708 709OS-initiated mode was able to scale better than platform-coordinated mode for 710multiple CPUs. The count and residency results for state3 (i.e. a cluster domain 711idle state) in OS-initiated mode for multiple CPUs were much closer to the 712results for a single CPU than in platform-coordinated mode. 713 714-------------- 715 716*Copyright (c) 2023, Arm Limited and Contributors. All rights reserved.* 717
