1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+--------------------------------------+ 10| CoT | Chain of Trust | 11+--------+--------------------------------------+ 12| DMA | Direct Memory Access | 13+--------+--------------------------------------+ 14| DTB | Device Tree Blob | 15+--------+--------------------------------------+ 16| DTS | Device Tree Source | 17+--------+--------------------------------------+ 18| EC | Execution Context | 19+--------+--------------------------------------+ 20| FIP | Firmware Image Package | 21+--------+--------------------------------------+ 22| FF-A | Firmware Framework for Arm A-profile | 23+--------+--------------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+--------------------------------------+ 26| NWd | Normal World | 27+--------+--------------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+--------------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+--------------------------------------+ 32| PA | Physical Address | 33+--------+--------------------------------------+ 34| PE | Processing Element | 35+--------+--------------------------------------+ 36| PM | Power Management | 37+--------+--------------------------------------+ 38| PVM | Primary VM | 39+--------+--------------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+--------------------------------------+ 42| SP | Secure Partition | 43+--------+--------------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+--------------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+--------------------------------------+ 48| SPMC | SPM Core | 49+--------+--------------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+--------------------------------------+ 52| SiP | Silicon Provider | 53+--------+--------------------------------------+ 54| SWd | Secure World | 55+--------+--------------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+--------------------------------------+ 58| TOS | Trusted Operating System | 59+--------+--------------------------------------+ 60| VM | Virtual Machine | 61+--------+--------------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1 or S-EL2: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 139 restoring) the EL2 system register context before entering (resp. 140 after leaving) the SPMC. It is mandatorily enabled when 141 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 142 and exhaustive list of registers is visible at `[4]`_. 143- **SP_LAYOUT_FILE**: this option specifies a text description file 144 providing paths to SP binary images and manifests in DTS format 145 (see `Describing secure partitions`_). It 146 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 147 secure partitions are to be loaded on behalf of the SPMC. 148 149+---------------+----------------------+------------------+ 150| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 151+---------------+----------------------+------------------+ 152| SPMC at S-EL1 | 0 | 0 | 153+---------------+----------------------+------------------+ 154| SPMC at S-EL2 | 1 | 1 (default when | 155| | | SPD=spmd) | 156+---------------+----------------------+------------------+ 157 158Other combinations of such build options either break the build or are not 159supported. 160 161Notes: 162 163- Only Arm's FVP platform is supported to use with the TF-A reference software 164 stack. 165- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 166 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 167 and S-EL2. 168- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 169 barely saving/restoring EL2 registers from an Arm arch perspective. As such 170 it is decoupled from the ``SPD=spmd`` option. 171- BL32 option is re-purposed to specify the SPMC image. It can specify either 172 the Hafnium binary path (built for the secure world) or the path to a TEE 173 binary implementing FF-A interfaces. 174- BL33 option can specify the TFTF binary or a normal world loader 175 such as U-Boot or the UEFI framework. 176 177Sample TF-A build command line when SPMC is located at S-EL1 178(e.g. when the FEAT_EL2 architecture extension is not implemented): 179 180.. code:: shell 181 182 make \ 183 CROSS_COMPILE=aarch64-none-elf- \ 184 SPD=spmd \ 185 SPMD_SPM_AT_SEL2=0 \ 186 BL32=<path-to-tee-binary> \ 187 BL33=<path-to-bl33-binary> \ 188 PLAT=fvp \ 189 all fip 190 191Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 192located at S-EL2: 193 194.. code:: shell 195 196 make \ 197 CROSS_COMPILE=aarch64-none-elf- \ 198 PLAT=fvp \ 199 SPD=spmd \ 200 CTX_INCLUDE_EL2_REGS=1 \ 201 ARM_ARCH_MINOR=5 \ 202 BRANCH_PROTECTION=1 \ 203 CTX_INCLUDE_PAUTH_REGS=1 \ 204 BL32=<path-to-hafnium-binary> \ 205 BL33=<path-to-bl33-binary> \ 206 SP_LAYOUT_FILE=sp_layout.json \ 207 all fip 208 209Same as above with enabling secure boot in addition: 210 211.. code:: shell 212 213 make \ 214 CROSS_COMPILE=aarch64-none-elf- \ 215 PLAT=fvp \ 216 SPD=spmd \ 217 CTX_INCLUDE_EL2_REGS=1 \ 218 ARM_ARCH_MINOR=5 \ 219 BRANCH_PROTECTION=1 \ 220 CTX_INCLUDE_PAUTH_REGS=1 \ 221 BL32=<path-to-hafnium-binary> \ 222 BL33=<path-to-bl33-binary> \ 223 SP_LAYOUT_FILE=sp_layout.json \ 224 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 225 TRUSTED_BOARD_BOOT=1 \ 226 COT=dualroot \ 227 ARM_ROTPK_LOCATION=devel_rsa \ 228 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 229 GENERATE_COT=1 \ 230 all fip 231 232FVP model invocation 233==================== 234 235The FVP command line needs the following options to exercise the S-EL2 SPMC: 236 237+---------------------------------------------------+------------------------------------+ 238| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 239| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 240+---------------------------------------------------+------------------------------------+ 241| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 242| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 243| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 244| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 245| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 246| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 247| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 248| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 249+---------------------------------------------------+------------------------------------+ 250| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 251| - cluster1.has_branch_target_exception=1 | | 252+---------------------------------------------------+------------------------------------+ 253| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 254| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 255+---------------------------------------------------+------------------------------------+ 256 257Sample FVP command line invocation: 258 259.. code:: shell 260 261 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 262 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 263 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 264 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 265 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 266 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 267 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 268 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 269 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 270 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 271 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 272 -C cluster0.has_branch_target_exception=1 \ 273 -C cluster1.has_branch_target_exception=1 \ 274 -C cluster0.restriction_on_speculative_execution=2 \ 275 -C cluster1.restriction_on_speculative_execution=2 276 277Boot process 278============ 279 280Loading Hafnium and secure partitions in the secure world 281--------------------------------------------------------- 282 283TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 284 285SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 286Thus they are supplied as distinct signed entities within the FIP flash 287image. The FIP image itself is not signed hence this provides the ability 288to upgrade SPs in the field. 289 290Booting through TF-A 291-------------------- 292 293SP manifests 294~~~~~~~~~~~~ 295 296An SP manifest describes SP attributes as defined in `[1]`_ 297(partition manifest at virtual FF-A instance) in DTS format. It is 298represented as a single file associated with the SP. A sample is 299provided by `[5]`_. A binding document is provided by `[6]`_. 300 301Secure Partition packages 302~~~~~~~~~~~~~~~~~~~~~~~~~ 303 304Secure partitions are bundled as independent package files consisting 305of: 306 307- a header 308- a DTB 309- an image payload 310 311The header starts with a magic value and offset values to SP DTB and 312image payload. Each SP package is loaded independently by BL2 loader 313and verified for authenticity and integrity. 314 315The SP package identified by its UUID (matching FF-A uuid property) is 316inserted as a single entry into the FIP at end of the TF-A build flow 317as shown: 318 319.. code:: shell 320 321 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 322 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 323 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 324 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 325 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 326 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 327 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 328 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 329 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 330 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 331 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 332 333.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 334 335Describing secure partitions 336~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 337 338A json-formatted description file is passed to the build flow specifying paths 339to the SP binary image and associated DTS partition manifest file. The latter 340is processed by the dtc compiler to generate a DTB fed into the SP package. 341This file also specifies the SP owner (as an optional field) identifying the 342signing domain in case of dual root CoT. 343The SP owner can either be the silicon or the platform provider. The 344corresponding "owner" field value can either take the value of "SiP" or "Plat". 345In absence of "owner" field, it defaults to "SiP" owner. 346 347.. code:: shell 348 349 { 350 "tee1" : { 351 "image": "tee1.bin", 352 "pm": "tee1.dts", 353 "owner": "SiP" 354 }, 355 356 "tee2" : { 357 "image": "tee2.bin", 358 "pm": "tee2.dts", 359 "owner": "Plat" 360 } 361 } 362 363SPMC manifest 364~~~~~~~~~~~~~ 365 366This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 367time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 368two different cases: 369 370- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 371 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 372 mode. 373- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 374 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 375 S-EL0. 376 377.. code:: shell 378 379 attribute { 380 spmc_id = <0x8000>; 381 maj_ver = <0x1>; 382 min_ver = <0x0>; 383 exec_state = <0x0>; 384 load_address = <0x0 0x6000000>; 385 entrypoint = <0x0 0x6000000>; 386 binary_size = <0x60000>; 387 }; 388 389- *spmc_id* defines the endpoint ID value that SPMC can query through 390 ``FFA_ID_GET``. 391- *maj_ver/min_ver*. SPMD checks provided version versus its internal 392 version and aborts if not matching. 393- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 394 Notice Hafnium used as a SPMC only supports AArch64. 395- *load_address* and *binary_size* are mostly used to verify secondary 396 entry points fit into the loaded binary image. 397- *entrypoint* defines the cold boot primary core entry point used by 398 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 399 400Other nodes in the manifest are consumed by Hafnium in the secure world. 401A sample can be found at [7]: 402 403- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 404 indicates a FF-A compliant SP. The *load_address* field specifies the load 405 address at which TF-A loaded the SP package. 406- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 407 Note the primary core is declared first, then secondary core are declared 408 in reverse order. 409- The *memory* node provides platform information on the ranges of memory 410 available to the SPMC. 411 412SPMC boot 413~~~~~~~~~ 414 415The SPMC is loaded by BL2 as the BL32 image. 416 417The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 418 419BL2 passes the SPMC manifest address to BL31 through a register. 420 421At boot time, the SPMD in BL31 runs from the primary core, initializes the core 422contexts and launches the SPMC (BL32) passing the following information through 423registers: 424 425- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 426- X1 holds the ``HW_CONFIG`` physical address. 427- X4 holds the currently running core linear id. 428 429Loading of SPs 430~~~~~~~~~~~~~~ 431 432At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 433below: 434 435.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 436 437Note this boot flow is an implementation sample on Arm's FVP platform. 438Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 439different implementation. 440 441Secure boot 442~~~~~~~~~~~ 443 444The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 445SPMC manifest, secure partitions and verifies them for authenticity and integrity. 446Refer to TBBR specification `[3]`_. 447 448The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 449the use of two root keys namely S-ROTPK and NS-ROTPK: 450 451- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 452- BL33 may be signed by the OEM using NS-ROTPK. 453- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 454 455Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 456 457Hafnium in the secure world 458=========================== 459 460General considerations 461---------------------- 462 463Build platform for the secure world 464~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 465 466In the Hafnium reference implementation specific code parts are only relevant to 467the secure world. Such portions are isolated in architecture specific files 468and/or enclosed by a ``SECURE_WORLD`` macro. 469 470Secure partitions CPU scheduling 471~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 472 473The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 474secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 475 476- the FFA_MSG_SEND_DIRECT_REQ interface. 477- the FFA_RUN interface. 478 479Platform topology 480~~~~~~~~~~~~~~~~~ 481 482The *execution-ctx-count* SP manifest field can take the value of one or the 483total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 484following SP types: 485 486- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 487 implement the same number of ECs as the number of PEs in the platform. 488- Migratable UP SPs: a single execution context can run and be migrated on any 489 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 490 receive a direct message request originating from any physical core targeting 491 the single execution context. 492 493Parsing SP partition manifests 494------------------------------ 495 496Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 497Note the current implementation may not implement all optional fields. 498 499The SP manifest may contain memory and device regions nodes. In case of 500an S-EL2 SPMC: 501 502- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 503 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 504 specify RX/TX buffer regions in which case it is not necessary for an SP 505 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 506- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 507 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 508 additional resources (e.g. interrupts). 509 510For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 511provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 512regime. 513 514Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 515same set of page tables. It is still open whether two sets of page tables shall 516be provided per SP. The memory region node as defined in the specification 517provides a memory security attribute hinting to map either to the secure or 518non-secure EL1&0 Stage-2 table if it exists. 519 520Passing boot data to the SP 521--------------------------- 522 523In `[1]`_ , the "Protocol for passing data" section defines a method for passing 524boot data to SPs (not currently implemented). 525 526Provided that the whole secure partition package image (see 527`Secure Partition packages`_) is mapped to the SP secure EL1&0 Stage-2 528translation regime, an SP can access its own manifest DTB blob and extract its 529partition manifest properties. 530 531SP Boot order 532------------- 533 534SP manifests provide an optional boot order attribute meant to resolve 535dependencies such as an SP providing a service required to properly boot 536another SP. 537 538It is possible for an SP to call into another SP through a direct request 539provided the latter SP has already been booted. 540 541Boot phases 542----------- 543 544Primary core boot-up 545~~~~~~~~~~~~~~~~~~~~ 546 547Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 548core. The SPMC performs its platform initializations and registers the SPMC 549secondary physical core entry point physical address by the use of the 550`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 551at secure physical FF-A instance). 552 553The SPMC then creates secure partitions based on SP packages and manifests. Each 554secure partition is launched in sequence (`SP Boot order`_) on their "primary" 555execution context. If the primary boot physical core linear id is N, an MP SP is 556started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 557UP SP, it is started using its unique EC0 on PE[N]. 558 559The SP primary EC (or the EC used when the partition is booted as described 560above): 561 562- Performs the overall SP boot time initialization, and in case of a MP SP, 563 prepares the SP environment for other execution contexts. 564- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 565 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 566 entry point for other execution contexts. 567- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 568 ``FFA_ERROR`` in case of failure. 569 570Secondary cores boot-up 571~~~~~~~~~~~~~~~~~~~~~~~ 572 573Once the system is started and NWd brought up, a secondary physical core is 574woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 575calls into the SPMD on the newly woken up physical core. Then the SPMC is 576entered at the secondary physical core entry point. 577 578In the current implementation, the first SP is resumed on the coresponding EC 579(the virtual CPU which matches the physical core). The implication is that the 580first SP must be a MP SP. 581 582In a linux based system, once secure and normal worlds are booted but prior to 583a NWd FF-A driver has been loaded: 584 585- The first SP has initialized all its ECs in response to primary core boot up 586 (at system initialization) and secondary core boot up (as a result of linux 587 invoking PSCI_CPU_ON for all secondary cores). 588- Other SPs have their first execution context initialized as a result of secure 589 world initialization on the primary boot core. Other ECs for those SPs have to 590 be run first through ffa_run to complete their initialization (which results 591 in the EC completing with FFA_MSG_WAIT). 592 593Refer to `Power management`_ for further details. 594 595Notifications 596------------- 597 598The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 599communication mechanism with non-blocking semantics. It allows for one FF-A 600endpoint to signal another for service provision, without hindering its current 601progress. 602 603Hafnium currently supports 64 notifications. The IDs of each notification define 604a position in a 64-bit bitmap. 605 606The signaling of notifications can interchangeably happen between NWd and SWd 607FF-A endpoints. 608 609The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 610VMs, and from VMs to SPs. An hypervisor component would only manage 611notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 612deployed in NWd, the Hypervisor or OS kernel must invoke the interface 613FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 614endpoint in the NWd that supports it. 615 616A sender can signal notifications once the receiver has provided it with 617permissions. Permissions are provided by invoking the interface 618FFA_NOTIFICATION_BIND. 619 620Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 621they are considered to be in a pending sate. The receiver can retrieve its 622pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 623are considered to be handled. 624 625Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 626that is in charge of donating CPU cycles for notifications handling. The 627FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 628which FF-A endpoints have pending notifications. The receiver scheduler is 629called and informed by the FF-A driver, and it should allocate CPU cycles to the 630receiver. 631 632There are two types of notifications supported: 633- Global, which are targeted to a FF-A endpoint and can be handled within any of 634its execution contexts, as determined by the scheduler of the system. 635- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 636a specific execution context, as determined by the sender. 637 638The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 639permissions to the sender. 640 641Notification signaling resorts to two interrupts: 642- Schedule Receiver Interrupt: Non-secure physical interrupt to be handled by 643the FF-A 'transport' driver within the receiver scheduler. At initialization 644the SPMC (as suggested by the spec) configures a secure SGI, as non-secure, and 645triggers it when there are pending notifications, and the respective receivers 646need CPU cycles to handle them. 647- Notifications Pending Interrupt: Virtual Interrupt to be handled by the 648receiver of the notification. Set when there are pending notifications. For 649per-vCPU the NPI is pended at the handling of FFA_NOTIFICATION_SET interface. 650 651The notifications receipt support is enabled in the partition FF-A manifest. 652 653The subsequent section provides more details about the each one of the 654FF-A interfaces for notifications support. 655 656Mandatory interfaces 657-------------------- 658 659The following interfaces are exposed to SPs: 660 661- ``FFA_VERSION`` 662- ``FFA_FEATURES`` 663- ``FFA_RX_RELEASE`` 664- ``FFA_RXTX_MAP`` 665- ``FFA_RXTX_UNMAP`` 666- ``FFA_PARTITION_INFO_GET`` 667- ``FFA_ID_GET`` 668- ``FFA_MSG_WAIT`` 669- ``FFA_MSG_SEND_DIRECT_REQ`` 670- ``FFA_MSG_SEND_DIRECT_RESP`` 671- ``FFA_MEM_DONATE`` 672- ``FFA_MEM_LEND`` 673- ``FFA_MEM_SHARE`` 674- ``FFA_MEM_RETRIEVE_REQ`` 675- ``FFA_MEM_RETRIEVE_RESP`` 676- ``FFA_MEM_RELINQUISH`` 677- ``FFA_MEM_RECLAIM`` 678 679As part of the support of FF-A v1.1, the following interfaces were added: 680 681 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 682 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 683 - ``FFA_NOTIFICATION_BIND`` 684 - ``FFA_NOTIFICATION_UNBIND`` 685 - ``FFA_NOTIFICATION_SET`` 686 - ``FFA_NOTIFICATION_GET`` 687 - ``FFA_NOTIFICATION_INFO_GET`` 688 - ``FFA_SPM_ID_GET`` 689 - ``FFA_SECONDARY_EP_REGISTER`` 690 691FFA_VERSION 692~~~~~~~~~~~ 693 694``FFA_VERSION`` requires a *requested_version* parameter from the caller. 695The returned value depends on the caller: 696 697- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 698 specified in the SPMC manifest. 699- SP: the SPMC returns its own implemented version. 700- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 701 702FFA_FEATURES 703~~~~~~~~~~~~ 704 705FF-A features supported by the SPMC may be discovered by secure partitions at 706boot (that is prior to NWd is booted) or run-time. 707 708The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 709FFA_SUCCESS from the SPMD. 710 711The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 712the response relayed back to the NWd. 713 714FFA_RXTX_MAP/FFA_RXTX_UNMAP 715~~~~~~~~~~~~~~~~~~~~~~~~~~~ 716 717When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 718receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 719regime as secure buffers in the MMU descriptors. 720 721When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 722SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 723descriptors. 724 725The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 726caller, either it being the Hypervisor or OS kernel, as well as a secure 727partition. 728 729FFA_PARTITION_INFO_GET 730~~~~~~~~~~~~~~~~~~~~~~ 731 732Partition info get call can originate: 733 734- from SP to SPMC 735- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 736 737FFA_ID_GET 738~~~~~~~~~~ 739 740The FF-A id space is split into a non-secure space and secure space: 741 742- FF-A ID with bit 15 clear relates to VMs. 743- FF-A ID with bit 15 set related to SPs. 744- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 745 and SPMC. 746 747The SPMD returns: 748 749- The default zero value on invocation from the Hypervisor. 750- The ``spmc_id`` value specified in the SPMC manifest on invocation from 751 the SPMC (see `SPMC manifest`_) 752 753This convention helps the SPMC to determine the origin and destination worlds in 754an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 755transactions in its world switch routine. It must not be permitted for a VM to 756use a secure FF-A ID as origin world by spoofing: 757 758- A VM-to-SP direct request/response shall set the origin world to be non-secure 759 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 760 set). 761- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 762 for both origin and destination IDs. 763 764An incoming direct message request arriving at SPMD from NWd is forwarded to 765SPMC without a specific check. The SPMC is resumed through eret and "knows" the 766message is coming from normal world in this specific code path. Thus the origin 767endpoint ID must be checked by SPMC for being a normal world ID. 768 769An SP sending a direct message request must have bit 15 set in its origin 770endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 771 772The SPMC shall reject the direct message if the claimed world in origin endpoint 773ID is not consistent: 774 775- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 776 world ID", 777- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 778 779 780FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 781~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 782 783This is a mandatory interface for secure partitions consisting in direct request 784and responses with the following rules: 785 786- An SP can send a direct request to another SP. 787- An SP can receive a direct request from another SP. 788- An SP can send a direct response to another SP. 789- An SP cannot send a direct request to an Hypervisor or OS kernel. 790- An Hypervisor or OS kernel can send a direct request to an SP. 791- An SP can send a direct response to an Hypervisor or OS kernel. 792 793FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 794~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 795 796The secure partitions notifications bitmap are statically allocated by the SPMC. 797Hence, this interface is not to be issued by secure partitions. 798 799At initialization, the SPMC is not aware of VMs/partitions deployed in the 800normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 801to be prepared to handle notifications for the provided VM ID. 802 803FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 804~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 805 806Pair of interfaces to manage permissions to signal notifications. Prior to 807handling notifications, an FF-A endpoint must allow a given sender to signal a 808bitmap of notifications. 809 810If the receiver doesn't have notification support enabled in its FF-A manifest, 811it won't be able to bind notifications, hence forbidding it to receive any 812notifications. 813 814FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 815~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 816 817If the notifications set are per-vCPU, the NPI interrupt is set as pending 818for a given receiver partition. 819 820The FFA_NOTIFICATION_GET will retrieve all pending global notifications and all 821pending per-vCPU notifications targeted to the current vCPU. 822 823Hafnium keeps the global counting of the pending notifications, which is 824incremented and decremented at the handling of FFA_NOTIFICATION_SET and 825FFA_NOTIFICATION_GET, respectively. If the counter reaches zero, prior to SPMC 826triggering the SRI, it won't be triggered. 827 828FFA_NOTIFICATION_INFO_GET 829~~~~~~~~~~~~~~~~~~~~~~~~~ 830 831Hafnium keeps the global counting of pending notifications whose info has been 832retrieved by this interface. The counting is incremented and decremented at the 833handling of FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET, respectively. 834It also tracks the notifications whose info has been retrieved individually, 835such that it avoids duplicating returned information for subsequent calls to 836FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 837reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 838 839FFA_SPM_ID_GET 840~~~~~~~~~~~~~~ 841 842Returns the FF-A ID allocated to the SPM component (which includes SPMC + SPMD). 843At initialization, the SPMC queries the SPMD for the SPM ID, using this 844same interface, and saves it. 845 846The call emitted at NS and secure physical FF-A instances returns the SPM ID 847specified in the SPMC manifest. 848 849Secure partitions call this interface at the virtual instance, to which the SPMC 850shall return the priorly retrieved SPM ID. 851 852The Hypervisor or OS kernel can issue an FFA_SPM_ID_GET call handled by the 853SPMD, which returns the SPM ID. 854 855FFA_SECONDARY_EP_REGISTER 856~~~~~~~~~~~~~~~~~~~~~~~~~ 857 858When the SPMC boots, all secure partitions are initialized on their primary 859Execution Context. 860 861The interface FFA_SECONDARY_EP_REGISTER is to be used by a secure partitions 862from its first execution context, to provide the entry point address for 863secondary execution contexts. 864 865A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 866the NWd or by invocation of FFA_RUN. 867 868SPMC-SPMD direct requests/responses 869----------------------------------- 870 871Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 872Using those IDs in source/destination fields of a direct request/response 873permits SPMD to SPMC communication and either way. 874 875- SPMC to SPMD direct request/response uses SMC conduit. 876- SPMD to SPMC direct request/response uses ERET conduit. 877 878PE MMU configuration 879-------------------- 880 881With secure virtualization enabled, two IPA spaces are output from the secure 882EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 883hardware is fed by: 884 885- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 886- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 887 enabled. 888 889``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 890NS/S IPA translations. 891``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 892 893- Stage-2 translations for the NS IPA space access the NS PA space. 894- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 895 896Secure and non-secure IPA regions use the same set of Stage-2 page tables within 897a SP. 898 899Interrupt management 900-------------------- 901 902GIC ownership 903~~~~~~~~~~~~~ 904 905The SPMC owns the GIC configuration. Secure and non-secure interrupts are 906trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 907IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 908virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 909 910Non-secure interrupt handling 911~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 912 913The following illustrate the scenarios of non secure physical interrupts trapped 914by the SPMC: 915 916- The SP handles a managed exit operation: 917 918.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 919 920- The SP is pre-empted without managed exit: 921 922.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 923 924Secure interrupt handling 925------------------------- 926 927This section documents the support implemented for secure interrupt handling in 928SPMC as per the guidance provided by FF-A v1.1 Beta0 specification. 929The following assumptions are made about the system configuration: 930 931 - In the current implementation, S-EL1 SPs are expected to use the para 932 virtualized ABIs for interrupt management rather than accessing virtual GIC 933 interface. 934 - Unless explicitly stated otherwise, this support is applicable only for 935 S-EL1 SPs managed by SPMC. 936 - Secure interrupts are configured as G1S or G0 interrupts. 937 - All physical interrupts are routed to SPMC when running a secure partition 938 execution context. 939 940A physical secure interrupt could preempt normal world execution. Moreover, when 941the execution is in secure world, it is highly likely that the target of a 942secure interrupt is not the currently running execution context of an SP. It 943could be targeted to another FF-A component. Consequently, secure interrupt 944management depends on the state of the target execution context of the SP that 945is responsible for handling the interrupt. Hence, the spec provides guidance on 946how to signal start and completion of secure interrupt handling as discussed in 947further sections. 948 949Secure interrupt signaling mechanisms 950~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 951 952Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 953context that it has a pending virtual interrupt and to further run the SP 954execution context, such that it can handle the virtual interrupt. SPMC uses 955either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 956to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 957the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 958running in S-EL2. 959 960+-----------+---------+---------------+---------------------------------------+ 961| SP State | Conduit | Interface and | Description | 962| | | parameters | | 963+-----------+---------+---------------+---------------------------------------+ 964| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 965| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 966| | | | resumes execution context of SP | 967| | | | through ERET. | 968+-----------+---------+---------------+---------------------------------------+ 969| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 970| | vIRQ | | is pending. It pends vIRQ signal and | 971| | | | resumes execution context of SP | 972| | | | through ERET. | 973+-----------+---------+---------------+---------------------------------------+ 974| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 975| | | | not resume execution context of SP. | 976+-----------+---------+---------------+---------------------------------------+ 977| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 978| | vIRQ | | execution context of SP through ERET. | 979+-----------+---------+---------------+---------------------------------------+ 980 981Secure interrupt completion mechanisms 982~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 983 984A SP signals secure interrupt handling completion to the SPMC through the 985following mechanisms: 986 987 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 988 - ``FFA_RUN`` ABI if its was in BLOCKED state. 989 990In the current implementation, S-EL1 SPs use para-virtualized HVC interface 991implemented by SPMC to perform priority drop and interrupt deactivation (we 992assume EOImode = 0, i.e. priority drop and deactivation are done together). 993 994If normal world execution was preempted by secure interrupt, SPMC uses 995FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 996and further return execution to normal world. If the current SP execution 997context was preempted by a secure interrupt to be handled by execution context 998of target SP, SPMC resumes current SP after signal completion by target SP 999execution context. 1000 1001An action is broadly a set of steps taken by the SPMC in response to a physical 1002interrupt. In order to simplify the design, the current version of secure 1003interrupt management support in SPMC (Hafnium) does not fully implement the 1004Scheduling models and Partition runtime models. However, the current 1005implementation loosely maps to the following actions that are legally allowed 1006by the specification. Please refer to the Table 8.4 in the spec for further 1007description of actions. The action specified for a type of interrupt when the 1008SP is in the message processing running state cannot be less permissive than the 1009action specified for the same type of interrupt when the SP is in the interrupt 1010handling running state. 1011 1012+--------------------+--------------------+------------+-------------+ 1013| Runtime Model | NS-Int | Self S-Int | Other S-Int | 1014+--------------------+--------------------+------------+-------------+ 1015| Message Processing | Signalable with ME | Signalable | Signalable | 1016+--------------------+--------------------+------------+-------------+ 1017| Interrupt Handling | Queued | Queued | Queued | 1018+--------------------+--------------------+------------+-------------+ 1019 1020Abbreviations: 1021 1022 - NS-Int: A Non-secure physical interrupt. It requires a switch to the Normal 1023 world to be handled. 1024 - Other S-Int: A secure physical interrupt targeted to an SP different from 1025 the one that is currently running. 1026 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1027 running. 1028 1029The following figure describes interrupt handling flow when secure interrupt 1030triggers while in normal world: 1031 1032.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1033 1034A brief description of the events: 1035 1036 - 1) Secure interrupt triggers while normal world is running. 1037 - 2) FIQ gets trapped to EL3. 1038 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1039 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1040 vIRQ). 1041 - 5) Since SP1 vCPU is in WAITING state, SPMC signals using FFA_INTERRUPT with 1042 interrupt id as argument and resume it using ERET. 1043 - 6) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1044 masked i.e., PSTATE.I = 0 1045 - 7) SP1 services the interrupt and invokes the de-activation HVC call. 1046 - 8) SPMC does internal state management and further de-activates the physical 1047 interrupt and resumes SP vCPU. 1048 - 9) SP performs secure interrupt completion through FFA_MSG_WAIT ABI. 1049 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1050 - 11) EL3 resumes normal world execution. 1051 1052The following figure describes interrupt handling flow when secure interrupt 1053triggers while in secure world: 1054 1055.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1056 1057A brief description of the events: 1058 1059 - 1) Secure interrupt triggers while SP2 is running and SP1 is blocked. 1060 - 2) Gets trapped to SPMC as IRQ. 1061 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1062 this secure interrupt. In this scenario, it is SP1. 1063 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1064 SPMC further resumes SP1 through ERET conduit. 1065 - 5) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1066 masked i.e., PSTATE.I = 0 1067 - 6) SP1 services the secure interrupt and invokes the de-activation HVC call. 1068 - 7) SPMC does internal state management, de-activates the physical interrupt 1069 and resumes SP1 vCPU. 1070 - 8) Assuming SP1 is in BLOCKED state, SP1 performs secure interrupt completion 1071 through FFA_RUN ABI. 1072 - 9) SPMC resumes the pre-empted vCPU of SP2. 1073 1074 1075Power management 1076---------------- 1077 1078In platforms with or without secure virtualization: 1079 1080- The NWd owns the platform PM policy. 1081- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1082- The EL3 PSCI library is in charge of the PM coordination and control 1083 (eventually writing to platform registers). 1084- While coordinating PM events, the PSCI library calls backs into the Secure 1085 Payload Dispatcher for events the latter has statically registered to. 1086 1087When using the SPMD as a Secure Payload Dispatcher: 1088 1089- A power management event is relayed through the SPD hook to the SPMC. 1090- In the current implementation only cpu on (svc_on_finish) and cpu off 1091 (svc_off) hooks are registered. 1092- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1093 The SPMC is entered through its secondary physical core entry point. 1094- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 1095 the PM event is conveyed to the SPMC is implementation-defined in context of 1096 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 1097 direct request/response conveying the PM event details and SPMC response. 1098 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1099 updates its internal state to reflect the physical core is being turned off. 1100 In the current implementation no SP is resumed as a consequence. This behavior 1101 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1102 userspace. 1103 1104SMMUv3 support in Hafnium 1105========================= 1106 1107An SMMU is analogous to an MMU in a CPU. It performs address translations for 1108Direct Memory Access (DMA) requests from system I/O devices. 1109The responsibilities of an SMMU include: 1110 1111- Translation: Incoming DMA requests are translated from bus address space to 1112 system physical address space using translation tables compliant to 1113 Armv8/Armv7 VMSA descriptor format. 1114- Protection: An I/O device can be prohibited from read, write access to a 1115 memory region or allowed. 1116- Isolation: Traffic from each individial device can be independently managed. 1117 The devices are differentiated from each other using unique translation 1118 tables. 1119 1120The following diagram illustrates a typical SMMU IP integrated in a SoC with 1121several I/O devices along with Interconnect and Memory system. 1122 1123.. image:: ../resources/diagrams/MMU-600.png 1124 1125SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1126support for SMMUv3 driver in both normal and secure world. A brief introduction 1127of SMMUv3 functionality and the corresponding software support in Hafnium is 1128provided here. 1129 1130SMMUv3 features 1131--------------- 1132 1133- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1134 translation support. It can either bypass or abort incoming translations as 1135 well. 1136- Traffic (memory transactions) from each upstream I/O peripheral device, 1137 referred to as Stream, can be independently managed using a combination of 1138 several memory based configuration structures. This allows the SMMUv3 to 1139 support a large number of streams with each stream assigned to a unique 1140 translation context. 1141- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1142 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1143 are supported by SMMUv3. 1144- SMMUv3 offers non-secure stream support with secure stream support being 1145 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1146 instance for secure and non-secure stream support. 1147- It also supports sub-streams to differentiate traffic from a virtualized 1148 peripheral associated with a VM/SP. 1149- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1150 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1151 for providing Secure Stage2 translation support to upstream peripheral 1152 devices. 1153 1154SMMUv3 Programming Interfaces 1155----------------------------- 1156 1157SMMUv3 has three software interfaces that are used by the Hafnium driver to 1158configure the behaviour of SMMUv3 and manage the streams. 1159 1160- Memory based data strutures that provide unique translation context for 1161 each stream. 1162- Memory based circular buffers for command queue and event queue. 1163- A large number of SMMU configuration registers that are memory mapped during 1164 boot time by Hafnium driver. Except a few registers, all configuration 1165 registers have independent secure and non-secure versions to configure the 1166 behaviour of SMMUv3 for translation of secure and non-secure streams 1167 respectively. 1168 1169Peripheral device manifest 1170-------------------------- 1171 1172Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1173These devices are dependent on PE endpoint to initiate and receive memory 1174management transactions on their behalf. The acccess to the MMIO regions of 1175any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1176uses the same stage 2 translations for the device as those used by partition 1177manager on behalf of the PE endpoint. This ensures that the peripheral device 1178has the same visibility of the physical address space as the endpoint. The 1179device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1180) must specify these additional properties for each peripheral device in the 1181system : 1182 1183- smmu-id: This field helps to identify the SMMU instance that this device is 1184 upstream of. 1185- stream-ids: List of stream IDs assigned to this device. 1186 1187.. code:: shell 1188 1189 smmuv3-testengine { 1190 base-address = <0x00000000 0x2bfe0000>; 1191 pages-count = <32>; 1192 attributes = <0x3>; 1193 smmu-id = <0>; 1194 stream-ids = <0x0 0x1>; 1195 interrupts = <0x2 0x3>, <0x4 0x5>; 1196 exclusive-access; 1197 }; 1198 1199SMMUv3 driver limitations 1200------------------------- 1201 1202The primary design goal for the Hafnium SMMU driver is to support secure 1203streams. 1204 1205- Currently, the driver only supports Stage2 translations. No support for 1206 Stage1 or nested translations. 1207- Supports only AArch64 translation format. 1208- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1209 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1210- No support for independent peripheral devices. 1211 1212S-EL0 Partition support 1213========================= 1214The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1215FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1216with ARMv8.4 and FEAT_SEL2). 1217 1218S-EL0 partitions are useful for simple partitions that don't require full 1219Trusted OS functionality. It is also useful to reduce jitter and cycle 1220stealing from normal world since they are more lightweight than VMs. 1221 1222S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1223the SPMC. They are differentiated primarily by the 'exception-level' property 1224and the 'execution-ctx-count' property in the SP manifest. They are host apps 1225under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1226call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1227can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1228for memory regions. 1229 1230S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1231capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1232a S-EL0 partition to accept a direct message from secure world and normal world, 1233and generate direct responses to them. 1234 1235Memory sharing between and with S-EL0 partitions is supported. 1236Indirect messaging, Interrupt handling and Notifications are not supported with 1237S-EL0 partitions and is work in progress, planned for future releases. 1238All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not 1239supported. 1240 1241 1242References 1243========== 1244 1245.. _[1]: 1246 1247[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1248 1249.. _[2]: 1250 1251[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1252 1253.. _[3]: 1254 1255[3] `Trusted Boot Board Requirements 1256Client <https://developer.arm.com/documentation/den0006/d/>`__ 1257 1258.. _[4]: 1259 1260[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1261 1262.. _[5]: 1263 1264[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1265 1266.. _[6]: 1267 1268[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1269 1270.. _[7]: 1271 1272[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1273 1274.. _[8]: 1275 1276[8] https://lists.trustedfirmware.org/pipermail/tf-a/2020-February/000296.html 1277 1278.. _[9]: 1279 1280[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1281 1282-------------- 1283 1284*Copyright (c) 2020-2021, Arm Limited and Contributors. All rights reserved.* 1285
