risc-v/mmu: Configure T-Head MMU to cache User Text, Data and Heap

[lupyuen]

Summary

This PR configures the T-Head MMU to cache the the User Text, Data and Heap. We enable the MMU Flags for Shareable, Bufferable and Cacheable, as explained in this article.

This PR fixes the Slow Memory Access for NuttX Apps on BL808 and SG2000 SoCs: apache#12696. With this fix, SG2000 NuttX CoreMark jumps from 21 to 2,422. (Close to SG2000 Debian CoreMark)

We introduce a Kconfig Option: ARCH_MMU_EXT_THEAD ("System Type > Enable T-Head MMU extension support"). Enabling this Kconfig Option will configure the T-Head MMU to cache the User Text, Data and Heap.

This PR enables the MMU cache for only SG2000 SoC (Milk-V Duo S SBC). The next PR will apply the same settings to BL808 SoC (Pine64 Ox64 SBC).

Modified Files

arch/risc-v/Kconfig: Added Kconfig Option ARCH_MMU_EXT_THEAD that will configure the T-Head MMU. Enabled ARCH_MMU_EXT_THEAD for SG2000 SoC.

arch/risc-v/src/common/riscv_mmu.h: Set the T-Head MMU Flags (Shareable, Bufferable and Cacheable) for User Text, Data and Heap, if ARCH_MMU_EXT_THEAD is enabled

arch/risc-v/src/common/riscv_addrenv.c: Extended the MMU Flags from 32 bits to 64 bits, to accommodate the T-Head MMU Flags

arch/risc-v/src/common/riscv_exception.c: Extended the MMU Flags from 32 bits to 64 bit, to accommodate the T-Head MMU Flags. This code is enabled only for MMU Paging (CONFIG_PAGING).

Impact

This PR affects only the platforms that have ARCH_MMU_EXT_THEAD enabled. Right now, only SG2000 SoC enables ARCH_MMU_EXT_THEAD.

Testing

We tested on Milk-V Duo S SBC with SG2000 SoC:

Before the PR: CoreMark is 17 (NuttX Log)

NuttShell (NSH) NuttX-12.4.0
nsh> uname -a
NuttX 12.4.0 cc8d453b5a7 Aug 27 2024 10:25:24 risc-v milkv_duos
nsh> coremark
CoreMark 1.0 : 17.104738 / GCC13.2.0 -Os -fno-strict-aliasing -fomit-frame-pointer -ffunction-sections -fdata-sections -nostdlib -g / HEAP

After the PR: CoreMark increases to 1,759 (NuttX Log)

NuttShell (NSH) NuttX-12.4.0
nsh> uname -a
NuttX 12.4.0 f055ff95c70 Aug 27 2024 10:20:29 risc-v milkv_duos
nsh> coremark
CoreMark 1.0 : 1759.014952 / GCC13.2.0 -Os -fno-strict-aliasing -fomit-frame-pointer -ffunction-sections -fdata-sections -nostdlib -g / HEAP

And OSTest completes successfully:

nsh> ostest
ostest_main: Exiting with status 0

Note that CoreMark will increase further to 2,422 when we compile CoreMark optimised with -O2

Regression Testing: We tested OSTest on QEMU RISC-V 32-bit and 64-bit (Flat Build and Kernel Build):

$ tools/configure.sh rv-virt:nsh / nsh64 / knsh / knsh64
...
nsh> uname -a
NuttX 12.4.0 f055ff95c7 Aug 27 2024 03:39:35 risc-v rv-virt
nsh> ostest
ostest_main: Exiting with status 0

• rv-virt:nsh Log
• rv-virt:nsh64 Log
• rv-virt:knsh Log
• rv-virt:knsh64 Log

MMU Paging: With CONFIG_PAGING enabled, rv-virt:knsh_paging compiles correctly. However we were unable to test knsh_paging. OSTest failed with the same error, before and after our PR:

$ tools/configure.sh rv-virt:knsh_paging
...
nsh> ostest
barrier_test: Thread 6 created
riscv_fillpage: EXCEPTION: Store/AMO page fault. MCAUSE: 0000000f, EPC: 8020ce96, MTVAL: 80c00000
riscv_fillpage: PANIC!!! virtual address not mappable: 80c00000
_assert: Current Version: NuttX  12.6.0-RC1 a6df724b4f Aug 27 2024 11:41:56 risc-v
_assert: Assertion failed panic: at file: :0 task: ostest process: ostest 0xc000001c

• Before PR: knsh_paging Log
• After PR: knsh_paging Log

[nuttxpr]

[Experimental Bot, please feedback here]

NuttX PR Requirement Check

This PR generally meets the requirements, but with some minor improvements needed:

Summary:

• Good: The summary clearly explains the "why", "what", and "how" of the change.
• Good: Relevant NuttX issue reference is provided.
• Improve: While mentioning the next steps for BL808 is helpful, the summary should focus solely on this PR's changes. Consider moving the BL808 mention to the "Impact" section.

Impact:

• Good: Impact on users, build process, hardware, and documentation is adequately addressed.
• Good: Security implications are considered.
• Missing: Backward/forward compatibility is not explicitly discussed. While this PR seems focused on specific SoCs, it's crucial to clarify if any compatibility concerns exist for other platforms or future development.

Testing:

• Good: Testing on Milk-V Duo S SBC (SG2000) with clear "before" and "after" results is provided.
• Good: Regression testing on QEMU RISC-V is appreciated.
• Improve:
  • Instead of just linking to GitHub Actions logs, include relevant snippets directly in the PR description for easier review.
  • "MMU Paging" testing encountered errors. It's vital to investigate and resolve these before merging, or clearly document them as known limitations if they are outside this PR's scope.

Overall:

The PR demonstrates a good understanding of NuttX requirements. Addressing the highlighted points, particularly compatibility and testing clarity, will significantly enhance its readiness for review.