Welcome to the documentation for the Miden compiler toolchain.
Warning
The compiler is currently in an experimental state, and has known bugs and limitations, it is not yet ready for production usage. However, we\u2019d encourage you to start experimenting with it yourself, and give us feedback on any issues or sharp edges you encounter.
The documentation found here should provide a good starting point for the current capabilities of the toolchain, however if you find something that is not covered, but is not listed as unimplemented or a known limitation, please let us know by reporting an issue on the compiler issue tracker.
"},{"location":"#what-is-provided","title":"What is provided?","text":"The compiler toolchain consists of the following primary components:
rustc, which is largely just what LLVM produces, but with the shadow stack placed at the start of linear memory rather than after read-only data. In the future we intend to support more variety in the structure of Wasm modules we accept, but for the time being we\u2019re primarily focused on using this as the path for lowering Rust to Miden.midenc executable, and a Rust crate, midenc-compiler to allow integrating the compiler into other tools. This plays the same role as rustc does in the Rust ecosystem.cargo-miden, that provides a convenient developer experience for creating and compiling Rust projects targeting Miden. It contains a project template for a basic Rust crate, and handles orchestrating rustc and midenc to compile the crate to WebAssembly, and then to Miden Assembly.midenc debug, which provides a UI very similar to lldb or gdb when using the TUI mode. You can use this to run a program, or step through it cycle-by-cycle. You can set various types of breakpoints; see the source code, call stack, and contents of the operand stack at the current program point; as well as interatively read memory and format it in various ways for display.cargo miden new automatically adds this as a dependency.That all sounds great, but what can you do with the compiler today? The answer depends a bit on what aspect of the compiler you are interested in:
"},{"location":"#rust","title":"Rust","text":"The most practically useful, and interesting capability provided by the compiler currently, is the ability to compile arbitrary Rust programs to Miden Assembly. See the guides for more information on setting up and compiling a Rust crate for execution via Miden.
"},{"location":"#webassembly","title":"WebAssembly","text":"More generally, the compiler frontend is capable of compiling WebAssembly modules, with some constraints, to Miden Assembly. As a result, it is possible to compile a wider variety of languages to Miden Assembly than just Rust, so long as the language can compile to WebAssembly. However, we do not currently provide any of the language-level support for languages other than Rust, and have limited ability to provide engineering support for languages other than Rust at this time.
Our Wasm frontend does not support all of the extensions to the WebAssembly MVP, most notably the reference types and GC proposals.
"},{"location":"#miden-ir","title":"Miden IR","text":"If you are interested in compiling to Miden from your own compiler, you can target Miden IR, and invoke the driver from your compiler to emit Miden artifacts. At this point in time, we don\u2019t have the resources to provide much in the way of engineering support for this use case, but if you find issues in your efforts to use the IR in your compiler, we would certainly like to know about them!
We do not currently perform any optimizations on the IR, since we are primarily working with the output of compiler backends which have already applied optimizations, at this time. This may change in the future, but for now it is expected that you implement your own optimization passes as needed.
"},{"location":"#known-bugs-and-limitations","title":"Known bugs and limitations","text":"For the latest information on known bugs, see the issue tracker.
See Known Limitations for details on what functionality is missing or only partially implemented.
"},{"location":"#where-to-start","title":"Where to start?","text":"Provided here are a set of guides which are focused on documenting a couple of supported workflows we expect will meet the needs of most users, within the constraints of the current feature set of the compiler. If you find that there is something you wish to do that is not covered, and is not one of our known limitations, please open an issue, and we will try to address the missing docs as soon as possible.
"},{"location":"#installation","title":"Installation","text":"To get started, there are a few ways you might use the Miden compiler. Select the one that applies to you, and the corresponding guide will walk you through getting up and running:
midenc executableThis document describes the various calling conventions recognized/handled by the compiler, including a specification for the interaction with the IR type system.
There are four calling conventions represented in the compiler:
C aka SystemV, which corresponds to the C ABI commonly used for C foreign-function interfaces (FFI). We specifically use the System V ABI because it is well understood, documented, and straightforward.Fast, this convention allows the compiler to follow either the C calling convention, or modify it as it sees fit on a function-by-function basis. This convention provides no guarantees about how a callee will expect arguments to be passed, so should not be used for functions which are expected to have a stable, predictable interface. This is a good choice for local functions, or functions which are only used within an executable/library and are not part of the public interface.Kernel, this is a special calling convention that is used when defining kernel modules in the IR. Functions which are part of the kernel\u2019s public API are required to use this convention, and it is not possible to call a function via syscall if the callee is not defined with this convention. Because of the semantics of syscall, this convention is highly restrictive. In particular, it is not permitted to pass pointer arguments, or aggregates containing pointers, as syscall involves a context switch, and thus memory in the caller is not accessible to the callee, and vice versa.Contract, this is a special calling convention that is used when defining smart contract functions, i.e. functions that can be call\u2018d. The compiler will not permit you to call a function if the callee is not defined with this convention, and functions with this convention cannot be called via exec. Like syscall, the call instruction involves a context switch, however, unlike the Kernel convention, the Contract convention is allowed to have types in its signature that are/contain pointers, with certain caveats around those pointers.All four conventions above are based on the System V C ABI, tailored to the Miden VM. The only exception is Fast, which may modify the ABI arbitrarily as it sees fit, and makes no guarantees about what modifications, if any, it will make.
The following is a description of how the IR type system is represented in the C calling convention. Later, a description of how the other conventions extend/restrict/modify this representation will be provided.
sizeof Alignment (bytes) Miden Type Integer _Bool/bool I1 1 1 u32 Integer char, signed char I8 1 1 i321 Integer unsigned char U8 1 1 u32 Integer short / signed short I16 2 2 i321 Integer unsigned short U16 2 2 u32 Integer int / signed int / enum I32 4 4 i3218 Integer unsigned int U32 4 4 u32 Integer long / signed long I32 4 4 i321 Integer unsigned long / size_t U32 4 4 u32 Integer long long / signed long long I64 8 8 i642 Integer unsigned long long U64 8 8 u643 Pointer any-type * / any-type (*)() Ptr(_) 4 4 u3267 Floating point float F32 4 4 u324 Floating point double F64 8 8 u644 Floating point long double 16 16 (none)5 Note
The compiler does not support scalars larger than one word (128 bits) at this time. As a result, anything that is larger than that must be allocated in linear memory, or in an automatic allocation (function-local memory), and passed around by reference.
The native scalar type for the Miden VM is a \u201cfield element\u201d, specifically a 64-bit value representing an integer in the \u201cGoldilocks\u201d field, i.e. 0..(2^64-2^32+1). A number of instructions in the VM operate on field elements directly. However, the native integral/pointer type, i.e. a \u201cmachine word\u201d, is actually u32. This is because a field element can fully represent 32-bit integers, but not the full 64-bit integer range. Values of u32 type are valid field element values, and can be used anywhere that a field element is expected (barring other constraints).
Miden also has the notion of a \u201cword\u201d, not to be confused with a \u201cmachine word\u201d (by which we mean the native integral type used to represent pointers), which corresponds to a set of 4 field elements. Words are commonly used in Miden, particularly to represent hashes, and a number of VM instructions operate on word-sized operands. As an aside, 128-bit integer values are represented using a word, or two 64-bit limbs (each limb consisting of two 32-bit limbs).
All integral types mentioned above, barring field elements, use two\u2019s complement encoding. Unsigned integral types make use of the sign bit to change the value range (i.e. 0..2^32-1, rather than -231..231-1), but the encoding follows two\u2019s complement rules.
The Miden VM only has native support for field elements, words, and u32; all other types are implemented in software using intrinsics.
Structures and unions assume the alignment of their most strictly aligned component. Each member is assigned to the lowest available offset with the appropriate alignment. The size of any object is always a multiple of the object\u2019s alignment. An array uses the same alignment as its elements. Structure and union objects can require padding to meet size and alignment constraints. The contents of any padding is undefined.
"},{"location":"appendix/calling-conventions/#memory-model","title":"Memory model","text":"Interacting with memory in Miden is quite similar to WebAssembly in some ways:
This is where it begins to differ though, and takes on qualities unique to Miden (in part, or whole):
call vs syscall vs exec. The first two perform a context switch when transferring control to the callee, and the callee has no access to the caller\u2019s memory (and the caller has no access to the callee\u2019s memory). As a result, references to memory cannot be passed from caller to callee in arguments, nor can they be returned from the callee to the caller.This presents some complications, particularly:
load instruction. Operations like this must be translated into instruction sequences that load whole words from memory, extract the data needed, and discard the unused bits. This makes the choice of where in memory to store something much more important than byte-addressable memory, as loads of values which are not aligned to element or word boundaries can be quite inefficient in some cases.The compiler solves this by providing a byte-addressable IR, and internally translating operations in the IR to the equivalent sequence of Miden instructions needed to emulate that operation. This translation is done during code generation, and uses the following semantics to determine how a particular operation gets lowered:
Because we\u2019re essentially emulating byte-addressable memory on word-addressable memory, loads/stores can range from simple and straightforward, to expensive and complicated, depending on the size and alignment of the value type. The process goes as follows:
The worst case scenario for an unaligned load or store involves a word-sized type starting somewhere in the last element of the first word. This will require loading elements from three consecutive words, plus a lot of shuffling bits around to get the final, aligned word-sized value on the operand stack. Luckily, such operations should be quite rare, as by default all word-sized scalar types are word-aligned or element-aligned, so an unaligned load or store would require either a packed struct, or a type such as an array of bytes starting at some arbitrary address. In practice, most loads/stores are likely to be element-aligned, so most overhead from emulation will come from values which cross an element or word boundary.
"},{"location":"appendix/calling-conventions/#function-calls","title":"Function calls","text":"This section describes the conventions followed when executing a function call via exec, including how arguments are passed on the operand stack, stack frames, etc. Later, we\u2019ll cover the differences when executing calls via call or syscall.
Miden does not have registers in the style of hardware architectures. Instead it has an operand stack, on which an arbitrary number of operands may be stored, and local variables. In both cases - an operand on the operand stack, or a single local variable - the value type is nominally a field element, but it is easier to reason about them as untyped element-sized values. The operand stack is used for function arguments, return values, temporary variables, and scratch space. Local variables are not always used, but are typically used to hold multiply-used values which you don\u2019t want to keep on the operand stack, function-scoped automatic allocations (i.e. alloca), and other such uses.
Miden does not have a stack frame per se. When you call a procedure in Miden Assembly, any local variables declared by that procedure are allocated space in a reserved region of linear memory in a single consecutive chunk. However, there is no stack or frame pointer, and because Miden is a Harvard architecture machine, there are no return addresses. Instead, languages (such as C) which have the concept of a stack frame with implications for the semantics of say, taking the address of a local variable, will need to emit code in function prologues and epilogues to maintain a shadow stack in Miden\u2019s linear memory. If all you need is local variables, you can get away with leaning on Miden\u2019s notion of local variables without implementing a shadow stack.
Because there are no registers, the notion of callee-saved or caller-saved registers does not have a direct equivalent in Miden. However, in its place, a somewhat equivalent set of rules defines the contract between caller and callee in terms of the state of the operand stack, those are described below in the section covering the operand stack.
"},{"location":"appendix/calling-conventions/#the-shadow-stack","title":"The shadow stack","text":"Miden is a Harvard architecture; as such, code and data are not in the same memory space. More precisely, in Miden, code is only addressable via the hash of the MAST root of that code, which must correspond to code that has been loaded into the VM. The hash of the MAST root of a function can be used to call that function both directly and indirectly, but that is the only action you can take with it. Code can not be generated and called on the fly, and it is not stored anywhere that is accessible to code that is currently executing.
One consequence of this is that there are no return addresses or instruction pointers visible to executing code. The runtime call stack is managed by the VM itself, and is not exposed to executing code in any way. This means that address-taken local C variables need to be on a separate stack in linear memory (which we refer to as a \u201cshadow stack\u201d). Not all functions necessarily require a frame in the shadow stack, as it cannot be used to perform unwinding, so only functions which have locals require a frame.
The Miden VM actually provides some built-in support for stack frames when using Miden Assembly. Procedures which are declared with some number of locals, will be automatically allocated sufficient space for those locals in a reserved region of linear memory when called. If you use the locaddr instruction to get the actual address of a local, that address can be passed as an argument to callees (within the constraints of the callee\u2019s calling convention).
Languages with more elaborate requirements with regard to the stack will need to implement their own shadow stack, and emit code in function prologues/epilogues to manage it.
"},{"location":"appendix/calling-conventions/#the-operand-stack","title":"The operand stack","text":"The Miden virtual machine is a stack machine, not a register machine. Rather than having a fixed set of registers that are used to store and manipulate scalar values, the Miden VM has the operand stack, which can hold an arbitrary number of operands (where each operand is a single field element), of which the first 16 can be directly manipulated using special stack instructions. The operand stack is, as the name implies, a last-in/first-out data structure.
The following are basic rules all conventions are expected to follow with regard to the operand stack:
Miden Abstract Syntax Trees (MASTs) do not have any notion of functions, and as such are not aware of parameters, return values, etc. For this document, that\u2019s not a useful level of abstraction to examine. Even a step higher, Miden Assembly (MASM) has functions (procedures in MASM parlance), but no function signature, i.e. given a MASM procedure, there is no way to know how many arguments it expects, how many values it returns, let alone the types of arguments/return values. Instead, we\u2019re going to specify calling conventions in terms of Miden IR, which has a fairly expressive type system more or less equivalent to that of LLVM, and how that translates to Miden primitives.
Functions in Miden IR always have a signature, which specify the following:
The following table relates IR types to how they are expected to be passed from the caller to the callee, and vice versa:
Type Parameter Result scalar direct direct empty struct or union1 ignored ignored scalar struct or union2 direct direct other struct or union indirect indirect array indirect N/AThe compiler will automatically generate code that follows these rules, but if emitting MASM from your own backend, it is necessary to do so manually. For example, a function whose signature specifies that it returns a non-scalar struct by value, must actually be written such that it expects to receive a pointer to memory allocated by the caller sufficient to hold the return value, as the first parameter of the function (i.e. the parameter is prepended to the parameter list). When returning, the function must write the return value to that pointer, rather than returning it on the operand stack. In this example, the return value is returned indirectly (by reference).
A universal rule is that the arguments are passed in reverse order, i.e. the first argument in the parameter list of a function will be on top of the operand stack. This is different than many Miden instructions which seemingly use the opposite convention, e.g. add, which expects the right-hand operand on top of the stack, so a + b is represented like push a, push b, add. If we were to implement add as a function, it would instead be push b, push a, exec.add. The rationale behind this is that, in general, the more frequently used arguments appear earlier in the parameter list, and thus we want those closer to the top of the operand stack to reduce the amount of stack manipulation we need to do.
Arguments/return values are laid out on the operand stack just like they would be as if you had just loaded it from memory, so all arguments are aligned, but may span multiple operands on the operand stack as necessary based on the size of the type (i.e. a struct type that contains a u32 and a i1 field would require two operands to represent). If the maximum number of operands allowed for the call is reached, any remaining arguments must be spilled to the caller\u2019s stack frame, or to the advice provider. The former is used in the case of exec/dynexec, while the latter is used for call and syscall, as caller memory is not accessible to the callee with those instructions.
While ostensibly 16 elements is the maximum number of operands on the operand stack that can represent function arguments, due to the way dynexec/dyncall work, it is actually limited to 12 elements, because at least 4 must be free to hold the hash of the function being indirectly called.
Zero-sized types have no representation in memory, so they are ignored/skipped\u00a0\u21a9\u21a9\u21a9\u21a9\u21a9
Any struct or union that recursively (including through nested structs, unions, and arrays) contains just a single scalar value and is not specified to have greater than natural alignment.\u00a0\u21a9\u21a9
u64 is not a native Miden type, but is implemented in software using two 32-bit limbs (i.e. a pair of field elements)\u00a0\u21a9
floating-point types are not currently supported, but will be implemented using compiler intrinsics\u00a0\u21a9\u21a9
long double values correspond to 128-bit IEEE-754 quad-precision binary128 values. These are not currently supported, and we have no plans to support them in the near term. Should we ever provide such support, we will do so using compiler intrinsics.\u00a0\u21a9
A null pointer (for all types) always has the value zero.\u00a0\u21a9
Miden\u2019s linear memory is word-addressable, not byte-addressable. The Ptr type has an AddressSpace parameter, that by default is set to the byte-addressable address space. The compiler translates values of Ptr type that are in this address space, into the Miden-native, word-addressable address space during codegen of load/store operations. See the section on the memory model below for more details.\u00a0\u21a9
An enum is i32 if all members of the enumeration can be represented by an int/unsigned int, otherwise it uses i64.\u00a0\u21a9
This document describes an issue that arises when trying to map the ad-hoc calling convention/ABI used by various Miden Assembly procedures, such as those comprising the transaction kernel, and the \u201ccanonical\u201d ABI(s) representable in Rust. It proposes a solution to this problem in the form of adapter functions, where the details of a given adapter are one of a closed set of known ABI transformation strategies.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#summary","title":"Summary","text":"The gist of the problem is that in Miden, the size and number of procedure results is only constrained by the maximum addressable operand stack depth. In most programming languages, particularly those in which interop is typically performed using some variant of the C ABI (commonly the one described in the System V specification), the number of results is almost always limited to a single result, and the size of the result type is almost always limited to the size of a single machine word, in some cases two. On these platforms, procedure results of greater arity or size are typically handled by reserving space in the caller\u2019s stack frame, and implicitly prepending the parameter list of the callee with an extra parameter: a pointer to the memory allocated for the return value. The callee will directly write the return value via this pointer, instead of returning a value in a register.
In the case of Rust, this means that attempting to represent a procedure that returns multiple values, or returns a larger-than-machine-word type, such as Word, will trigger the implicit transformation described above, as this is allowed by the standard Rust calling conventions. Since various Miden procedures that are part of the standard library and the transaction kernel are affected by this, the question becomes \u201chow do we define bindings for these procedures in Rust?\u201d.
The solution is to have the compiler emit glue code that closes the gap between the two ABIs. It does so by generating adapter functions, which wrap functions that have an ABI unrepresentable in Rust, and orchestrate lifting/lowering arguments and results between the adapter and the \u201creal\u201d function.
When type signatures are available for all Miden Assembly procedures, we can completely automate this process. For now, we will require a manually curated list of known procedures, their signatures, and the strategy used to \u201cadapt\u201d those procedures for binding in Rust.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#background","title":"Background","text":"After analyzing all of the functions in the transaction kernel API, the most common cause of a mismatch between Miden and Rust ABIs, is due to implicit \u201csret\u201d parameters, i.e. the transformation mentioned above which inserts an implicit pointer to the caller\u2019s stack frame for the callee to write the return value to, rather than doing so in a register (or in our case, on the operand stack). This seems to happen for any type that is larger than 8 bytes (i64).
Tip
For a complete list of the transaction kernel functions, in WIT format, see miden.wit.
For most transaction kernel functions, the adapter function can be generated automatically using the pattern recognition and adapter functions described below.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#prerequisites","title":"Prerequisites","text":"The compiler will analyze every component import to determine if that import requires an adapter, as determined by matching against a predefined set of patterns. The adapter generation will take place in the frontend, as it has access to all of the needed information, and ensures that we do not have any transformations or analyses that make decisions on the un-adapted procedure.
The following pseudo-code can be used to recognize the various Miden ABI patterns:
pub enum MidenAbiPattern {\n /// Calling this procedure will require an sret parameter on the Rust side, so\n /// we need to emit an adapter that will lift/lower calls according to that\n /// strategy.\n ReturnViaPointer,\n /// The underlying procedure is fully representable in Rust, and requires no adaptation.\n NoAdapterNeeded,\n}\n\npub struct MidenAbiPatternRecognition {\n pattern: Option<MidenAbiPattern>,\n component_function: ComponentFunctionType,\n wasm_core_func: Signature,\n tx_kernel_function: Signature,\n}\n\npub fn recognize_miden_abi_pattern(\n component_function: &ComponentFunctionType,\n wasm_core_func: &Signature,\n tx_kernel_func: &Signature) -> MidenAbiPatternRecognition {\n if wasm_core_func == tx_kernel_func {\n return MidenAbiPatternRecognition {\n pattern: Some(NoAdapterNeeded),\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n } else if component_function.returns[0].byte_size > 8 && wasm_core_func.params.last() == I32 {\n return MidenAbiPatternRecognition {\n pattern: Some(ReturnViaPointer),\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n } else {\n return MidenAbiPatternRecognition {\n pattern: None,\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n }\n}\nThe following pseudo-code can then be used to generate the adapter function:
pub fn generate_adapter(recognition: MidenAbiPatternRecognition) {\n match recognition.pattern {\n Some(pattern) => generate_adapter(\n pattern,\n recognition.component_function,\n recognition.wasm_core_function,\n recognition.tx_kernel_function\n ),\n None => use_manual_adapter(\n recognition.component_function,\n recognition.wasm_core_function,\n recognition.tx_kernel_function\n ),\n }\n}\n\n/// Escape hatch for the cases when the compiler can't generate an adapter function automatically\n/// and we need to provide the adapter function manually.\npub fn use_manual_adapter(...) {\n // Find and use the manual adapter in the adapter library for the tx_kernel_function\n}\nThe manual adapter library is a collection of adapter functions that are used when the compiler can\u2019t generate an adapter function automatically so its expected to be provided. The manual adapter library is a part of the Miden compiler. It is not anticipated that we will have many, or any, of these; however in the near term we are going to manually map procedures to their adapter strategies, as we have not yet automated the pattern recognition step.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#return-via-pointer-adapter","title":"Return-via-pointer adapter","text":"The return value is expected to be returned by storing its flattened representation in a pointer passed as an argument.
Recognize this Miden ABI pattern by looking at the Wasm component function type. If the return value is bigger than 64 bits, expect the last argument in the Wasm core(HIR) signature to be i32 (a pointer).
The adapter function calls the tx kernel function and stores the result in the provided pointer (the last argument of the Wasm core function).
Here is the pseudo-code for generating the adapter function for the return-via-pointer Miden ABI pattern:
let ptr = wasm_core_function.params.last();\nlet adapter_function = FunctionBuilder::new(wasm_core_function.clone());\nlet tx_kernel_function_params = wasm_core_function.params.drop_last();\nlet tx_kernel_func_val = adapter_function.call(tx_kernel_function, tx_kernel_function_params);\nadapter_function.store(tx_kernel_func_val, ptr);\nadapter_function.build();\nHere is how the adapter might look like in a pseudo-code for the add_asset function:
/// Takes an Asset as an argument and returns a new Asset\nfunc wasm_core_add_asset(v0: f64, v1: f64, v2: f64, v3: f64, ptr: i32) {\n v4 = call tx_kernel_add_asset(v0, v1, v2, v3);\n // v4 is a tuple of 4 f64 values\n store v4 in ptr;\n}\nNo adapter is needed. The Wasm core function type is the same as the tx kernel ad-hoc signature.
This Miden ABI pattern is selected if no other Miden ABI pattern is applicable and the wasm core function signature is the same as the tx kernel ad-hoc signature.
For example, the get_id function falls under this Miden ABI pattern and its calls will be translated to the tx kernel function calls without any modifications.
get_assets","text":"get_assets:func() -> list<core-asset> in the note interface is the only function that requires attention. In Canonical ABI, any function that returns a dynamic list of items needs to allocate memory in the caller\u2019s module due to the shared-nothing nature of the Wasm component model. For this case, a realloc function is passed as a part of lift/lower Canonical ABI options for the caller to allocate memory in the caller\u2019s module.
Here are the signatures of the get_assets function in the WIT, core Wasm, and the tx kernel ad-hoc ABI: Comment from the miden-base
#! Writes the assets of the currently executing note into memory starting at the specified address.\n#!\n#! Inputs: [dest_ptr]\n#! Outputs: [num_assets, dest_ptr]\n#!\n#! - dest_ptr is the memory address to write the assets.\n#! - num_assets is the number of assets in the currently executing note.\nWasm component function type: get-assets: func() -> list<core-asset>;
Wasm core signature: wasm_core_get_assets(i32) -> ()
If we add a new get_assets_count: func() -> u32; function to the tx kernel and add the assets count parameter to the get_assets function (get_assets: func(assets_count: u32) -> list<core-asset>;) we should have everything we need to manually write the adapter function for the get_assets function.
The list is expected to be returned by storing the pointer to its first item in a ptr pointer passed as an argument and item count at ptr + 4 bytes address (ptr points to two pointers).
We could try to recognize this Miden ABI pattern by looking at the Wasm component function type. If the return value is a list, expect the last argument in the Wasm core(HIR) signature to be i32 (a pointer). The problem is recognizing the list count parameter in the Wasm core(HIR) signature.
The adapter function calls allocates asset_count * item_size memory via the realloc call and passes the pointer to the newly allocated memory to the tx kernel function.
Here is how the adapter function might look like in a pseudo-code for the get_assets function:
func wasm_core_get_assets(asset_count: u32, ptr_ptr: i32) {\n mem_size = asset_count * item_size;\n ptr = realloc(mem_size);\n (actual_asset_count, ptr) = call tx_kernel_get_assets(ptr);\n assert(actual_asset_count == asset_count);\n store ptr in ptr_ptr;\n store account_count in ptr_ptr + 4;\n}\nNote
Since the get_assets tx kernel function in the current form can trash the provided memory if the actual assets count differs from the returned by get_assets_count, we can introduce the asset count parameter to the get_assets tx kernel function and check that it the same as the actual assets count written to memory.
add_asset (return-via-pointer Miden ABI pattern)","text":"Comment from the miden-base
#! Add the specified asset to the vault.\n#!\n#! Panics:\n#! - If the asset is not valid.\n#! - If the total value of two fungible assets is greater than or equal to 2^63.\n#! - If the vault already contains the same non-fungible asset.\n#!\n#! Stack: [ASSET]\n#! Output: [ASSET']\n#!\n#! - ASSET' final asset in the account vault is defined as follows:\n#! - If ASSET is a non-fungible asset, then ASSET' is the same as ASSET.\n#! - If ASSET is a fungible asset, then ASSET' is the total fungible asset in the account vault\n#! after ASSET was added to it.\nWasm component function type: add-asset(core-asset) -> core-asset
Wasm core signature: wasm_core_add_asset(f64, f64, f64, f64, i32) -> () The last i32 is a pointer to a returned value (word)
Tx kernel ad-hoc signature: tx_kernel_add_asset(felt, felt, felt, felt) -> (felt, felt, felt, felt)
get_id (no-adapter-needed Miden ABI pattern)","text":"Comment from the miden-base
#! Returns the account id.\n#!\n#! Stack: []\n#! Output: [acct_id]\n#!\n#! - acct_id is the account id.\nWasm component function type: get-id() -> account-id
Wasm core signature: wasm_core_get_id() -> f64
Tx kernel ad-hoc signature: tx_kernel_get_id() -> felt
Tip
See the issue tracker for information on known bugs. This document focuses on missing/incomplete features, rather than bugs.
The compiler is still in its early stages of development, so there are various features that are unimplemented, or only partially implemented, and the test suite is still limited in scope, so we are still finding bugs on a regular basis. We are rapidly improving this situation, but it is important to be aware of this when using the compiler.
The features discussed below are broken up into sections, to make them easier to navigate and reference.
"},{"location":"appendix/known-limitations/#rust-language-support","title":"Rust language support","text":""},{"location":"appendix/known-limitations/#floating-point-types","title":"Floating point types","text":"In order to represent Felt \u201cnatively\u201d in Rust, we were forced to piggy-back on the f32 type, which is propagated through to WebAssembly, and allows us to handle those values specially.
As a result, floating-point types in Rust are not supported at all. Any attempt to use them will result in a compilation error. We considered this a fair design tradeoff, as floating point math is unused/rare in the context in which Miden is used, in comparison to fixed-point or field arithmetic. In addition, implementing floating-point operations in software on the Miden VM would be extraordinarily expensive, which generally works against the purpose for using floats in the first place.
At this point in time, we have no plans to support floats, but this may change if we are able to find a better/more natural representation for Felt in WebAssembly.
This feature corresponds to call_indirect in WebAssembly, and is associated with Rust features such as trait objects (which use indirection to call trait methods), and closures. Note that the Rust compiler is able to erase the indirection associated with certain abstractions statically in some cases, shown below. If Rust is unable to statically resolve all call targets, then midenc will raise an error when it encounters any use of call_indirect.
Warning
The following examples rely on rustc/LLVM inlining enough code to be able to convert indirect calls to direct calls. This may require you to enable link-time optimization with lto = \"fat\" and compile all of the code in the crate together with codegen-units = 1, in order to maximize the amount of inlining that can occur. Even then, it may not be possible to remove some forms of indirection, in which case you will need to find another workaround.
pub fn is_zeroed(bytes: &[u8; 32]) -> bool {\n // Rust is able to convert this to a loop, erasing the closure completely\n bytes.iter().copied().all(|b| b == 0)\n}\npub fn call<F, T>(fun: F) -> T\nwhere\n F: Fn() -> T,\n{\n fun()\n}\n\n#[inline(never)]\npub fn foo() -> bool { true }\n\nfn main() {\n // Rust is able to inline the body of `call` after monomorphization, which results in\n // the call to `foo` being resolved statically.\n call(foo)\n}\npub trait Foo {\n fn is_foo(&self) -> bool;\n}\n\nimpl Foo for u32 {\n #[inline(never)]\n fn is_foo(&self) -> bool { true }\n}\n\nfn has_foo(items: &[dyn Foo]) -> bool {\n items.iter().any(|item| item.is_foo())\n}\n\nfn main() -> u32 {\n // Rust inlines `has_foo`, converts the iterator chain to a loop, and is able to realize\n // that the `dyn Foo` items are actually `u32`, and resolves the call to `is_foo` to\n // `<u32 as Foo>::is_foo`.\n let foo: &dyn Foo = &u32::MAX as &dyn Foo;\n has_foo(&[foo]) as u32\n}\nThe Miden SDK for Rust, is a Rust crate that provides the implementation of native Miden types, as well as bindings to the Miden standard library and transaction kernel APIs.
Currently, only a very limited subset of the API surface has had bindings implemented. This means that there is a fair amount of native Miden functionality that is not yet available from Rust. We will be expanding the SDK rapidly over the next few weeks and months, but for the time being, if you encounter a missing API that you need, let us know, so we can ensure it is prioritized above APIs which are lesser used.
"},{"location":"appendix/known-limitations/#rustmiden-ffi-foreign-function-interface-and-interop","title":"Rust/Miden FFI (foreign function interface) and interop","text":"While the compiler has functionality to link against native Miden Assembly libraries, binding against procedures exported from those libraries from Rust can require glue code to be emitted by the compiler in some cases, and the set of procedures for which this is done is currently restricted to a hardcoded whitelist of known Miden procedures.
This affects any procedure which returns a type larger than u32 (excluding Felt, which for this purpose has the same size). For example, returing a Miden Word from a procedure, a common return type, is not compatible with Rust\u2019s ABI - it will attempt to generate code which allocates stack space in the caller, which it expects the callee to write to, inserting a new parameter at the start of the parameter list, and expecting nothing to be returned by value. The compiler handles situations like these using a set of ABI \u201ctransformation strategies\u201d, which lift/lower differences between the Rust and Miden ABIs at call boundaries.
To expose the FFI machinery for use with any Miden procedure, we need type signatures for those procedures at a minimum, and in some cases we may require details of the calling convention/ABI. This metadata does not currently exist, but is on the roadmap for inclusion into Miden Assembly and Miden packaging. Once present, we can open up the FFI for general use.
"},{"location":"appendix/known-limitations/#core-miden-functionality","title":"Core Miden functionality","text":""},{"location":"appendix/known-limitations/#dynamic-procedure-invocation","title":"Dynamic procedure invocation","text":"This is a dependency of Function Call Indirection described above, and is the mechanism by which we can perform indirect calls in Miden. In order to implement support for indirect calls in the Wasm frontend, we need underlying support for dynexec, which is not yet implemented.
This feature adds support for lowering indirect calls to dynexec or dyncall instructions, depending on the ABI of the callee. dyncall has an additional dependency on support for Cross-Context Procedure Invocation.
A known issue with this feature is that dyn(exec|call) consumes a word on the operand stack for the hash of the callee being invoked, but this word remains on the stack when entering the callee, which has the effect of requiring procedures to have a different ABI depending on whether they expect to be dynamically-invoked or not.
Our solution to that issue is to generate stubs which are used as the target of dyn(exec|call), the body of which drop the callee hash, fix up the operand stack as necessary, and then uses a simple exec or call to invoke the \u201creal\u201d callee. We will emit a single stub for every function which has its \u201caddress\u201d taken, and use the hash of the stub in place of the actual callee hash.
This is required in order to support representing Miden accounts and note scripts in Rust, and compilation to Miden Assembly.
Currently, you can write code in Rust that is very close to how accounts and note scripts will look like in the language, but it is not possible to actually implement either of those in Rust today. The reasons for this are covered in depth in the tracking issue linked above, but to briefly summarize, the primary issue has to do with the fact that Rust programs are compiled for a \u201cshared-everything\u201d environment, i.e. you can pass references to memory from caller to callee, write to caller memory from the callee, etc. In Miden however, contexts are \u201cshared-nothing\u201d units of isolation, and thus cross-context operations, such as performing a call from a note script to a method on an account, are not compatible with the usual calling conventions used by Rust and LLVM.
The solution to this relies on compiling the Rust code for the wasm32-wasip2 target, which emits a new kind of WebAssembly module, known as a component. These components adhere to the rules of the WebAssembly Component Model. Of primary interest to us, is the fact that components in this model are \u201cshared-nothing\u201d, and the ABI used to communicate across component boundaries, is specially designed to enforce shared-nothing semantics on caller and callee. In addition to compiling for a specific Wasm target, we also rely on some additional tooling for describing component interfaces, types, and to generate Rust bindings for those descriptions, to ensure that calls across the boundary remain opaque, even to the linker, which ensures that the assumptions of the caller and callee with regard to what address space they operate in are preserved (i.e. a callee can never be inlined into the caller, and thus end up executing in the caller\u2019s context rather than the expected callee context).
This is one of our top priorities, as it is critical to being able to use Rust to compile code for the Miden rollup, but it is also the most complex feature on our roadmap, hence why it is scheduled for our Beta 2 milestone, rather than Beta 1 (the next release), as it depends on multiple other subfeatures being implemented first.
"},{"location":"appendix/known-limitations/#packaging","title":"Packaging","text":""},{"location":"appendix/known-limitations/#package-format","title":"Package format","text":"This feature represents the ability to compile and distribute a single artifact that contains the compiled MAST, and all required and optional metadata to make linking against, and executing packages as convenient as a dynamic library or executable.
The compiler currently produces, by default, an experimental implementation of a package format that meets the minimum requirements to support libraries and programs compiled from Rust:
However, this package format is not yet understood by the Miden VM itself. This means you cannot, currently, compile a package and then run it using miden run directly. Instead, you can use midenc run to load and run code from a package, as the compiler ships with the VM embedded for use with the interactive debugger, and provides native support for packaging on top of it. You can also use midenc debug to execute your program interactively in the debugger, depending on your needs. See Debugging Programs for more information on how to use the debugger, and midenc help run for more information on executing programs with the midenc run command.
While it is possible to emit raw MAST from midenc, rather than the experimental package format, the resulting artifact cannot be run without some fragile and error-prone manual setup, in order to ensure that the advice provider is correctly initialized with any read-only data segments. For now, it is recommended that you use the midenc tooling for testing programs, until the format is stabilized.
TODO
For the list of the unsupported Wasm core types, instructions and features, see the README.
"},{"location":"design/overview/","title":"Compiler architecture","text":"This is an index of various design documents for the compiler and its components. Some of these are planned topics, and some have documentation that hasn\u2019t been polished up yet. We\u2019ll slowly start to flesh out the documentation in this section as the compiler matures.
This chapter will walk through how to develop Miden programs in Rust using the standard library provided by the miden-stdlib-sys crate (see the README.
Import the standard library from the miden-stdlib-sys crate:
use miden_stdlib_sys::*;\nFelt (field element) type","text":"The Felt type is a field element type that is used to represent the field element values of the Miden VM.
To initialize a Felt value from an integer constant checking the range at compile time, use the felt! macro:
let a = felt!(42);\nOtherwise, use the Felt::new constructor:
let a = Felt::new(some_integer_var).unwrap();\nThe constructor returns an error if the value is not a valid field element, e.g. if it is not in the range 0..=M where M is the modulus of the field (2^64 - 2^32 + 1).
The Felt type implements the standard arithmetic operations, e.g. addition, subtraction, multiplication, division, etc. which are accessible through the standard Rust operators +, -, *, /, etc. All arithmetic operations are wrapping, i.e. performed modulo M.
TODO: Add examples of using operations on Felt type and available functions (assert*, etc.).
This chapter walks you through how to develop Miden rollup accounts and note scripts in Rust using the Miden SDK crate.
"},{"location":"guides/rust_to_wasm/","title":"Compiling Rust To WebAssembly","text":"This chapter will walk you through compiling a Rust crate to a WebAssembly (Wasm) module in binary (i.e. .wasm) form. The Miden compiler has a frontend which can take such modules and compile them on to Miden Assembly, which will be covered in the next chapter.
First, let\u2019s set up a simple Rust project that contains an implementation of the Fibonacci function (I know, it\u2019s overdone, but we\u2019re trying to keep things as simple as possible to make it easier to show the results at each step, so bear with me):
Start by creating a new library crate:
cargo new --lib wasm-fib && cd wasm-fib\nTo compile to WebAssembly, you must have the appropriate Rust toolchain installed, so let\u2019s add a toolchain file to our project root so that rustup and cargo will know what we need, and use them by default:
cat <<EOF > rust-toolchain.toml\n[toolchain]\nchannel = \"stable\"\ntargets = [\"wasm32-wasip1\"]\nEOF\nNext, edit the Cargo.toml file as follows:
[package]\nname = \"wasm-fib\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[lib]\n# Build this crate as a self-contained, C-style dynamic library\n# This is required to emit the proper Wasm module type\ncrate-type = [\"cdylib\"]\n\n[dependencies]\n# Use a tiny allocator in place of the default one, if we want\n# to make use of types in the `alloc` crate, e.g. String. We\n# don't need that now, but it's good information to have in hand.\n#miden-sdk-alloc = \"0.0.5\"\n\n# When we build for Wasm, we'll use the release profile\n[profile.release]\n# Explicitly disable panic infrastructure on Wasm, as\n# there is no proper support for them anyway, and it\n# ensures that panics do not pull in a bunch of standard\n# library code unintentionally\npanic = \"abort\"\n# Enable debug information so that we get useful debugging output\ndebug = true\n# Optimize the output for size\nopt-level = \"z\"\nMost of these things are done to keep the generated code size as small as possible. Miden is a target where the conventional wisdom about performance should be treated very carefully: we\u2019re almost always going to benefit from less code, even if conventionally that code would be less efficient, simply due to the difference in proving time accumulated due to extra instructions. That said, there are no hard and fast rules, but these defaults are good ones to start with.
Tip
We reference a simple bump allocator provided by miden-sdk-alloc above, but any simple allocator will do. The trade offs made by these small allocators are not generally suitable for long-running, or allocation-heavy applications, as they \u201cleak\u201d memory (generally because they make little to no attempt to recover freed allocations), however they are very useful for one-shot programs that do minimal allocation, which is going to be the typical case for Miden programs.
Next, edit src/lib.rs as shown below:
// Do not link against libstd (i.e. anything defined in `std::`)\n#![no_std]\n\n// However, we could still use some standard library types while\n// remaining no-std compatible, if we uncommented the following lines:\n//\n// extern crate alloc;\n// use alloc::{string::String, vec::Vec};\n\n// If we wanted to use the types mentioned above, it would also be\n// a good idea to use the allocator we pulled in as a dependency\n// in Cargo.toml, like so:\n//#[global_allocator]\n//static ALLOC: miden_sdk_alloc::BumpAlloc = miden_sdk_alloc::BumpAlloc::new();\n\n// Required for no-std crates\n#[panic_handler]\nfn panic(_info: &core::panic::PanicInfo) -> ! {\n // Compiles to a trap instruction in WebAssembly\n core::arch::wasm32::unreachable()\n}\n\n// Marking the function no_mangle ensures that it is exported\n// from the compiled binary as `fib`, otherwise it would have\n// a mangled name that has no stable form.\n//\n// You can specify a different name from the library than the\n// name in the source code using the `#[export_name = \"foo\"]`\n// attribute, which will make the function callable as `foo`\n// externally (in this example)\n#[no_mangle]\npub fn fib(n: u32) -> u32 {\n let mut a = 0;\n let mut b = 1;\n for _ in 0..n {\n let c = a + b;\n a = b;\n b = c;\n }\n a\n}\nThis exports our fib function from the library, making it callable from within a larger Miden program.
All that remains is to compile to WebAssembly:
cargo build --release --target=wasm32-wasip1\nThis places a wasm_fib.wasm file under the target/wasm32-wasip1/release/ directory, which we can then examine with wasm2wat to set the code we generated:
wasm2wat target/wasm32-wasip1/release/wasm_fib.wasm\nWhich dumps the following output (may differ slightly on your machine, depending on the specific compiler version):
(module $wasm_fib.wasm\n (type (;0;) (func (param i32) (result i32)))\n (func $fib (type 0) (param i32) (result i32)\n (local i32 i32 i32)\n i32.const 0\n local.set 1\n i32.const 1\n local.set 2\n loop (result i32) ;; label = @1\n local.get 2\n local.set 3\n block ;; label = @2\n local.get 0\n br_if 0 (;@2;)\n local.get 1\n return\n end\n local.get 0\n i32.const -1\n i32.add\n local.set 0\n local.get 1\n local.get 3\n i32.add\n local.set 2\n local.get 3\n local.set 1\n br 0 (;@1;)\n end)\n (memory (;0;) 16)\n (global $__stack_pointer (mut i32) (i32.const 1048576))\n (export \"memory\" (memory 0))\n (export \"fib\" (func $fib)))\nSuccess!
"},{"location":"guides/rust_to_wasm/#next-steps","title":"Next steps","text":"In Compiling WebAssembly to Miden Assembly, we walk through how to take the WebAssembly module we just compiled, and lower it to Miden Assembly using midenc!
This guide will walk you through compiling a WebAssembly (Wasm) module, in binary form (i.e. a .wasm file), to Miden Assembly (Masm), both in its binary package form (a .masp file), and in textual Miden Assembly syntax form (i.e. a .masm file).
We will be making use of the example crate we created in Compiling Rust to WebAssembly, which produces a small Wasm module that is easy to examine in Wasm text format, and demonstrates a good set of default choices for a project compiling to Miden Assembly from Rust.
In this chapter, we will be compiling Wasm to Masm using the midenc executable, so ensure that you have followed the instructions in the Getting Started with midenc guide and then return here.
Note
While we are using midenc for this guide, the more common use case will be to use the cargo-miden Cargo extension to handle the gritty details of compiling from Rust to Wasm for you. However, the purpose of this guide is to show you what cargo-miden is handling for you, and to give you a foundation for using midenc yourself if needed.
In the last chapter, we compiled a Rust crate to WebAssembly that contains an implementation of the Fibonacci function called fib, that was emitted to target/wasm32-wasip1/release/wasm_fib.wasm. All that remains is to tell midenc to compile this module to Miden Assembly.
Currently, by default, the compiler will emit an experimental package format that the Miden VM does not yet support. We will instead use midenc run to execute the package using the VM for us, but once the package format is stabilized, this same approach will work with miden run as well.
We also want to examine the Miden Assembly generated by the compiler, so we\u2019re going to ask the compiler to emit both types of artifacts:
midenc compile --emit masm=wasm_fib.masm,masp target/wasm32-wasip1/release/wasm_fib.wasm\nThis will compile our Wasm module to a Miden package with the .masp extension, and also emit the textual Masm to wasm_fib.masm so we can review it. The wasm_fib.masp file will be emitted in the default output directory, which is the current working directory by default.
If we dump the contents of wasm_fib.masm, we\u2019ll see the following generated code:
export.fib\n push.0\n push.1\n movup.2\n swap.1\n dup.1\n neq.0\n push.1\n while.true\n if.true\n push.4294967295\n movup.2\n swap.1\n u32wrapping_add\n dup.1\n swap.1\n swap.3\n swap.1\n u32wrapping_add\n movup.2\n swap.1\n dup.1\n neq.0\n push.1\n else\n drop\n drop\n push.0\n end\n end\nend\nIf you compare this to the WebAssembly text format, you can see that this is a fairly faithful translation, but there may be areas where we generate sub-optimal Miden Assembly.
Note
At the moment the compiler does only minimal optimization, late in the pipeline during codegen, and only in an effort to minimize operand stack management code. So if you see an instruction sequence you think is bad, bring it to our attention, and if it is something that we can solve as part of our overall optimization efforts, we will be sure to do so. There are limits to what we can generate compared to what one can write by hand, particularly because Rust\u2019s memory model requires us to emulate byte-addressable memory on top of Miden\u2019s word-addressable memory, however our goal is to keep this overhead within an acceptable bound in the general case, and easily-recognized patterns that can be simplified using peephole optimization are precisely the kind of thing we\u2019d like to know about, as those kinds of optimizations are likely to produce the most significant wins.
"},{"location":"guides/wasm_to_masm/#testing-with-the-miden-vm","title":"Testing with the Miden VM","text":"Note
Because the compiler ships with the VM embedded for midenc debug, you can run your program without having to install the VM separately, though you should do that as well, as midenc only exposes a limited set of commands for executing programs, intended for debugging.
We can test our compiled program like so:
$ midenc run --num-outputs 1 wasm_fib.masp -- 10\n============================================================\nRun program: wasm_fib.masp\n============================================================\nExecuted program with hash 0xe5ba88695040ec2477821b26190e9addbb1c9571ae30c564f5bbfd6cabf6c535 in 19 milliseconds\nOutput: [55]\nVM cycles: 295 extended to 512 steps (42% padding).\n\u251c\u2500\u2500 Stack rows: 295\n\u251c\u2500\u2500 Range checker rows: 67\n\u2514\u2500\u2500 Chiplets rows: 250\n\u251c\u2500\u2500 Hash chiplet rows: 248\n\u251c\u2500\u2500 Bitwise chiplet rows: 0\n\u251c\u2500\u2500 Memory chiplet rows: 1\n\u2514\u2500\u2500 Kernel ROM rows: 0\nSuccess! We got the expected result of 55.
This guide is not comprehensive, as we have not yet examined in detail the differences between compiling libraries vs programs, linking together multiple libraries, packages, or discussed some of the more esoteric compiler options. We will be updating this documentation with those details and more in the coming weeks and months, so bear with us while we flesh out our guides!
"},{"location":"usage/cargo-miden/","title":"Getting started with Cargo","text":"As part of the Miden compiler toolchain, we provide a Cargo extension, cargo-miden, which provides a template to spin up a new Miden project in Rust, and takes care of orchestrating rustc and midenc to compile the Rust crate to a Miden package.
Warning
Currently, midenc (and as a result, cargo-miden), requires the nightly Rust toolchain, so make sure you have it installed first:
rustup toolchain install nightly-2024-05-07\nNOTE: You can also use the latest nightly, but the specific nightly shown here is known to work.
To install the extension, simply run the following in your shell:
cargo +nightly-2024-05-07 install cargo-miden\nThis will take a minute to compile, but once complete, you can run cargo help miden or just cargo miden to see the set of available commands and options.
To get help for a specific command, use cargo miden help <command> or cargo miden <command> --help.
Your first step will be to create a new Rust project set up for compiling to Miden:
cargo miden new foo\nIn this above example, this will create a new directory foo, containing a Cargo project for a crate named foo, generated from our Miden project template.
The template we use sets things up so that you can pretty much just build and run. Since the toolchain depends on Rust\u2019s native WebAssembly target, it is set up just like a minimal WebAssembly crate, with some additional tweaks for Miden specfically.
Out of the box, you will get a Rust crate that depends on the Miden SDK, and sets the global allocator to a simple bump allocator we provide as part of the SDK, and is well suited for most Miden use cases, avoiding the overhead of more complex allocators.
As there is no panic infrastructure, panic = \"abort\" is set, and the panic handler is configured to use the native WebAssembly unreachable intrinsic, so the compiler will strip out all of the usual panic formatting code.
Now that you\u2019ve created your project, compiling it to Miden Assembly is as easy as running the following command from the root of the project directory:
cargo miden build\nThis will emit the compiled artifacts to target/miden.
A very useful tool in the Miden compiler suite, is its TUI-based interactive debugger, accessible via the midenc debug command.
Warning
The debugger is still quite new, and while very useful already, still has a fair number of UX annoyances. Please report any bugs you encounter, and we\u2019ll try to get them patched ASAP!
"},{"location":"usage/debugger/#getting-started","title":"Getting started","text":"The debugger is launched by executing midenc debug, and giving it a path to a program compiled by midenc compile. See Program Inputs for information on how to provide inputs to the program you wish to debug. Run midenc help debug for more detailed usage documentation.
The debugger may also be used as a library, but that is left as an exercise for the reader for now.
"},{"location":"usage/debugger/#example","title":"Example","text":"# Compile a program to MAST from a rustc-generated Wasm module\nmidenc compile foo.wasm -o foo.masl\n\n# Load that program into the debugger and start executing it\nmidenc debug foo.masl\nTo pass arguments to the program on the operand stack, or via the advice provider, you have two options, depending on the needs of the program:
midenc debug in the same order you wish them to appear on the stack. That is, the first argument you specify will be on top of the stack, and so on.--inputs option.To specify the contents of the operand stack, you can do so following the raw arguments separator --. Each operand must be a valid field element value, in either decimal or hexadecimal format. For example:
midenc debug foo.masl -- 1 2 0xdeadbeef\nIf you pass arguments via the command line in conjunction with --inputs, then the command line arguments will be used instead of the contents of the inputs.stack option (if set). This lets you specify a baseline set of inputs, and then try out different arguments using the command line.
While simply passing operands to the midenc debug command is useful, it only allows you to specify inputs to be passed via operand stack. To provide inputs via the advice provider, you will need to use the --inputs option. The configuration file expected by --inputs also lets you tweak the execution options for the VM, such as the maximum and expected cycle counts.
An example configuration file looks like so:
# This section is used for execution options\n[options]\nmax_cycles = 5000\nexpected_cycles = 4000\n\n# This section is the root table for all inputs\n[inputs]\n# Specify elements to place on the operand stack, leftmost element will be on top of the stack\nstack = [1, 2, 0xdeadbeef]\n\n# This section contains input options for the advice provider\n[inputs.advice]\n# Specify elements to place on the advice stack, leftmost element will be on top\nstack = [1, 2, 3, 4]\n\n# The `inputs.advice.map` section is a list of advice map entries that should be\n# placed in the advice map before the program is executed. Entries with duplicate\n# keys are handled on a last-write-wins basis.\n[[inputs.advice.map]]\n# The key for this entry in the advice map\ndigest = '0x3cff5b58a573dc9d25fd3c57130cc57e5b1b381dc58b5ae3594b390c59835e63'\n# The values to be stored under this key\nvalues = [1, 2, 3, 4]\n\n[[inputs.advice.map]]\ndigest = '0x20234ee941e53a15886e733cc8e041198c6e90d2a16ea18ce1030e8c3596dd38''\nvalues = [5, 6, 7, 8]\nOnce started, you will be dropped into the main debugger UI, stopped at the first cycle of the program. The UI is organized into pages and panes, with the main/home page being the one you get dropped into when the debugger starts. The home page contains the following panes:
On the home page, the following keyboard shortcuts are available:
Shortcut Mnemonic Descriptionq quit exit the debugger h next pane cycle focus to the next pane l prev pane cycle focus to the previous pane s step advance the VM one cycle n step next advance the VM to the next instruction c continue advance the VM to the next breakpoint, else to completion e exit frame advance the VM until we exit the current call frame, a breakpoint is triggered, or execution terminates d delete delete an item (where applicable, e.g. the breakpoints pane) : command prompt bring up the command prompt (see below for details) When various panes have focus, additional keyboard shortcuts are available, in any pane with a list of items, or multiple lines (e.g. source code), j and k (or the up and down arrows) will select the next item up and down, respectively. As more features are added, I will document their keyboard shortcuts below.
From the home page, typing : will bring up the command prompt in the footer pane.
You will know the prompt is active because the keyboard shortcuts normally shown there will no longer appear, and instead you will see the prompt, starting with :. It supports any of the following commands:
quit q quit exit the debugger debug show debug log display the internal debug log for the debugger itself reload reload program reloads the program from disk, and resets the UI (except breakpoints) breakpoint break, b create breakpoint see Breakpoints read r read memory inspect linear memory (see Reading Memory"},{"location":"usage/debugger/#breakpoints","title":"Breakpoints","text":"One of the most common things you will want to do with the debugger is set and manage breakpoints. Using the command prompt, you can create breakpoints by typing b (or break or breakpoint), followed by a space, and then the desired breakpoint expression to do any of the following:
The syntax for each of these can be found below, in the same order (shown using b as the command):
b FILE[:LINE] Break when an instruction with a source location in FILE (a glob pattern) and that occur on LINE (literal, if provided) are hit. b in NAME Break when the glob pattern NAME matches the fully-qualified procedure name containing the current instruction b for OPCODE Break when the an instruction with opcode OPCODE is exactly matched (including immediate values) b next Break on the next instruction b after N Break after N cycles b at CYCLE Break when the cycle count reaches CYCLE. If CYCLE has already occurred, this has no effect When a breakpoint is hit, it will be highlighted, and the breakpoint window will display the number of hit breakpoints in the lower right.
After a breakpoint is hit, it expires if it is one of the following types:
When a breakpoint expires, it is removed from the breakpoint list on the next cycle.
"},{"location":"usage/debugger/#reading-memory","title":"Reading memory","text":"Another useful diagnostic task is examining the contents of linear memory, to verify that expected data has been written. You can do this via the command prompt, using r (or read), followed by a space, and then the desired memory address and options:
The format for read expressions is :r ADDR [OPTIONS..], where ADDR is a memory address in decimal or hexadecimal format (the latter requires the 0x prefix). The read command supports the following for OPTIONS:
-mode MODE -m words (word ,w)bytes (byte, b) words Specify a memory addressing mode -format FORMAT -f decimal (d)hex (x)binary (bin, b)decimal Specify the format used to print integral values -count N -c 1 Specify the number of units to read -type TYPE -t See Types word Specify the type of value to readThis also has the effect of modifying the default -format and unit size for -count Any invalid combination of options, or invalid syntax, will display an error in the status bar.
"},{"location":"usage/debugger/#types","title":"Types","text":"Type DescriptioniN A signed integer of N bits uN An unsigned integer of N bits felt A field element word A Miden word, i.e. an array of four field elements ptr or pointer A 32-bit memory address (implies -format hex)"},{"location":"usage/debugger/#roadmap","title":"Roadmap","text":"The following are some features planned for the near future:
read commandmidenc","text":"The midenc executable is the command-line interface for the compiler driver, as well as other helpful tools, such as the interactive debugger.
While it is a lower-level tool compared to cargo-miden, just like the difference between rustc and cargo, it provides a lot of functionality for emitting diagnostic information, controlling the output of the compiler, and configuring the compilation pipeline. Most users will want to use cargo-miden, but understanding midenc is helpful for those times where you need to get your hands dirty.
First, you\u2019ll need to have Rust installed, with the nightly toolchain (currently we\u2019re building against the nightly-2024-05-07 toolchain, but we regularly update this).
Then, simply install midenc using Cargo in one of the following ways:
# From crates.io:\ncargo +nightly install midenc\n\n# If you have cloned the git repo, and are in the project root:\ncargo make install\n\n# If you have Rust installed, but have not cloned the git repo:\ncargo install --git https://github.com/0xpolygonmiden/compiler midenc\nAdvice
This installation method relies on Cargo-managed binaries being in your shell PATH, which is almost always the case, but if you have disabled this functionality, you\u2019ll need to add midenc to your PATH manually.
Once installed, you should be able to invoke the compiler, you should see output similar to this:
midenc help compile\nUsage: midenc compile [OPTIONS] [-- <INPUTS>...]\n\nArguments:\n [INPUTS]...\n Path(s) to the source file(s) to compile.\n\n You may also use `-` as a file name to read a file from stdin.\n\nOptions:\n --output-dir <DIR>\n Write all compiler artifacts to DIR\n\n -W <LEVEL>\n Modify how warnings are treated by the compiler\n\n [default: auto]\n\n Possible values:\n - none: Disable all warnings\n - auto: Enable all warnings\n - error: Promotes warnings to errors\n\n -v, --verbose\n When set, produces more verbose output during compilation\n\n -h, --help\n Print help (see a summary with '-h')\nThe actual help output covers quite a bit more than shown here, this is just for illustrative purposes.
The midenc executable supports two primary functions at this time:
midenc compile to compile one of our supported input formats to Miden Assemblymidenc debug to run a Miden program attached to an interactive debuggermidenc run to run a Miden program non-interactively, equivalent to miden runSee the help output for midenc compile for detailed information on its options and their behavior. However, the following is an example of how one might use midenc compile in practice:
midenc compile --target rollup \\\n --entrypoint 'foo::main' \\\n -lextra \\\n -L ./masm \\\n --emit=hir=-,masp \\\n -o out.masp \\\n target/wasm32-wasip1/release/foo.wasm\nIn this scenario, we are in the root of a Rust crate, named foo, which we have compiled for the wasm32-wasip1 target, which placed the resulting WebAssembly module in the target/wasm32-wasip1/release directory. This crate exports a function named main, which we want to use as the entrypoint of the program.
Additionally, our Rust code links against some hand-written Miden Assembly code, namespaced under extra, which can be found in ./masm/extra. We are telling midenc to link the extra library, and to add the ./masm directory to the library search path.
Lastly, we\u2019re configuring the output:
--emit to request midenc to dump Miden IR (hir) to stdout (specified via the - shorthand), in addition to the Miden package artifact (masp).midenc to write the compiled output to out.masp in the current directory, rather than the default path that would have been used (target/miden/foo.masp).See Debugging Programs for details on using midenc debug to debug Miden programs.
We have put together two useful guides to walk through more detail on compiling Rust to WebAssembly:
midenc compile on the resulting Wasm module, see this guide.You may also be interested in our basic account project template, as a starting point for your own Rust project.
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Getting started","text":"Welcome to the documentation for the Miden compiler toolchain.
Warning
The compiler is currently in an experimental state, and has known bugs and limitations, it is not yet ready for production usage. However, we\u2019d encourage you to start experimenting with it yourself, and give us feedback on any issues or sharp edges you encounter.
The documentation found here should provide a good starting point for the current capabilities of the toolchain, however if you find something that is not covered, but is not listed as unimplemented or a known limitation, please let us know by reporting an issue on the compiler issue tracker.
"},{"location":"#what-is-provided","title":"What is provided?","text":"The compiler toolchain consists of the following primary components:
rustc, which is largely just what LLVM produces, but with the shadow stack placed at the start of linear memory rather than after read-only data. In the future we intend to support more variety in the structure of Wasm modules we accept, but for the time being we\u2019re primarily focused on using this as the path for lowering Rust to Miden.midenc executable, and a Rust crate, midenc-compiler to allow integrating the compiler into other tools. This plays the same role as rustc does in the Rust ecosystem.cargo-miden, that provides a convenient developer experience for creating and compiling Rust projects targeting Miden. It contains a project template for a basic Rust crate, and handles orchestrating rustc and midenc to compile the crate to WebAssembly, and then to Miden Assembly.midenc debug, which provides a UI very similar to lldb or gdb when using the TUI mode. You can use this to run a program, or step through it cycle-by-cycle. You can set various types of breakpoints; see the source code, call stack, and contents of the operand stack at the current program point; as well as interatively read memory and format it in various ways for display.cargo miden new automatically adds this as a dependency.That all sounds great, but what can you do with the compiler today? The answer depends a bit on what aspect of the compiler you are interested in:
"},{"location":"#rust","title":"Rust","text":"The most practically useful, and interesting capability provided by the compiler currently, is the ability to compile arbitrary Rust programs to Miden Assembly. See the guides for more information on setting up and compiling a Rust crate for execution via Miden.
"},{"location":"#webassembly","title":"WebAssembly","text":"More generally, the compiler frontend is capable of compiling WebAssembly modules, with some constraints, to Miden Assembly. As a result, it is possible to compile a wider variety of languages to Miden Assembly than just Rust, so long as the language can compile to WebAssembly. However, we do not currently provide any of the language-level support for languages other than Rust, and have limited ability to provide engineering support for languages other than Rust at this time.
Our Wasm frontend does not support all of the extensions to the WebAssembly MVP, most notably the reference types and GC proposals.
"},{"location":"#miden-ir","title":"Miden IR","text":"If you are interested in compiling to Miden from your own compiler, you can target Miden IR, and invoke the driver from your compiler to emit Miden artifacts. At this point in time, we don\u2019t have the resources to provide much in the way of engineering support for this use case, but if you find issues in your efforts to use the IR in your compiler, we would certainly like to know about them!
We do not currently perform any optimizations on the IR, since we are primarily working with the output of compiler backends which have already applied optimizations, at this time. This may change in the future, but for now it is expected that you implement your own optimization passes as needed.
"},{"location":"#known-bugs-and-limitations","title":"Known bugs and limitations","text":"For the latest information on known bugs, see the issue tracker.
See Known Limitations for details on what functionality is missing or only partially implemented.
"},{"location":"#where-to-start","title":"Where to start?","text":"Provided here are a set of guides which are focused on documenting a couple of supported workflows we expect will meet the needs of most users, within the constraints of the current feature set of the compiler. If you find that there is something you wish to do that is not covered, and is not one of our known limitations, please open an issue, and we will try to address the missing docs as soon as possible.
"},{"location":"#installation","title":"Installation","text":"To get started, there are a few ways you might use the Miden compiler. Select the one that applies to you, and the corresponding guide will walk you through getting up and running:
midenc executableThis document describes the various calling conventions recognized/handled by the compiler, including a specification for the interaction with the IR type system.
There are four calling conventions represented in the compiler:
C aka SystemV, which corresponds to the C ABI commonly used for C foreign-function interfaces (FFI). We specifically use the System V ABI because it is well understood, documented, and straightforward.Fast, this convention allows the compiler to follow either the C calling convention, or modify it as it sees fit on a function-by-function basis. This convention provides no guarantees about how a callee will expect arguments to be passed, so should not be used for functions which are expected to have a stable, predictable interface. This is a good choice for local functions, or functions which are only used within an executable/library and are not part of the public interface.Kernel, this is a special calling convention that is used when defining kernel modules in the IR. Functions which are part of the kernel\u2019s public API are required to use this convention, and it is not possible to call a function via syscall if the callee is not defined with this convention. Because of the semantics of syscall, this convention is highly restrictive. In particular, it is not permitted to pass pointer arguments, or aggregates containing pointers, as syscall involves a context switch, and thus memory in the caller is not accessible to the callee, and vice versa.Contract, this is a special calling convention that is used when defining smart contract functions, i.e. functions that can be call\u2018d. The compiler will not permit you to call a function if the callee is not defined with this convention, and functions with this convention cannot be called via exec. Like syscall, the call instruction involves a context switch, however, unlike the Kernel convention, the Contract convention is allowed to have types in its signature that are/contain pointers, with certain caveats around those pointers.All four conventions above are based on the System V C ABI, tailored to the Miden VM. The only exception is Fast, which may modify the ABI arbitrarily as it sees fit, and makes no guarantees about what modifications, if any, it will make.
The following is a description of how the IR type system is represented in the C calling convention. Later, a description of how the other conventions extend/restrict/modify this representation will be provided.
sizeof Alignment (bytes) Miden Type Integer _Bool/bool I1 1 1 u32 Integer char, signed char I8 1 1 i321 Integer unsigned char U8 1 1 u32 Integer short / signed short I16 2 2 i321 Integer unsigned short U16 2 2 u32 Integer int / signed int / enum I32 4 4 i3218 Integer unsigned int U32 4 4 u32 Integer long / signed long I32 4 4 i321 Integer unsigned long / size_t U32 4 4 u32 Integer long long / signed long long I64 8 8 i642 Integer unsigned long long U64 8 8 u643 Pointer any-type * / any-type (*)() Ptr(_) 4 4 u3267 Floating point float F32 4 4 u324 Floating point double F64 8 8 u644 Floating point long double 16 16 (none)5 Note
The compiler does not support scalars larger than one word (128 bits) at this time. As a result, anything that is larger than that must be allocated in linear memory, or in an automatic allocation (function-local memory), and passed around by reference.
The native scalar type for the Miden VM is a \u201cfield element\u201d, specifically a 64-bit value representing an integer in the \u201cGoldilocks\u201d field, i.e. 0..(2^64-2^32+1). A number of instructions in the VM operate on field elements directly. However, the native integral/pointer type, i.e. a \u201cmachine word\u201d, is actually u32. This is because a field element can fully represent 32-bit integers, but not the full 64-bit integer range. Values of u32 type are valid field element values, and can be used anywhere that a field element is expected (barring other constraints).
Miden also has the notion of a \u201cword\u201d, not to be confused with a \u201cmachine word\u201d (by which we mean the native integral type used to represent pointers), which corresponds to a set of 4 field elements. Words are commonly used in Miden, particularly to represent hashes, and a number of VM instructions operate on word-sized operands. As an aside, 128-bit integer values are represented using a word, or two 64-bit limbs (each limb consisting of two 32-bit limbs).
All integral types mentioned above, barring field elements, use two\u2019s complement encoding. Unsigned integral types make use of the sign bit to change the value range (i.e. 0..2^32-1, rather than -231..231-1), but the encoding follows two\u2019s complement rules.
The Miden VM only has native support for field elements, words, and u32; all other types are implemented in software using intrinsics.
Structures and unions assume the alignment of their most strictly aligned component. Each member is assigned to the lowest available offset with the appropriate alignment. The size of any object is always a multiple of the object\u2019s alignment. An array uses the same alignment as its elements. Structure and union objects can require padding to meet size and alignment constraints. The contents of any padding is undefined.
"},{"location":"appendix/calling-conventions/#memory-model","title":"Memory model","text":"Interacting with memory in Miden is quite similar to WebAssembly in some ways:
This is where it begins to differ though, and takes on qualities unique to Miden (in part, or whole):
call vs syscall vs exec. The first two perform a context switch when transferring control to the callee, and the callee has no access to the caller\u2019s memory (and the caller has no access to the callee\u2019s memory). As a result, references to memory cannot be passed from caller to callee in arguments, nor can they be returned from the callee to the caller.This presents some complications, particularly:
load instruction. Operations like this must be translated into instruction sequences that load whole words from memory, extract the data needed, and discard the unused bits. This makes the choice of where in memory to store something much more important than byte-addressable memory, as loads of values which are not aligned to element or word boundaries can be quite inefficient in some cases.The compiler solves this by providing a byte-addressable IR, and internally translating operations in the IR to the equivalent sequence of Miden instructions needed to emulate that operation. This translation is done during code generation, and uses the following semantics to determine how a particular operation gets lowered:
Because we\u2019re essentially emulating byte-addressable memory on word-addressable memory, loads/stores can range from simple and straightforward, to expensive and complicated, depending on the size and alignment of the value type. The process goes as follows:
The worst case scenario for an unaligned load or store involves a word-sized type starting somewhere in the last element of the first word. This will require loading elements from three consecutive words, plus a lot of shuffling bits around to get the final, aligned word-sized value on the operand stack. Luckily, such operations should be quite rare, as by default all word-sized scalar types are word-aligned or element-aligned, so an unaligned load or store would require either a packed struct, or a type such as an array of bytes starting at some arbitrary address. In practice, most loads/stores are likely to be element-aligned, so most overhead from emulation will come from values which cross an element or word boundary.
"},{"location":"appendix/calling-conventions/#function-calls","title":"Function calls","text":"This section describes the conventions followed when executing a function call via exec, including how arguments are passed on the operand stack, stack frames, etc. Later, we\u2019ll cover the differences when executing calls via call or syscall.
Miden does not have registers in the style of hardware architectures. Instead it has an operand stack, on which an arbitrary number of operands may be stored, and local variables. In both cases - an operand on the operand stack, or a single local variable - the value type is nominally a field element, but it is easier to reason about them as untyped element-sized values. The operand stack is used for function arguments, return values, temporary variables, and scratch space. Local variables are not always used, but are typically used to hold multiply-used values which you don\u2019t want to keep on the operand stack, function-scoped automatic allocations (i.e. alloca), and other such uses.
Miden does not have a stack frame per se. When you call a procedure in Miden Assembly, any local variables declared by that procedure are allocated space in a reserved region of linear memory in a single consecutive chunk. However, there is no stack or frame pointer, and because Miden is a Harvard architecture machine, there are no return addresses. Instead, languages (such as C) which have the concept of a stack frame with implications for the semantics of say, taking the address of a local variable, will need to emit code in function prologues and epilogues to maintain a shadow stack in Miden\u2019s linear memory. If all you need is local variables, you can get away with leaning on Miden\u2019s notion of local variables without implementing a shadow stack.
Because there are no registers, the notion of callee-saved or caller-saved registers does not have a direct equivalent in Miden. However, in its place, a somewhat equivalent set of rules defines the contract between caller and callee in terms of the state of the operand stack, those are described below in the section covering the operand stack.
"},{"location":"appendix/calling-conventions/#the-shadow-stack","title":"The shadow stack","text":"Miden is a Harvard architecture; as such, code and data are not in the same memory space. More precisely, in Miden, code is only addressable via the hash of the MAST root of that code, which must correspond to code that has been loaded into the VM. The hash of the MAST root of a function can be used to call that function both directly and indirectly, but that is the only action you can take with it. Code can not be generated and called on the fly, and it is not stored anywhere that is accessible to code that is currently executing.
One consequence of this is that there are no return addresses or instruction pointers visible to executing code. The runtime call stack is managed by the VM itself, and is not exposed to executing code in any way. This means that address-taken local C variables need to be on a separate stack in linear memory (which we refer to as a \u201cshadow stack\u201d). Not all functions necessarily require a frame in the shadow stack, as it cannot be used to perform unwinding, so only functions which have locals require a frame.
The Miden VM actually provides some built-in support for stack frames when using Miden Assembly. Procedures which are declared with some number of locals, will be automatically allocated sufficient space for those locals in a reserved region of linear memory when called. If you use the locaddr instruction to get the actual address of a local, that address can be passed as an argument to callees (within the constraints of the callee\u2019s calling convention).
Languages with more elaborate requirements with regard to the stack will need to implement their own shadow stack, and emit code in function prologues/epilogues to manage it.
"},{"location":"appendix/calling-conventions/#the-operand-stack","title":"The operand stack","text":"The Miden virtual machine is a stack machine, not a register machine. Rather than having a fixed set of registers that are used to store and manipulate scalar values, the Miden VM has the operand stack, which can hold an arbitrary number of operands (where each operand is a single field element), of which the first 16 can be directly manipulated using special stack instructions. The operand stack is, as the name implies, a last-in/first-out data structure.
The following are basic rules all conventions are expected to follow with regard to the operand stack:
Miden Abstract Syntax Trees (MASTs) do not have any notion of functions, and as such are not aware of parameters, return values, etc. For this document, that\u2019s not a useful level of abstraction to examine. Even a step higher, Miden Assembly (MASM) has functions (procedures in MASM parlance), but no function signature, i.e. given a MASM procedure, there is no way to know how many arguments it expects, how many values it returns, let alone the types of arguments/return values. Instead, we\u2019re going to specify calling conventions in terms of Miden IR, which has a fairly expressive type system more or less equivalent to that of LLVM, and how that translates to Miden primitives.
Functions in Miden IR always have a signature, which specify the following:
The following table relates IR types to how they are expected to be passed from the caller to the callee, and vice versa:
Type Parameter Result scalar direct direct empty struct or union1 ignored ignored scalar struct or union2 direct direct other struct or union indirect indirect array indirect N/AThe compiler will automatically generate code that follows these rules, but if emitting MASM from your own backend, it is necessary to do so manually. For example, a function whose signature specifies that it returns a non-scalar struct by value, must actually be written such that it expects to receive a pointer to memory allocated by the caller sufficient to hold the return value, as the first parameter of the function (i.e. the parameter is prepended to the parameter list). When returning, the function must write the return value to that pointer, rather than returning it on the operand stack. In this example, the return value is returned indirectly (by reference).
A universal rule is that the arguments are passed in reverse order, i.e. the first argument in the parameter list of a function will be on top of the operand stack. This is different than many Miden instructions which seemingly use the opposite convention, e.g. add, which expects the right-hand operand on top of the stack, so a + b is represented like push a, push b, add. If we were to implement add as a function, it would instead be push b, push a, exec.add. The rationale behind this is that, in general, the more frequently used arguments appear earlier in the parameter list, and thus we want those closer to the top of the operand stack to reduce the amount of stack manipulation we need to do.
Arguments/return values are laid out on the operand stack just like they would be as if you had just loaded it from memory, so all arguments are aligned, but may span multiple operands on the operand stack as necessary based on the size of the type (i.e. a struct type that contains a u32 and a i1 field would require two operands to represent). If the maximum number of operands allowed for the call is reached, any remaining arguments must be spilled to the caller\u2019s stack frame, or to the advice provider. The former is used in the case of exec/dynexec, while the latter is used for call and syscall, as caller memory is not accessible to the callee with those instructions.
While ostensibly 16 elements is the maximum number of operands on the operand stack that can represent function arguments, due to the way dynexec/dyncall work, it is actually limited to 12 elements, because at least 4 must be free to hold the hash of the function being indirectly called.
Zero-sized types have no representation in memory, so they are ignored/skipped\u00a0\u21a9\u21a9\u21a9\u21a9\u21a9
Any struct or union that recursively (including through nested structs, unions, and arrays) contains just a single scalar value and is not specified to have greater than natural alignment.\u00a0\u21a9\u21a9
u64 is not a native Miden type, but is implemented in software using two 32-bit limbs (i.e. a pair of field elements)\u00a0\u21a9
floating-point types are not currently supported, but will be implemented using compiler intrinsics\u00a0\u21a9\u21a9
long double values correspond to 128-bit IEEE-754 quad-precision binary128 values. These are not currently supported, and we have no plans to support them in the near term. Should we ever provide such support, we will do so using compiler intrinsics.\u00a0\u21a9
A null pointer (for all types) always has the value zero.\u00a0\u21a9
Miden\u2019s linear memory is word-addressable, not byte-addressable. The Ptr type has an AddressSpace parameter, that by default is set to the byte-addressable address space. The compiler translates values of Ptr type that are in this address space, into the Miden-native, word-addressable address space during codegen of load/store operations. See the section on the memory model below for more details.\u00a0\u21a9
An enum is i32 if all members of the enumeration can be represented by an int/unsigned int, otherwise it uses i64.\u00a0\u21a9
This document describes an issue that arises when trying to map the ad-hoc calling convention/ABI used by various Miden Assembly procedures, such as those comprising the transaction kernel, and the \u201ccanonical\u201d ABI(s) representable in Rust. It proposes a solution to this problem in the form of adapter functions, where the details of a given adapter are one of a closed set of known ABI transformation strategies.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#summary","title":"Summary","text":"The gist of the problem is that in Miden, the size and number of procedure results is only constrained by the maximum addressable operand stack depth. In most programming languages, particularly those in which interop is typically performed using some variant of the C ABI (commonly the one described in the System V specification), the number of results is almost always limited to a single result, and the size of the result type is almost always limited to the size of a single machine word, in some cases two. On these platforms, procedure results of greater arity or size are typically handled by reserving space in the caller\u2019s stack frame, and implicitly prepending the parameter list of the callee with an extra parameter: a pointer to the memory allocated for the return value. The callee will directly write the return value via this pointer, instead of returning a value in a register.
In the case of Rust, this means that attempting to represent a procedure that returns multiple values, or returns a larger-than-machine-word type, such as Word, will trigger the implicit transformation described above, as this is allowed by the standard Rust calling conventions. Since various Miden procedures that are part of the standard library and the transaction kernel are affected by this, the question becomes \u201chow do we define bindings for these procedures in Rust?\u201d.
The solution is to have the compiler emit glue code that closes the gap between the two ABIs. It does so by generating adapter functions, which wrap functions that have an ABI unrepresentable in Rust, and orchestrate lifting/lowering arguments and results between the adapter and the \u201creal\u201d function.
When type signatures are available for all Miden Assembly procedures, we can completely automate this process. For now, we will require a manually curated list of known procedures, their signatures, and the strategy used to \u201cadapt\u201d those procedures for binding in Rust.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#background","title":"Background","text":"After analyzing all of the functions in the transaction kernel API, the most common cause of a mismatch between Miden and Rust ABIs, is due to implicit \u201csret\u201d parameters, i.e. the transformation mentioned above which inserts an implicit pointer to the caller\u2019s stack frame for the callee to write the return value to, rather than doing so in a register (or in our case, on the operand stack). This seems to happen for any type that is larger than 8 bytes (i64).
Tip
For a complete list of the transaction kernel functions, in WIT format, see miden.wit.
For most transaction kernel functions, the adapter function can be generated automatically using the pattern recognition and adapter functions described below.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#prerequisites","title":"Prerequisites","text":"The compiler will analyze every component import to determine if that import requires an adapter, as determined by matching against a predefined set of patterns. The adapter generation will take place in the frontend, as it has access to all of the needed information, and ensures that we do not have any transformations or analyses that make decisions on the un-adapted procedure.
The following pseudo-code can be used to recognize the various Miden ABI patterns:
pub enum MidenAbiPattern {\n /// Calling this procedure will require an sret parameter on the Rust side, so\n /// we need to emit an adapter that will lift/lower calls according to that\n /// strategy.\n ReturnViaPointer,\n /// The underlying procedure is fully representable in Rust, and requires no adaptation.\n NoAdapterNeeded,\n}\n\npub struct MidenAbiPatternRecognition {\n pattern: Option<MidenAbiPattern>,\n component_function: ComponentFunctionType,\n wasm_core_func: Signature,\n tx_kernel_function: Signature,\n}\n\npub fn recognize_miden_abi_pattern(\n component_function: &ComponentFunctionType,\n wasm_core_func: &Signature,\n tx_kernel_func: &Signature) -> MidenAbiPatternRecognition {\n if wasm_core_func == tx_kernel_func {\n return MidenAbiPatternRecognition {\n pattern: Some(NoAdapterNeeded),\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n } else if component_function.returns[0].byte_size > 8 && wasm_core_func.params.last() == I32 {\n return MidenAbiPatternRecognition {\n pattern: Some(ReturnViaPointer),\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n } else {\n return MidenAbiPatternRecognition {\n pattern: None,\n component_function,\n wasm_core_function,\n tx_kernel_function,\n };\n }\n}\nThe following pseudo-code can then be used to generate the adapter function:
pub fn generate_adapter(recognition: MidenAbiPatternRecognition) {\n match recognition.pattern {\n Some(pattern) => generate_adapter(\n pattern,\n recognition.component_function,\n recognition.wasm_core_function,\n recognition.tx_kernel_function\n ),\n None => use_manual_adapter(\n recognition.component_function,\n recognition.wasm_core_function,\n recognition.tx_kernel_function\n ),\n }\n}\n\n/// Escape hatch for the cases when the compiler can't generate an adapter function automatically\n/// and we need to provide the adapter function manually.\npub fn use_manual_adapter(...) {\n // Find and use the manual adapter in the adapter library for the tx_kernel_function\n}\nThe manual adapter library is a collection of adapter functions that are used when the compiler can\u2019t generate an adapter function automatically so its expected to be provided. The manual adapter library is a part of the Miden compiler. It is not anticipated that we will have many, or any, of these; however in the near term we are going to manually map procedures to their adapter strategies, as we have not yet automated the pattern recognition step.
"},{"location":"appendix/canonabi-adhocabi-mismatch/#return-via-pointer-adapter","title":"Return-via-pointer adapter","text":"The return value is expected to be returned by storing its flattened representation in a pointer passed as an argument.
Recognize this Miden ABI pattern by looking at the Wasm component function type. If the return value is bigger than 64 bits, expect the last argument in the Wasm core(HIR) signature to be i32 (a pointer).
The adapter function calls the tx kernel function and stores the result in the provided pointer (the last argument of the Wasm core function).
Here is the pseudo-code for generating the adapter function for the return-via-pointer Miden ABI pattern:
let ptr = wasm_core_function.params.last();\nlet adapter_function = FunctionBuilder::new(wasm_core_function.clone());\nlet tx_kernel_function_params = wasm_core_function.params.drop_last();\nlet tx_kernel_func_val = adapter_function.call(tx_kernel_function, tx_kernel_function_params);\nadapter_function.store(tx_kernel_func_val, ptr);\nadapter_function.build();\nHere is how the adapter might look like in a pseudo-code for the add_asset function:
/// Takes an Asset as an argument and returns a new Asset\nfunc wasm_core_add_asset(v0: f64, v1: f64, v2: f64, v3: f64, ptr: i32) {\n v4 = call tx_kernel_add_asset(v0, v1, v2, v3);\n // v4 is a tuple of 4 f64 values\n store v4 in ptr;\n}\nNo adapter is needed. The Wasm core function type is the same as the tx kernel ad-hoc signature.
This Miden ABI pattern is selected if no other Miden ABI pattern is applicable and the wasm core function signature is the same as the tx kernel ad-hoc signature.
For example, the get_id function falls under this Miden ABI pattern and its calls will be translated to the tx kernel function calls without any modifications.
get_assets","text":"get_assets:func() -> list<core-asset> in the note interface is the only function that requires attention. In Canonical ABI, any function that returns a dynamic list of items needs to allocate memory in the caller\u2019s module due to the shared-nothing nature of the Wasm component model. For this case, a realloc function is passed as a part of lift/lower Canonical ABI options for the caller to allocate memory in the caller\u2019s module.
Here are the signatures of the get_assets function in the WIT, core Wasm, and the tx kernel ad-hoc ABI: Comment from the miden-base
#! Writes the assets of the currently executing note into memory starting at the specified address.\n#!\n#! Inputs: [dest_ptr]\n#! Outputs: [num_assets, dest_ptr]\n#!\n#! - dest_ptr is the memory address to write the assets.\n#! - num_assets is the number of assets in the currently executing note.\nWasm component function type: get-assets: func() -> list<core-asset>;
Wasm core signature: wasm_core_get_assets(i32) -> ()
If we add a new get_assets_count: func() -> u32; function to the tx kernel and add the assets count parameter to the get_assets function (get_assets: func(assets_count: u32) -> list<core-asset>;) we should have everything we need to manually write the adapter function for the get_assets function.
The list is expected to be returned by storing the pointer to its first item in a ptr pointer passed as an argument and item count at ptr + 4 bytes address (ptr points to two pointers).
We could try to recognize this Miden ABI pattern by looking at the Wasm component function type. If the return value is a list, expect the last argument in the Wasm core(HIR) signature to be i32 (a pointer). The problem is recognizing the list count parameter in the Wasm core(HIR) signature.
The adapter function calls allocates asset_count * item_size memory via the realloc call and passes the pointer to the newly allocated memory to the tx kernel function.
Here is how the adapter function might look like in a pseudo-code for the get_assets function:
func wasm_core_get_assets(asset_count: u32, ptr_ptr: i32) {\n mem_size = asset_count * item_size;\n ptr = realloc(mem_size);\n (actual_asset_count, ptr) = call tx_kernel_get_assets(ptr);\n assert(actual_asset_count == asset_count);\n store ptr in ptr_ptr;\n store account_count in ptr_ptr + 4;\n}\nNote
Since the get_assets tx kernel function in the current form can trash the provided memory if the actual assets count differs from the returned by get_assets_count, we can introduce the asset count parameter to the get_assets tx kernel function and check that it the same as the actual assets count written to memory.
add_asset (return-via-pointer Miden ABI pattern)","text":"Comment from the miden-base
#! Add the specified asset to the vault.\n#!\n#! Panics:\n#! - If the asset is not valid.\n#! - If the total value of two fungible assets is greater than or equal to 2^63.\n#! - If the vault already contains the same non-fungible asset.\n#!\n#! Stack: [ASSET]\n#! Output: [ASSET']\n#!\n#! - ASSET' final asset in the account vault is defined as follows:\n#! - If ASSET is a non-fungible asset, then ASSET' is the same as ASSET.\n#! - If ASSET is a fungible asset, then ASSET' is the total fungible asset in the account vault\n#! after ASSET was added to it.\nWasm component function type: add-asset(core-asset) -> core-asset
Wasm core signature: wasm_core_add_asset(f64, f64, f64, f64, i32) -> () The last i32 is a pointer to a returned value (word)
Tx kernel ad-hoc signature: tx_kernel_add_asset(felt, felt, felt, felt) -> (felt, felt, felt, felt)
get_id (no-adapter-needed Miden ABI pattern)","text":"Comment from the miden-base
#! Returns the account id.\n#!\n#! Stack: []\n#! Output: [acct_id]\n#!\n#! - acct_id is the account id.\nWasm component function type: get-id() -> account-id
Wasm core signature: wasm_core_get_id() -> f64
Tx kernel ad-hoc signature: tx_kernel_get_id() -> felt
Tip
See the issue tracker for information on known bugs. This document focuses on missing/incomplete features, rather than bugs.
The compiler is still in its early stages of development, so there are various features that are unimplemented, or only partially implemented, and the test suite is still limited in scope, so we are still finding bugs on a regular basis. We are rapidly improving this situation, but it is important to be aware of this when using the compiler.
The features discussed below are broken up into sections, to make them easier to navigate and reference.
"},{"location":"appendix/known-limitations/#rust-language-support","title":"Rust language support","text":""},{"location":"appendix/known-limitations/#floating-point-types","title":"Floating point types","text":"In order to represent Felt \u201cnatively\u201d in Rust, we were forced to piggy-back on the f32 type, which is propagated through to WebAssembly, and allows us to handle those values specially.
As a result, floating-point types in Rust are not supported at all. Any attempt to use them will result in a compilation error. We considered this a fair design tradeoff, as floating point math is unused/rare in the context in which Miden is used, in comparison to fixed-point or field arithmetic. In addition, implementing floating-point operations in software on the Miden VM would be extraordinarily expensive, which generally works against the purpose for using floats in the first place.
At this point in time, we have no plans to support floats, but this may change if we are able to find a better/more natural representation for Felt in WebAssembly.
This feature corresponds to call_indirect in WebAssembly, and is associated with Rust features such as trait objects (which use indirection to call trait methods), and closures. Note that the Rust compiler is able to erase the indirection associated with certain abstractions statically in some cases, shown below. If Rust is unable to statically resolve all call targets, then midenc will raise an error when it encounters any use of call_indirect.
Warning
The following examples rely on rustc/LLVM inlining enough code to be able to convert indirect calls to direct calls. This may require you to enable link-time optimization with lto = \"fat\" and compile all of the code in the crate together with codegen-units = 1, in order to maximize the amount of inlining that can occur. Even then, it may not be possible to remove some forms of indirection, in which case you will need to find another workaround.
pub fn is_zeroed(bytes: &[u8; 32]) -> bool {\n // Rust is able to convert this to a loop, erasing the closure completely\n bytes.iter().copied().all(|b| b == 0)\n}\npub fn call<F, T>(fun: F) -> T\nwhere\n F: Fn() -> T,\n{\n fun()\n}\n\n#[inline(never)]\npub fn foo() -> bool { true }\n\nfn main() {\n // Rust is able to inline the body of `call` after monomorphization, which results in\n // the call to `foo` being resolved statically.\n call(foo)\n}\npub trait Foo {\n fn is_foo(&self) -> bool;\n}\n\nimpl Foo for u32 {\n #[inline(never)]\n fn is_foo(&self) -> bool { true }\n}\n\nfn has_foo(items: &[dyn Foo]) -> bool {\n items.iter().any(|item| item.is_foo())\n}\n\nfn main() -> u32 {\n // Rust inlines `has_foo`, converts the iterator chain to a loop, and is able to realize\n // that the `dyn Foo` items are actually `u32`, and resolves the call to `is_foo` to\n // `<u32 as Foo>::is_foo`.\n let foo: &dyn Foo = &u32::MAX as &dyn Foo;\n has_foo(&[foo]) as u32\n}\nThe Miden SDK for Rust, is a Rust crate that provides the implementation of native Miden types, as well as bindings to the Miden standard library and transaction kernel APIs.
Currently, only a very limited subset of the API surface has had bindings implemented. This means that there is a fair amount of native Miden functionality that is not yet available from Rust. We will be expanding the SDK rapidly over the next few weeks and months, but for the time being, if you encounter a missing API that you need, let us know, so we can ensure it is prioritized above APIs which are lesser used.
"},{"location":"appendix/known-limitations/#rustmiden-ffi-foreign-function-interface-and-interop","title":"Rust/Miden FFI (foreign function interface) and interop","text":"While the compiler has functionality to link against native Miden Assembly libraries, binding against procedures exported from those libraries from Rust can require glue code to be emitted by the compiler in some cases, and the set of procedures for which this is done is currently restricted to a hardcoded whitelist of known Miden procedures.
This affects any procedure which returns a type larger than u32 (excluding Felt, which for this purpose has the same size). For example, returing a Miden Word from a procedure, a common return type, is not compatible with Rust\u2019s ABI - it will attempt to generate code which allocates stack space in the caller, which it expects the callee to write to, inserting a new parameter at the start of the parameter list, and expecting nothing to be returned by value. The compiler handles situations like these using a set of ABI \u201ctransformation strategies\u201d, which lift/lower differences between the Rust and Miden ABIs at call boundaries.
To expose the FFI machinery for use with any Miden procedure, we need type signatures for those procedures at a minimum, and in some cases we may require details of the calling convention/ABI. This metadata does not currently exist, but is on the roadmap for inclusion into Miden Assembly and Miden packaging. Once present, we can open up the FFI for general use.
"},{"location":"appendix/known-limitations/#core-miden-functionality","title":"Core Miden functionality","text":""},{"location":"appendix/known-limitations/#dynamic-procedure-invocation","title":"Dynamic procedure invocation","text":"This is a dependency of Function Call Indirection described above, and is the mechanism by which we can perform indirect calls in Miden. In order to implement support for indirect calls in the Wasm frontend, we need underlying support for dynexec, which is not yet implemented.
This feature adds support for lowering indirect calls to dynexec or dyncall instructions, depending on the ABI of the callee. dyncall has an additional dependency on support for Cross-Context Procedure Invocation.
A known issue with this feature is that dyn(exec|call) consumes a word on the operand stack for the hash of the callee being invoked, but this word remains on the stack when entering the callee, which has the effect of requiring procedures to have a different ABI depending on whether they expect to be dynamically-invoked or not.
Our solution to that issue is to generate stubs which are used as the target of dyn(exec|call), the body of which drop the callee hash, fix up the operand stack as necessary, and then uses a simple exec or call to invoke the \u201creal\u201d callee. We will emit a single stub for every function which has its \u201caddress\u201d taken, and use the hash of the stub in place of the actual callee hash.
This is required in order to support representing Miden accounts and note scripts in Rust, and compilation to Miden Assembly.
Currently, you can write code in Rust that is very close to how accounts and note scripts will look like in the language, but it is not possible to actually implement either of those in Rust today. The reasons for this are covered in depth in the tracking issue linked above, but to briefly summarize, the primary issue has to do with the fact that Rust programs are compiled for a \u201cshared-everything\u201d environment, i.e. you can pass references to memory from caller to callee, write to caller memory from the callee, etc. In Miden however, contexts are \u201cshared-nothing\u201d units of isolation, and thus cross-context operations, such as performing a call from a note script to a method on an account, are not compatible with the usual calling conventions used by Rust and LLVM.
The solution to this relies on compiling the Rust code for the wasm32-wasip2 target, which emits a new kind of WebAssembly module, known as a component. These components adhere to the rules of the WebAssembly Component Model. Of primary interest to us, is the fact that components in this model are \u201cshared-nothing\u201d, and the ABI used to communicate across component boundaries, is specially designed to enforce shared-nothing semantics on caller and callee. In addition to compiling for a specific Wasm target, we also rely on some additional tooling for describing component interfaces, types, and to generate Rust bindings for those descriptions, to ensure that calls across the boundary remain opaque, even to the linker, which ensures that the assumptions of the caller and callee with regard to what address space they operate in are preserved (i.e. a callee can never be inlined into the caller, and thus end up executing in the caller\u2019s context rather than the expected callee context).
This is one of our top priorities, as it is critical to being able to use Rust to compile code for the Miden rollup, but it is also the most complex feature on our roadmap, hence why it is scheduled for our Beta 2 milestone, rather than Beta 1 (the next release), as it depends on multiple other subfeatures being implemented first.
"},{"location":"appendix/known-limitations/#packaging","title":"Packaging","text":""},{"location":"appendix/known-limitations/#package-format","title":"Package format","text":"This feature represents the ability to compile and distribute a single artifact that contains the compiled MAST, and all required and optional metadata to make linking against, and executing packages as convenient as a dynamic library or executable.
The compiler currently produces, by default, an experimental implementation of a package format that meets the minimum requirements to support libraries and programs compiled from Rust:
However, this package format is not yet understood by the Miden VM itself. This means you cannot, currently, compile a package and then run it using miden run directly. Instead, you can use midenc run to load and run code from a package, as the compiler ships with the VM embedded for use with the interactive debugger, and provides native support for packaging on top of it. You can also use midenc debug to execute your program interactively in the debugger, depending on your needs. See Debugging Programs for more information on how to use the debugger, and midenc help run for more information on executing programs with the midenc run command.
While it is possible to emit raw MAST from midenc, rather than the experimental package format, the resulting artifact cannot be run without some fragile and error-prone manual setup, in order to ensure that the advice provider is correctly initialized with any read-only data segments. For now, it is recommended that you use the midenc tooling for testing programs, until the format is stabilized.
TODO
For the list of the unsupported Wasm core types, instructions and features, see the README.
"},{"location":"design/overview/","title":"Compiler architecture","text":"This is an index of various design documents for the compiler and its components. Some of these are planned topics, and some have documentation that hasn\u2019t been polished up yet. We\u2019ll slowly start to flesh out the documentation in this section as the compiler matures.
This chapter will walk through how to develop Miden programs in Rust using the standard library provided by the miden-stdlib-sys crate (see the README.
Import the standard library from the miden-stdlib-sys crate:
use miden_stdlib_sys::*;\nFelt (field element) type","text":"The Felt type is a field element type that is used to represent the field element values of the Miden VM.
To initialize a Felt value from an integer constant checking the range at compile time, use the felt! macro:
let a = felt!(42);\nOtherwise, use the Felt::new constructor:
let a = Felt::new(some_integer_var).unwrap();\nThe constructor returns an error if the value is not a valid field element, e.g. if it is not in the range 0..=M where M is the modulus of the field (2^64 - 2^32 + 1).
The Felt type implements the standard arithmetic operations, e.g. addition, subtraction, multiplication, division, etc. which are accessible through the standard Rust operators +, -, *, /, etc. All arithmetic operations are wrapping, i.e. performed modulo M.
TODO: Add examples of using operations on Felt type and available functions (assert*, etc.).
This chapter walks you through how to develop Miden rollup accounts and note scripts in Rust using the Miden SDK crate.
"},{"location":"guides/rust_to_wasm/","title":"Compiling Rust To WebAssembly","text":"This chapter will walk you through compiling a Rust crate to a WebAssembly (Wasm) module in binary (i.e. .wasm) form. The Miden compiler has a frontend which can take such modules and compile them on to Miden Assembly, which will be covered in the next chapter.
First, let\u2019s set up a simple Rust project that contains an implementation of the Fibonacci function (I know, it\u2019s overdone, but we\u2019re trying to keep things as simple as possible to make it easier to show the results at each step, so bear with me):
Start by creating a new library crate:
cargo new --lib wasm-fib && cd wasm-fib\nTo compile to WebAssembly, you must have the appropriate Rust toolchain installed, so let\u2019s add a toolchain file to our project root so that rustup and cargo will know what we need, and use them by default:
cat <<EOF > rust-toolchain.toml\n[toolchain]\nchannel = \"stable\"\ntargets = [\"wasm32-wasip1\"]\nEOF\nNext, edit the Cargo.toml file as follows:
[package]\nname = \"wasm-fib\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[lib]\n# Build this crate as a self-contained, C-style dynamic library\n# This is required to emit the proper Wasm module type\ncrate-type = [\"cdylib\"]\n\n[dependencies]\n# Use a tiny allocator in place of the default one, if we want\n# to make use of types in the `alloc` crate, e.g. String. We\n# don't need that now, but it's good information to have in hand.\n#miden-sdk-alloc = \"0.0.5\"\n\n# When we build for Wasm, we'll use the release profile\n[profile.release]\n# Explicitly disable panic infrastructure on Wasm, as\n# there is no proper support for them anyway, and it\n# ensures that panics do not pull in a bunch of standard\n# library code unintentionally\npanic = \"abort\"\n# Enable debug information so that we get useful debugging output\ndebug = true\n# Optimize the output for size\nopt-level = \"z\"\nMost of these things are done to keep the generated code size as small as possible. Miden is a target where the conventional wisdom about performance should be treated very carefully: we\u2019re almost always going to benefit from less code, even if conventionally that code would be less efficient, simply due to the difference in proving time accumulated due to extra instructions. That said, there are no hard and fast rules, but these defaults are good ones to start with.
Tip
We reference a simple bump allocator provided by miden-sdk-alloc above, but any simple allocator will do. The trade offs made by these small allocators are not generally suitable for long-running, or allocation-heavy applications, as they \u201cleak\u201d memory (generally because they make little to no attempt to recover freed allocations), however they are very useful for one-shot programs that do minimal allocation, which is going to be the typical case for Miden programs.
Next, edit src/lib.rs as shown below:
// Do not link against libstd (i.e. anything defined in `std::`)\n#![no_std]\n\n// However, we could still use some standard library types while\n// remaining no-std compatible, if we uncommented the following lines:\n//\n// extern crate alloc;\n// use alloc::{string::String, vec::Vec};\n\n// If we wanted to use the types mentioned above, it would also be\n// a good idea to use the allocator we pulled in as a dependency\n// in Cargo.toml, like so:\n//#[global_allocator]\n//static ALLOC: miden_sdk_alloc::BumpAlloc = miden_sdk_alloc::BumpAlloc::new();\n\n// Required for no-std crates\n#[panic_handler]\nfn panic(_info: &core::panic::PanicInfo) -> ! {\n // Compiles to a trap instruction in WebAssembly\n core::arch::wasm32::unreachable()\n}\n\n// Marking the function no_mangle ensures that it is exported\n// from the compiled binary as `fib`, otherwise it would have\n// a mangled name that has no stable form.\n//\n// You can specify a different name from the library than the\n// name in the source code using the `#[export_name = \"foo\"]`\n// attribute, which will make the function callable as `foo`\n// externally (in this example)\n#[no_mangle]\npub fn fib(n: u32) -> u32 {\n let mut a = 0;\n let mut b = 1;\n for _ in 0..n {\n let c = a + b;\n a = b;\n b = c;\n }\n a\n}\nThis exports our fib function from the library, making it callable from within a larger Miden program.
All that remains is to compile to WebAssembly:
cargo build --release --target=wasm32-wasip1\nThis places a wasm_fib.wasm file under the target/wasm32-wasip1/release/ directory, which we can then examine with wasm2wat to set the code we generated:
wasm2wat target/wasm32-wasip1/release/wasm_fib.wasm\nWhich dumps the following output (may differ slightly on your machine, depending on the specific compiler version):
(module $wasm_fib.wasm\n (type (;0;) (func (param i32) (result i32)))\n (func $fib (type 0) (param i32) (result i32)\n (local i32 i32 i32)\n i32.const 0\n local.set 1\n i32.const 1\n local.set 2\n loop (result i32) ;; label = @1\n local.get 2\n local.set 3\n block ;; label = @2\n local.get 0\n br_if 0 (;@2;)\n local.get 1\n return\n end\n local.get 0\n i32.const -1\n i32.add\n local.set 0\n local.get 1\n local.get 3\n i32.add\n local.set 2\n local.get 3\n local.set 1\n br 0 (;@1;)\n end)\n (memory (;0;) 16)\n (global $__stack_pointer (mut i32) (i32.const 1048576))\n (export \"memory\" (memory 0))\n (export \"fib\" (func $fib)))\nSuccess!
"},{"location":"guides/rust_to_wasm/#next-steps","title":"Next steps","text":"In Compiling WebAssembly to Miden Assembly, we walk through how to take the WebAssembly module we just compiled, and lower it to Miden Assembly using midenc!
This guide will walk you through compiling a WebAssembly (Wasm) module, in binary form (i.e. a .wasm file), to Miden Assembly (Masm), both in its binary package form (a .masp file), and in textual Miden Assembly syntax form (i.e. a .masm file).
We will be making use of the example crate we created in Compiling Rust to WebAssembly, which produces a small Wasm module that is easy to examine in Wasm text format, and demonstrates a good set of default choices for a project compiling to Miden Assembly from Rust.
In this chapter, we will be compiling Wasm to Masm using the midenc executable, so ensure that you have followed the instructions in the Getting Started with midenc guide and then return here.
Note
While we are using midenc for this guide, the more common use case will be to use the cargo-miden Cargo extension to handle the gritty details of compiling from Rust to Wasm for you. However, the purpose of this guide is to show you what cargo-miden is handling for you, and to give you a foundation for using midenc yourself if needed.
In the last chapter, we compiled a Rust crate to WebAssembly that contains an implementation of the Fibonacci function called fib, that was emitted to target/wasm32-wasip1/release/wasm_fib.wasm. All that remains is to tell midenc to compile this module to Miden Assembly.
Currently, by default, the compiler will emit an experimental package format that the Miden VM does not yet support. We will instead use midenc run to execute the package using the VM for us, but once the package format is stabilized, this same approach will work with miden run as well.
We also want to examine the Miden Assembly generated by the compiler, so we\u2019re going to ask the compiler to emit both types of artifacts:
midenc compile --emit masm=wasm_fib.masm,masp target/wasm32-wasip1/release/wasm_fib.wasm\nThis will compile our Wasm module to a Miden package with the .masp extension, and also emit the textual Masm to wasm_fib.masm so we can review it. The wasm_fib.masp file will be emitted in the default output directory, which is the current working directory by default.
If we dump the contents of wasm_fib.masm, we\u2019ll see the following generated code:
export.fib\n push.0\n push.1\n movup.2\n swap.1\n dup.1\n neq.0\n push.1\n while.true\n if.true\n push.4294967295\n movup.2\n swap.1\n u32wrapping_add\n dup.1\n swap.1\n swap.3\n swap.1\n u32wrapping_add\n movup.2\n swap.1\n dup.1\n neq.0\n push.1\n else\n drop\n drop\n push.0\n end\n end\nend\nIf you compare this to the WebAssembly text format, you can see that this is a fairly faithful translation, but there may be areas where we generate sub-optimal Miden Assembly.
Note
At the moment the compiler does only minimal optimization, late in the pipeline during codegen, and only in an effort to minimize operand stack management code. So if you see an instruction sequence you think is bad, bring it to our attention, and if it is something that we can solve as part of our overall optimization efforts, we will be sure to do so. There are limits to what we can generate compared to what one can write by hand, particularly because Rust\u2019s memory model requires us to emulate byte-addressable memory on top of Miden\u2019s word-addressable memory, however our goal is to keep this overhead within an acceptable bound in the general case, and easily-recognized patterns that can be simplified using peephole optimization are precisely the kind of thing we\u2019d like to know about, as those kinds of optimizations are likely to produce the most significant wins.
"},{"location":"guides/wasm_to_masm/#testing-with-the-miden-vm","title":"Testing with the Miden VM","text":"Note
Because the compiler ships with the VM embedded for midenc debug, you can run your program without having to install the VM separately, though you should do that as well, as midenc only exposes a limited set of commands for executing programs, intended for debugging.
We can test our compiled program like so:
$ midenc run --num-outputs 1 wasm_fib.masp -- 10\n============================================================\nRun program: wasm_fib.masp\n============================================================\nExecuted program with hash 0xe5ba88695040ec2477821b26190e9addbb1c9571ae30c564f5bbfd6cabf6c535 in 19 milliseconds\nOutput: [55]\nVM cycles: 295 extended to 512 steps (42% padding).\n\u251c\u2500\u2500 Stack rows: 295\n\u251c\u2500\u2500 Range checker rows: 67\n\u2514\u2500\u2500 Chiplets rows: 250\n\u251c\u2500\u2500 Hash chiplet rows: 248\n\u251c\u2500\u2500 Bitwise chiplet rows: 0\n\u251c\u2500\u2500 Memory chiplet rows: 1\n\u2514\u2500\u2500 Kernel ROM rows: 0\nSuccess! We got the expected result of 55.
This guide is not comprehensive, as we have not yet examined in detail the differences between compiling libraries vs programs, linking together multiple libraries, packages, or discussed some of the more esoteric compiler options. We will be updating this documentation with those details and more in the coming weeks and months, so bear with us while we flesh out our guides!
"},{"location":"usage/cargo-miden/","title":"Getting started with Cargo","text":"As part of the Miden compiler toolchain, we provide a Cargo extension, cargo-miden, which provides a template to spin up a new Miden project in Rust, and takes care of orchestrating rustc and midenc to compile the Rust crate to a Miden package.
Warning
Currently, midenc (and as a result, cargo-miden), requires the nightly Rust toolchain, so make sure you have it installed first:
rustup toolchain install nightly-2024-05-07\nNOTE: You can also use the latest nightly, but the specific nightly shown here is known to work.
To install the extension, simply run the following in your shell:
cargo +nightly-2024-05-07 install cargo-miden\nThis will take a minute to compile, but once complete, you can run cargo help miden or just cargo miden to see the set of available commands and options.
To get help for a specific command, use cargo miden help <command> or cargo miden <command> --help.
Your first step will be to create a new Rust project set up for compiling to Miden:
cargo miden new foo\nIn this above example, this will create a new directory foo, containing a Cargo project for a crate named foo, generated from our Miden project template.
The template we use sets things up so that you can pretty much just build and run. Since the toolchain depends on Rust\u2019s native WebAssembly target, it is set up just like a minimal WebAssembly crate, with some additional tweaks for Miden specfically.
Out of the box, you will get a Rust crate that depends on the Miden SDK, and sets the global allocator to a simple bump allocator we provide as part of the SDK, and is well suited for most Miden use cases, avoiding the overhead of more complex allocators.
As there is no panic infrastructure, panic = \"abort\" is set, and the panic handler is configured to use the native WebAssembly unreachable intrinsic, so the compiler will strip out all of the usual panic formatting code.
Now that you\u2019ve created your project, compiling it to Miden Assembly is as easy as running the following command from the root of the project directory:
cargo miden build --release\nThis will emit the compiled artifacts to target/miden.
A very useful tool in the Miden compiler suite, is its TUI-based interactive debugger, accessible via the midenc debug command.
Warning
The debugger is still quite new, and while very useful already, still has a fair number of UX annoyances. Please report any bugs you encounter, and we\u2019ll try to get them patched ASAP!
"},{"location":"usage/debugger/#getting-started","title":"Getting started","text":"The debugger is launched by executing midenc debug, and giving it a path to a program compiled by midenc compile. See Program Inputs for information on how to provide inputs to the program you wish to debug. Run midenc help debug for more detailed usage documentation.
The debugger may also be used as a library, but that is left as an exercise for the reader for now.
"},{"location":"usage/debugger/#example","title":"Example","text":"# Compile a program to MAST from a rustc-generated Wasm module\nmidenc compile foo.wasm -o foo.masl\n\n# Load that program into the debugger and start executing it\nmidenc debug foo.masl\nTo pass arguments to the program on the operand stack, or via the advice provider, you have two options, depending on the needs of the program:
midenc debug in the same order you wish them to appear on the stack. That is, the first argument you specify will be on top of the stack, and so on.--inputs option.To specify the contents of the operand stack, you can do so following the raw arguments separator --. Each operand must be a valid field element value, in either decimal or hexadecimal format. For example:
midenc debug foo.masl -- 1 2 0xdeadbeef\nIf you pass arguments via the command line in conjunction with --inputs, then the command line arguments will be used instead of the contents of the inputs.stack option (if set). This lets you specify a baseline set of inputs, and then try out different arguments using the command line.
While simply passing operands to the midenc debug command is useful, it only allows you to specify inputs to be passed via operand stack. To provide inputs via the advice provider, you will need to use the --inputs option. The configuration file expected by --inputs also lets you tweak the execution options for the VM, such as the maximum and expected cycle counts.
An example configuration file looks like so:
# This section is used for execution options\n[options]\nmax_cycles = 5000\nexpected_cycles = 4000\n\n# This section is the root table for all inputs\n[inputs]\n# Specify elements to place on the operand stack, leftmost element will be on top of the stack\nstack = [1, 2, 0xdeadbeef]\n\n# This section contains input options for the advice provider\n[inputs.advice]\n# Specify elements to place on the advice stack, leftmost element will be on top\nstack = [1, 2, 3, 4]\n\n# The `inputs.advice.map` section is a list of advice map entries that should be\n# placed in the advice map before the program is executed. Entries with duplicate\n# keys are handled on a last-write-wins basis.\n[[inputs.advice.map]]\n# The key for this entry in the advice map\ndigest = '0x3cff5b58a573dc9d25fd3c57130cc57e5b1b381dc58b5ae3594b390c59835e63'\n# The values to be stored under this key\nvalues = [1, 2, 3, 4]\n\n[[inputs.advice.map]]\ndigest = '0x20234ee941e53a15886e733cc8e041198c6e90d2a16ea18ce1030e8c3596dd38''\nvalues = [5, 6, 7, 8]\nOnce started, you will be dropped into the main debugger UI, stopped at the first cycle of the program. The UI is organized into pages and panes, with the main/home page being the one you get dropped into when the debugger starts. The home page contains the following panes:
On the home page, the following keyboard shortcuts are available:
Shortcut Mnemonic Descriptionq quit exit the debugger h next pane cycle focus to the next pane l prev pane cycle focus to the previous pane s step advance the VM one cycle n step next advance the VM to the next instruction c continue advance the VM to the next breakpoint, else to completion e exit frame advance the VM until we exit the current call frame, a breakpoint is triggered, or execution terminates d delete delete an item (where applicable, e.g. the breakpoints pane) : command prompt bring up the command prompt (see below for details) When various panes have focus, additional keyboard shortcuts are available, in any pane with a list of items, or multiple lines (e.g. source code), j and k (or the up and down arrows) will select the next item up and down, respectively. As more features are added, I will document their keyboard shortcuts below.
From the home page, typing : will bring up the command prompt in the footer pane.
You will know the prompt is active because the keyboard shortcuts normally shown there will no longer appear, and instead you will see the prompt, starting with :. It supports any of the following commands:
quit q quit exit the debugger debug show debug log display the internal debug log for the debugger itself reload reload program reloads the program from disk, and resets the UI (except breakpoints) breakpoint break, b create breakpoint see Breakpoints read r read memory inspect linear memory (see Reading Memory"},{"location":"usage/debugger/#breakpoints","title":"Breakpoints","text":"One of the most common things you will want to do with the debugger is set and manage breakpoints. Using the command prompt, you can create breakpoints by typing b (or break or breakpoint), followed by a space, and then the desired breakpoint expression to do any of the following:
The syntax for each of these can be found below, in the same order (shown using b as the command):
b FILE[:LINE] Break when an instruction with a source location in FILE (a glob pattern) and that occur on LINE (literal, if provided) are hit. b in NAME Break when the glob pattern NAME matches the fully-qualified procedure name containing the current instruction b for OPCODE Break when the an instruction with opcode OPCODE is exactly matched (including immediate values) b next Break on the next instruction b after N Break after N cycles b at CYCLE Break when the cycle count reaches CYCLE. If CYCLE has already occurred, this has no effect When a breakpoint is hit, it will be highlighted, and the breakpoint window will display the number of hit breakpoints in the lower right.
After a breakpoint is hit, it expires if it is one of the following types:
When a breakpoint expires, it is removed from the breakpoint list on the next cycle.
"},{"location":"usage/debugger/#reading-memory","title":"Reading memory","text":"Another useful diagnostic task is examining the contents of linear memory, to verify that expected data has been written. You can do this via the command prompt, using r (or read), followed by a space, and then the desired memory address and options:
The format for read expressions is :r ADDR [OPTIONS..], where ADDR is a memory address in decimal or hexadecimal format (the latter requires the 0x prefix). The read command supports the following for OPTIONS:
-mode MODE -m words (word ,w)bytes (byte, b) words Specify a memory addressing mode -format FORMAT -f decimal (d)hex (x)binary (bin, b)decimal Specify the format used to print integral values -count N -c 1 Specify the number of units to read -type TYPE -t See Types word Specify the type of value to readThis also has the effect of modifying the default -format and unit size for -count Any invalid combination of options, or invalid syntax, will display an error in the status bar.
"},{"location":"usage/debugger/#types","title":"Types","text":"Type DescriptioniN A signed integer of N bits uN An unsigned integer of N bits felt A field element word A Miden word, i.e. an array of four field elements ptr or pointer A 32-bit memory address (implies -format hex)"},{"location":"usage/debugger/#roadmap","title":"Roadmap","text":"The following are some features planned for the near future:
read commandmidenc","text":"The midenc executable is the command-line interface for the compiler driver, as well as other helpful tools, such as the interactive debugger.
While it is a lower-level tool compared to cargo-miden, just like the difference between rustc and cargo, it provides a lot of functionality for emitting diagnostic information, controlling the output of the compiler, and configuring the compilation pipeline. Most users will want to use cargo-miden, but understanding midenc is helpful for those times where you need to get your hands dirty.
First, you\u2019ll need to have Rust installed, with the nightly toolchain (currently we\u2019re building against the nightly-2024-05-07 toolchain, but we regularly update this).
Then, simply install midenc using Cargo in one of the following ways:
# From crates.io:\ncargo +nightly install midenc\n\n# If you have cloned the git repo, and are in the project root:\ncargo make install\n\n# If you have Rust installed, but have not cloned the git repo:\ncargo install --git https://github.com/0xpolygonmiden/compiler midenc\nAdvice
This installation method relies on Cargo-managed binaries being in your shell PATH, which is almost always the case, but if you have disabled this functionality, you\u2019ll need to add midenc to your PATH manually.
Once installed, you should be able to invoke the compiler, you should see output similar to this:
midenc help compile\nUsage: midenc compile [OPTIONS] [-- <INPUTS>...]\n\nArguments:\n [INPUTS]...\n Path(s) to the source file(s) to compile.\n\n You may also use `-` as a file name to read a file from stdin.\n\nOptions:\n --output-dir <DIR>\n Write all compiler artifacts to DIR\n\n -W <LEVEL>\n Modify how warnings are treated by the compiler\n\n [default: auto]\n\n Possible values:\n - none: Disable all warnings\n - auto: Enable all warnings\n - error: Promotes warnings to errors\n\n -v, --verbose\n When set, produces more verbose output during compilation\n\n -h, --help\n Print help (see a summary with '-h')\nThe actual help output covers quite a bit more than shown here, this is just for illustrative purposes.
The midenc executable supports two primary functions at this time:
midenc compile to compile one of our supported input formats to Miden Assemblymidenc debug to run a Miden program attached to an interactive debuggermidenc run to run a Miden program non-interactively, equivalent to miden runSee the help output for midenc compile for detailed information on its options and their behavior. However, the following is an example of how one might use midenc compile in practice:
midenc compile --target rollup \\\n --entrypoint 'foo::main' \\\n -lextra \\\n -L ./masm \\\n --emit=hir=-,masp \\\n -o out.masp \\\n target/wasm32-wasip1/release/foo.wasm\nIn this scenario, we are in the root of a Rust crate, named foo, which we have compiled for the wasm32-wasip1 target, which placed the resulting WebAssembly module in the target/wasm32-wasip1/release directory. This crate exports a function named main, which we want to use as the entrypoint of the program.
Additionally, our Rust code links against some hand-written Miden Assembly code, namespaced under extra, which can be found in ./masm/extra. We are telling midenc to link the extra library, and to add the ./masm directory to the library search path.
Lastly, we\u2019re configuring the output:
--emit to request midenc to dump Miden IR (hir) to stdout (specified via the - shorthand), in addition to the Miden package artifact (masp).midenc to write the compiled output to out.masp in the current directory, rather than the default path that would have been used (target/miden/foo.masp).See Debugging Programs for details on using midenc debug to debug Miden programs.
We have put together two useful guides to walk through more detail on compiling Rust to WebAssembly:
midenc compile on the resulting Wasm module, see this guide.You may also be interested in our basic account project template, as a starting point for your own Rust project.
"}]} \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index 50cb4697..ccb0c40b 100644 Binary files a/sitemap.xml.gz and b/sitemap.xml.gz differ diff --git a/usage/cargo-miden/index.html b/usage/cargo-miden/index.html index ecdf2e52..a1c178c9 100644 --- a/usage/cargo-miden/index.html +++ b/usage/cargo-miden/index.html @@ -854,7 +854,7 @@Now that you’ve created your project, compiling it to Miden Assembly is as easy as running the following command from the root of the project directory:
-cargo miden build
+cargo miden build --release
This will emit the compiled artifacts to target/miden.