Call-back from native to Mu
Created by: wks
Overview
Rationale
Some existing C libraries or system interfaces use call-back functions, i.e. user-provided function pointers which are called by C or system libraries. Mu should provide appropriate mechanisms to interface with those libraries.
This is part of the (unsafe) native interface. See super issue: https://github.com/microvm/microvm-meta/issues/24
Exposing appropriate Mu functions as C-style function pointers
"Appropriate" Mu functions must only use the following types as their parameter types or return types: int<n>
, float
, double
, vector<T>
, ptr<T>
or struct
types whose components are these types. In the case of ptr<T>
, T
can also be array<T n>
or hybrid<F V>
where T
, F
and V
are one of the above types. In other words, (traced) references and Mu-specific opaque types are not allowed.
The Mu ABI will be designed to be compatible with the C calling convention as defined by the platform ABI.
way 1: (simple) Mu functions are declared with the optional WITH_FP
clauses to create their associated C-style function pointers. For example:
.funcdecl @some_func WITH_FP(@fp_some_func DEFAULT @COOKIE) <@sig>
.funcdef @other_func VERSION @other_func_v1 WITH_FP(@fp_other_func DEFAULT @COOKIE) <@sig2> WITH_FP @fp_other_func (%param0) {
...
}
With the above definitions, @some_func
has type func<@sig>
, which is a Mu function reference value. @fp_some_func
has type funcptr<@sig>
, which is a C-style function pointer. Similarly @other_func
is a func<@sig2>
, while @fp_other_func
is a funcptr<@sig2>
. DEFAULT
is the calling convention. @COOKIE
is a "cookie" (see way 2 below).
The Mu IR program or the API can pass the function pointer to the native program. When called, the Mu function will run and return its return value to the native caller.
- pros:
- simple
- The native funcptr is immediately available after loading the Mu bundle.
- cons: does not support "closures" well. Some languages/implementations (e.g. LuaJIT) would like to expose closures (rather than just functions) to C as callbacks.
way 2: (complex) Mu functions are exposed with a run-time invocation of a Mu instruction or a Mu API message.
Format:
- Instruction: fp =
EXPOSE_MU_FUNC
<
sig>
mufunc cookie - API: fpHandle = ca.exposeMuFunc( hMuFunc, hCookie )
The resulting fp has type funcptr<sig>
and can be called from C. A function can be exposed multiple times, and the resulting function pointers are mutually inequal. The cookie is an int<64>
value associated to the resulting function pointer. If a Mu function is called through a particular function pointer, a special instruction NATIVE_COOKIE
will return the associated cookie value.
Example:
%fp1 = EXPOSE_MU_FUNC <@sig> @some_func @some_int64_value
%fp2 = EXPOSE_MU_FUNC <@sig> @some_func @other_int64_value
...
UNEXPOSE_MU_FUNC %fp1
UNEXPOSE_MU_FUNC %fp2
// in @some_func
%cookie = NATIVE_COOKIE
%eq = EQ <@i64> %cookie @some_int64_value
...
val hFP = ca.exposeMuFunc(hFunc, hSomeInt64Value)
...
ca.unexposeMuFunc(hFP)
Both %fp1
and %fp2
have type funcptr<@sig>
. But if the Mu fucntion @some_func
is called from C via %fp1
, the NATIVE_COOKIE
instruction will return @some_int64_value
. If called via %fp2
, then NATIVE_COOKIE
returns @other_int64_value
, instead.
- pro: the cookie can be used to identify different closures and look up the contexts of the closures.
- con:
- Not as simple as way1.
- Exposing a Mu function requires a Mu instruction or an API message. This makes "implementing the Mu client API directly as exposed Mu functions" difficult. (In this case, exposing a Mu function requires an API function, which is also an exposed Mu function.)
Contexts necessary for Mu functions to run
Even if a Mu function is exposed to the native program as a functpr<sig>
, some contexts must be set up so that the Mu function can make use of Mu-specific features. These include:
- Thread-local garbage collection states: including thread-local allocation pools, and registering the thread for yielding as requested by the GC.
-
Stack context: Each Mu stack has an associated
stack
value (the opaque reference to the current stack). This is necessary for swap-stack.
Similar to the JNI's "attaching a native thread to the JVM", Mu will also require attaching Mu contexts to a native thread before any exposed Mu function pointers can be called.
If the native program is executed because some Mu program called the native function through the native interface (via CCALL
), the context is already set up and the C program can safely call back to Mu.
Mixed native/Mu stacks
With the possibility of both C-to-Mu and Mu-to-C calling, a stack may have mixed C or Mu frames. It has some implications for stack introspection and exception handling. Possible approaches are:
- Stack introspection cannot go deeper than the last contiguous Mu frame from the top. i.e. introspection is immediately unavailable when reached a native frame. Exceptions may not go into native frames. This approach has the weakest promise from Mu, and is thus the easiest.
- Mu can skip non-Mu frames and unwind to other Mu frames underneath.
- Stack introspection and stack unwinding caused by exceptions can go through frames which are supported by the native debugger. This is harder than the previous one, but still practicable.
- Support non-standard frames (such as JavaScript frames of SpiderMonkey or V8). Too hard.