Redefine the API in XML (or another high-level language)
There is a directory
https://gitlab.anu.edu.au/mu/mu-spec/tree/master/scripts that contains some scripts. In the past, every time I modified intrinsics (still called CommInsts at that moment), I called the
synchronise_everything.sh script to make things consistent.
synchronise_everything.sh: This executes the following scripts.
muapi-irbuilder-to-comminsts.py: This parses the
muapi.hheader file, finds the IR builder API part, and generates the corresponding intrinsics (CommInsts) in the
common-insts.rstdocument. The purpose is that all API functions should be available as intrinsics (CommInsts), with differences if appropriate.
comminsts-to-muapi.py: It parses the
common-insts.rstfile, grep the comminst definitions, and generate the corresponding definitions in the
You see, it is quite dirty. I have to parse the embedded code in the
.rst documents as well as the
.h header, and inject contents into each other. In fact, this is my least favourite part of the spec.
I have long been thinking that it should be done differently.
- The API should be defined in a more machine-readable format, such as XML, and
- the API should be defined in a higher-level type system, not C, but can be easily mapped to C, Java, or other high-level languages and generate bindings accordingly.
For example, here is an API function that creates a new thread on an unbound stack, and resumes the stack normally:
MuThreadRefValue (*new_thread_nor)(MuCtx *ctx, MuStackRefValue stack, MuRefValue threadlocal, MuValue *vals, MuArraySize nvals); /// MUAPIPARSER threadlocal:optional;vals:array:nvals
MUAPIPARSER magic is followed by several other magics that tells the parser that the
threadlocal parameter is optional, and
vals is an array, and its length is
nvals. What it really means, in Java, is:
MuThreadRefValue new_thread_nor( MuStackRefValue stack, Optional<MuRefValue> threadlocal, List<MuValue> vals);
Yes. I have been careful to limit what type the API can use, hoping to make it less susceptible to the complexity of the API (such as struct layout). As a result,
- The possible types of parameters and the return value only includes scalar types (including pointers) and lists (as C arrrays), but not structs, and
- If a parameter has pointer type, it may or may not be optional, but it is always documented, and
- If a parameter has list type, it is always required to supply its length in another parameter.
But we cannot express the idea of "optional" and "array with length" in plain C, and those are what the
MUAPIPARSER magic is for. Using Java is okay since its type system can express those ideas, but it is still difficult to parse. XML should be a better choice.
<class name="MuCtx"> <method name="new_thread_nor"> <param name="stack" type="MuStackRefValue" /> <param name="threadlocal" type="optional:MuRefValue" /> <param name="vals" type="list:vals" /> <return type="MuThreadRefValue" /> </method> </class>
It should be easy to convert such XML snippet into a C function declaration (perhaps not the other way around).
I am not sure if the documentation should be included as part of the XML, because I still think reStructuredText is more appropriate for documentation.
I don't know how intrinsics should be described. Maybe inline code snippets are still appropriate, because they should be in Mu IR, and should be precisely specified.
I think most Mu-related projects are not as active as a few years ago, so it should be safe to fix the implementations, too, without worrying about getting in the way of other contributors.