Redefine the API in XML (or another high-level language)
As @U60333591 Javad suggested fixing the terminologies (#20, #21), I suggest fixing the scripts that manages the API headers and the intrinsics documentation, too.
There is a directory https://gitlab.anu.edu.au/mu/mu-spec/tree/master/scripts
that contains some scripts. In the past, every time I modified intrinsics (still called CommInsts at that moment), I called the synchronise_everything.sh
script to make things consistent.
-
synchronise_everything.sh
: This executes the following scripts.-
muapi-irbuilder-to-comminsts.py
: This parses themuapi.h
header file, finds the IR builder API part, and generates the corresponding intrinsics (CommInsts) in thecommon-insts.rst
document. The purpose is that all API functions should be available as intrinsics (CommInsts), with differences if appropriate. -
comminsts-to-muapi.py
: It parses thecommon-insts.rst
file, grep the comminst definitions, and generate the corresponding definitions in themuapi.h
header.
-
You see, it is quite dirty. I have to parse the embedded code in the .rst
documents as well as the .h
header, and inject contents into each other. In fact, this is my least favourite part of the spec.
I have long been thinking that it should be done differently.
- The API should be defined in a more machine-readable format, such as XML, and
- the API should be defined in a higher-level type system, not C, but can be easily mapped to C, Java, or other high-level languages and generate bindings accordingly.
For example, here is an API function that creates a new thread on an unbound stack, and resumes the stack normally:
MuThreadRefValue (*new_thread_nor)(MuCtx *ctx, MuStackRefValue stack,
MuRefValue threadlocal,
MuValue *vals, MuArraySize nvals); /// MUAPIPARSER threadlocal:optional;vals:array:nvals
The MUAPIPARSER
magic is followed by several other magics that tells the parser that the threadlocal
parameter is optional, and vals
is an array, and its length is nvals
. What it really means, in Java, is:
MuThreadRefValue new_thread_nor(
MuStackRefValue stack,
Optional<MuRefValue> threadlocal,
List<MuValue> vals);
Yes. I have been careful to limit what type the API can use, hoping to make it less susceptible to the complexity of the API (such as struct layout). As a result,
- The possible types of parameters and the return value only includes scalar types (including pointers) and lists (as C arrrays), but not structs, and
- If a parameter has pointer type, it may or may not be optional, but it is always documented, and
- If a parameter has list type, it is always required to supply its length in another parameter.
But we cannot express the idea of "optional" and "array with length" in plain C, and those are what the MUAPIPARSER
magic is for. Using Java is okay since its type system can express those ideas, but it is still difficult to parse. XML should be a better choice.
<class name="MuCtx">
<method name="new_thread_nor">
<param name="stack" type="MuStackRefValue" />
<param name="threadlocal" type="optional:MuRefValue" />
<param name="vals" type="list:vals" />
<return type="MuThreadRefValue" />
</method>
</class>
It should be easy to convert such XML snippet into a C function declaration (perhaps not the other way around).
I am not sure if the documentation should be included as part of the XML, because I still think reStructuredText is more appropriate for documentation.
I don't know how intrinsics should be described. Maybe inline code snippets are still appropriate, because they should be in Mu IR, and should be precisely specified.
I think most Mu-related projects are not as active as a few years ago, so it should be safe to fix the implementations, too, without worrying about getting in the way of other contributors.