Generalized Modelica URIs

RationaleMCP/0037/ReadMe.md

@@ -0,0 +1,52 @@
+# Modelica Change Proposal MCP-0037<br/>Generalized Modelica URIs
+Henrik Tidefelt
+
+**(In Development)**
+
+## Summary
+This MCP defines the next generation handling of external resources in Modelica.  The current forms of a Modelica URI have a problem with the case insensitivity of the _host_ part of the URI, so they need to be replaced one way or another.  This MCP takes the opportunity to combine a solution to the case insensitivity problem with a few other improvements to the Modelica URIs as well as handling of external resources more generally, see [below](#Rationale).
+
+## Revisions
+| Date | Description |
+| --- | --- |
+| 2020-07-22 | Henrik Tidefelt. Filling this document with initial content. |
+
+## Contributor License Agreement
+All authors of this MCP or their organizations have signed the "Modelica Contributor License Agreement".
+
+## Rationale
+This MCP consists of three parts:
+- `resolveURI` a new operator with function syntax to replace the MSL function `loadResource`, see [separate document](resolve-uri.md).
+- New forms of Modelica URIs, see [separate document](modelica-uris.md).
+- New structure for storing external resources with a Modelica class on a file system, see [separate document](resource-directory.md).
+
+See [#2387](https://github.com/modelica/ModelicaSpecification/pull/2387) for an extensive early discussion about the goals for this MCP.  Since then, the [MCP for figure annotations](https://github.com/modelica/ModelicaSpecification/pull/2482) has matured, adding new use cases for referencing resource within and across classes.
+
+Having `resolveURI` in the Modelica Language Specification instead of `loadResource` in the Modelica Stanard Library is the natural place for the basic utility for dealing with a concept entirely defined in the Modelica Language Specification.  In addition, making it an operator with function syntax means we can use it to resolve Modelica URIs in ways that aren't possible with a normal function.
+
+Besides addressing the problem with case insensitivity of the _host_ part of a URI, the new forms are designed to address two major shortcomings of the current forms of Modelica URIs:
+- A class should be able to refer to its own resources without hard-coding its own fully qualified name.
+- References to special views of a class (_icon_, etc) shouldn't interfere with user-defined anchors and fragment specifiers.  Instead, Modelica URIs must have a flexible form allowing for new kinds of resources attached to a class, for example:
+  - Figures and plots ([MCP-0033](https://github.com/modelica/ModelicaSpecification/pull/2482)).
+  - Figure style sheets (topic for future MCP).
+  - Named experiments (topic for future MCP).
+  - A component of the instantiated class.
+
+The file system storage of external resources together with a Modelica class makes use of a new, special, directory name for external resources, removing amiguity in how to reference an external resource, as well as making it easier to determine which parts of a file system hierarchy that might contain external resources.
+
+## Backwards Compatibility
+Introducing the new operator with function syntax `resolveURI` means that Modelica code using this name for other purposes will break.  However, the name is unlikely to be in use in existing Modelica libraries.
+
+The new Modelica URIs are distinct from the existing Modelica URIs, making this part of the MCP fully backwards compatible.  However, the existing Modelica URIs will become a deprecated form according to this MCP.
+
+The new file system storage structure uses a directory name that couldn't be the name of a Modelica class.  Hence, the only kind of backwards incompatibility caused by this part of the MCP would be if a Modelica class is already using that name in the file hierarchy for something else.  Again, the name is unlikely to be used in existing Modelica libraries.
+
+## Tool Implementation
+
+### Experience with Prototype
+None yet.
+
+## Required Patents
+To the best of our knowledge, there are no patents that would conflict with the incorporation of this MCP.
+
+## References

RationaleMCP/0037/modelica-uris.md

@@ -0,0 +1,120 @@
+# Modelica URIs
+
+The Modelica URI format is extended to give meaning to the forms described below.  (Examples and more details were included in the old email thread, and might be of interest to also include here.)
+
+The design originates from the structure of a general URI.  Quoting [Wikipedia](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier):
+```
+URI = scheme:[//authority]path[?query][#fragment]
+```
+
+Users that are unsure about the details of the general URI syntax are recommended to try one of the many freely available online tools for URI parsing, such as: https://www.freeformatter.com/url-parser-query-string-splitter.html
+
+## Basic structure
+
+The _path_ part of a Modelica URI is denoted the Modelice URI's _class reference_, and unlike the old Modelica URI format, qualified Modelica class names use the normal URI path segment separator `/` instead of `.`.
+
+The new format takes advantage of the _authority_ being optional.  The meaning of a non-empty _authority_ is currently reserved for the deprecated form of a Modelica URI.  An empty _authority_ cannot be mistaken for the deprecated form, and is allowed in the new form as an alternative to not specifying the _authority_ at all.
+
+As usual, some characters need to be URL encoded when put in a URI.  For instance, the Modelica class `.Slashy.'Foo/Bar'.Baz` is referenced like so:
+- _modelica:/Slashy/'Foo%2FBar'/Baz_
+
+## Class reference
+
+This section presents a couple of alternative designs, meant to be discussed in the language group before deciding which one to proceed with.
+
+Before going into the alternative new designs, let us first mention the current form:
+- _modelica://host/…_ (non-empty host) — This is the form defined today, becoming deprecated as of this MCP.  In this MCP, this is referred to as the _deprecated form_ (of a Modelica URI).
+  * Example: _modelica://Modelica.Electrical.Analog/media/foo.png_
+
+### Modelica URI class tree context
+
+Among the proposed forms of class references below, there are some that are only meaningful relative to a context given by a position in the Modelica class tree.  These forms can only be used where a _class tree context_ is given.  It is an error if class reference relative to the class tree context doesn't resolve to a class within the same top level package as that of the context — references to resources in a different top level package must use the fully qualified form.  The class tree context can be given in one of two ways:
+- A string literal (or a substring thereof) appearing in a Modelica class definition, and in a position where a Modelica URI is given special meaning.  This generally excludes Modelica string literals denoting normal Modelica `String` values — a Modelica tool does not need to keep track of the origin of all string values in the form of Modelica URIs in order to preserve the class tree context.  Instead, it is in the context of certain annotations that a string literal can be in a position where a Modelica URI is expected, as in the `href` of an `a` tag in the `Documentation` annotation.
+- The [`resolveURI` operator](resolve-uri.md), also introduced by this MCP.
+
+
+### Base proposal: No authority, slash-separation, and Modelica lookup
+
+These are the different ways of referencing a class, where the _host_, _fullclass_, and _relclass_ represent a slash-separated Modelica class reference (that is, Modelica identifiers, separated by the forward slash (`/`) charater):
+- _modelica:/fullclass_ — Class given by its fully qualified name.  It is an error if _fullclass_ does not refer to an existing class.
+  * Example: _modelica:/Modelica/Electrical/Analog_
+  * Example: _modelica:///Modelica/Electrical/Analog_ (empty _authority_; the _fullclass_ form is the only possible form of class reference when _authority_ is present)
+- _modelica:relclass_ — Class given by lookup of _relclass_ in the class tree.  Requires class tree context, and it is an error if _relclass_ doesn't resolve to an existing class.
+  * Example: _modelica:Examples_
+  * Example: _modelica:_ (empty _relclass_)
+  * Example: _modelica:?figure=Disturbances&plot=Wind_ (empty _relclass_)
+- _modelica:./relclass_ — Same as _modelica:./relclass_, possibly useful to add clarity.
+  * Example: _modelica:./Examples_
+  * Example: _modelica:._ (empty _relclass_)
+  * Wrong: _modelica:///._ (malformed absolute reference with _fullclass_ `/.`)
+  * Wrong: _modelica:///./Examples_ (malformed absolute reference with _fullclass_ `/./Examples`)
+- _modelica:~/relclass_ — Similar to the form above, but lookup of _relclass_ is made from the point of the nearest enclosing encapsulated class, or the current top level class in case there is no enclosing encapsulated class.  For the MSL, where class encapsultion is currently not used much at all, this provides a convenient way to access resources organized in a hierarchy which is separate from the package hierarchy.
+  * Example: _modelica:~/Icons_
+  * Example: _modelica:~?resource=images/logo.png_ (empty _relclass_)
+  * Example: _modelica:~/Documentation?view=info#introduction_
+
+The forms containing _relclass_ are referred to as the _lookup-based forms_ (of a Modelica URI).  The form with _fullclass_ is referred to as the _fully qualified form_ (of a Modelica URI).
+
+### Lookup-free variation: Staying in the class tree
+
+While powerful and tightly coupled with the rest of the Modelica language, the lookup-based forms also come with some disadvantages:
+- Implementation and execution complexity: Performing lookup requires a significant amount of Modelica know-how to get it right, and performing the operation is not cheap.
+- There is currently no Modelica URIs relying on resources associated with the class in the instance tree; all current resources are associated with the class in the class tree, but locating a class in the class tree could be much simpler than performing a normal Modelica lookup.
+
+This leads to the alternative approach of making class references only through the class tree:
+- _modelica:relclass_ — Class given by appending _relclass_ to the class tree context, and it is an error if _relclass_ doesn't resolve to an existing class.
+- _modelica:~/relclass_ — Similar to the form above, but _relclass_ is appended to the nearest enclosing encapsulated class, or the current top level class in case there is no enclosing encapsulated class.  For the MSL, this would just as fine as the lookup-based form.
+- _modelica:{../}relclass_ (one or more leading '..' path segments) — Similar to the '.' form, but moving one level up the class tree with each '..' segment, and then appending _relclass_.  This is useful for locating sibling classes in a relative manner without depending on class encapsulation.
+  * Example: _modelica:../Resistor_
+
+### Class references with '.' separator
+
+While not following the structure of a general URI, it would also be possible to define the new Modelica URIs with the class reference using `.` as separator rahter than `/`.  The advantage of doing this is that the URI path can then also be used to specify a relative file path to an external resource of the class, just like in the old Modelica URI format:
+* _modelica:/Modelica.Electrical.Analog/media/foo.png_
+
+Variations of the class reference can still be made by prepending the same special leading segments to the path, for example:
+* _modelica:./Examples/media/foo.png_
+* _modelica:~//media/foo.png_ (empty _relclass_)
+
+One has to watch out for the pitfal of just leaving the _relclass_ empty when there are additional path segments:
+  * Wrong: _modelica://media/foo.png_ (deprecated form where _media_ would be the class reference)
+  * Wrong: _modelica:/media/foo.png_ (fully qualified form — not the intention)
+  * Correct: _modelica:.//media/foo.png_ (empty _relclass_)
+
+### Further generalizations
+
+As demonstrated by the forms above, these schemes can easily be extended with other ways of referring to a class by giving meaning to some new special content of the first path segments that cannot be mistaken for a (qualified) class name.  For instance, a reference relative to the current top level class could be indicated by a '~~' (two '~') in the first path segment.
+
+## Class-relative resource
+
+This section is divided into one subsection for each type of resource being referenced.  Here, _current class_ refers to the class referenced by the class part of the Modelica URI (not the class giving context to a relative class reference).
+
+### External resource
+
+An _external resource_ is a file stored in a structure reflecting the Modelica class hierarchy of the current class.  The resource is specified using the _resource_ query (the name is chosen to remind of the special directory name [_resources.d_](resource-directory.md)), for example:
+- _modelica:/Modelica/Electrical/Analog?resource=media/foo.png_
+
+For further details see, [resoruce-directory.md].
+
+### Class view
+
+Different views of a class can be specified by using the `view` query:
+- _modelica:classref?view=diagram_ — Class diagram.
+- _modelica:classref?view=icon_ — Class icon.
+  * Example: `<img src="modelica:?view=icon" alt="class icon"/>` (icon of the current class)
+- _modelica:classref?view=text_ — Textual class source code.
+- _modelica:classref?view=info_ — Class documentation.  The fragment specifier can be used to reference a link anchor (present in the `Documentation.info` of the current class).
+  * Example: _modelica:/Modelica/Electrical/Analog?view=info#overview_
+
+### Figure and plot
+
+In the event that ([MCP-0033](https://github.com/modelica/ModelicaSpecification/pull/2482)) is accepted before this MCP, a figure is referenced using its `identifier` in the `figure` query.
+  * Example: _modelica:/Modelica/Electrical/Analog/Examples/Rectifier?figure=voltcurr_
+
+To reference a plot inside a figure, one can also use the `plot` query (note that the `plot` identifier is only meaningful within a given figure).
+  * Example: _modelica:/Modelica/Electrical/Analog/Examples/Rectifier?figure=voltcurr&plot=sumc1c2_
+
+Note that inheritance of figures means that the `figure` identifier is not necessarily referring to a figure defined in the current class, but could come from one of its parent classes.
+
+While the example above was using the _fullclass_ form of a Modelica URI, the most common place where a figure is references is probably in the documentation or other figures of the current class.  Then, using a lookup-based form makes the class more self-contained.
+  * Example: _modelica:?figure=voltcurr&plot=sumc1c2_ (appearing in the `Documentation.info` of the current class)

RationaleMCP/0037/resolve-uri.md

@@ -0,0 +1,41 @@
+# The resolveURI operator
+
+An operator with function syntax called `resolveURI` is introduced.  Compared to `loadResource`, it is more powerful, will have more clearly defined semantics, and is properly defined as Modelica lanugage feature instead of bypassing the Modelica Language Specification by means of `ModelicaServices`.
+
+The result of `resolveURI(uri)` is a `String` giving the absolute filename of the external resource referenced by `uri`.  Unlike a normal function, it knows about its call site, so that it can apply normal lookup rules.
+
+`resolveURI` accepts two kinds of URIs:
+- _file:///…_ — only allowing absolute _file_ URIs
+- _modelica:…_ — any Modelica URI except the deprecated _host_ form
+
+The reason that relative file URIs are not supported by `resolveURI` is that the base URI would most naturally be the file in which the Modelica source code of the `resolveURI` expression is stored, but a Modelica tool is not required to store classes in a file system, so this also encourages the use of Modelica URIs to handle resources attached to a Modelica class.
+
+For a Modelica URI, `resolveURI` provides a context for lookup-based Modelica URIs, namely the class containing the `resolveURI` expression.  The following example illustrates how `resolveURI` gives context to a lookup-based URI:
+```
+package P
+  model M
+    model A
+    end A;
+    constant String uri;
+    String filename = resolveURI(uri);
+  end M;
+
+  model A
+  end A;
+
+  M m(uri = "modelica:A?resource=table.mat");
+end P;
+```
+Here, the lookup of `uri` doesn't take place where the string literal appears (namely `P`, which would have resulted in _…/P/A/resources.d/table.mat_).  Instead, the lookup happens in the context `P.M`, resulting in `m.filename` being _…/P/M/A/resources.d/table.mat_.
+
+The example above may seem counter-intuitive, but the example is contrived and the potential confusion is typically avoided by applying `resolveURI` directly to a string literal; then the context of the string literal coincides with the context given by the `resolveURI` expression.
+
+Note that a `resolveURI` expression is an expression resulting in an absolute filename, and shall not be used where a URI is expected (as in the `src` of an `img` tag in the `Documentation` annotation).
+
+## Evaluation semantics
+
+It would make sense to standardize on either of the following two variants:
+- It only operates on constant strings.  This way, application of lookup rules doesn't require a runtime representation of the class tree.
+- It only evaluates at runtime, allowing a built simulation to be transferred from one tool installation to another.
+
+A clear drawback of runtime evaluation is that it becomes impossible to make external resources part of a `constant` expression — the variability of a `resolveURI` expression would be no less than `parameter`.

RationaleMCP/0037/resource-directory.md

@@ -0,0 +1,17 @@
+# The _package-resources_ directory
+
+Consider the following reference to an external resource:
+- _modelica:/Modelica/Electrical/Analog?resource=media/foo.png_
+
+The URI path part is the class reference, and could take different forms, as described in [modelica-uris.md].  The referenced class is denoted _current class_, here `Modelica.Electrical.Analog`.
+
+Here, the _media/foo.png_ is a relative file system path that is resolved within a resource directory associated with the current class.  The details of this mapping for the deprecated _host_ form of a Modelica URI are omitted here; the following only applies to the non-deprecated forms, when the current Modelica package is stored in a file system hierarchy:
+- The fully qualified class name of the current class (after resolving any _relclass_ with respect to the class tree context) is mapped to a nested directory structure, with the constant directory name _resources.d_ (alternatively _package-resources_) appended.
+- The relative file system path is relative to _resources.d_ and may not contain the special path segments "." or "..".
+
+Note that the period (".") in the name _resources.d_ makes it distinguishable from a Modelica identifier.  In combination with the constraints on the relative file system path, this implies that that the referenced resourse resides inside the _resources.d_ directory of the class, and that any Modelica URI referencing this resource must do so via reference to the current class.
+
+In the example Modelica URI above, assume `Modelica` is stored in _/Users/jdoe/modelica-packages/Modelica-4.3.2/Modelica_.  Then, the resolved external resource is:
+- _/Users/jdoe/modelica-packages/Modelica-4.3.2/Modelica/Electrical/Analog/resources.d/media/foo.png_
+
+Note that the mapping of the fully qualified class name to a directory is the same regardless of whether the package itself uses a directory hierarchy for storage — the directory hierarchy for external resoruces is fixed, and it is only one of several options for the package itself to be stored in the same hierarchy.