[crusso]

Implement Martin's revised deserialization for Candid options

This PR fixes coercion of values into nested optional types.

Before the fix from this PR, 5 : nat64 would coerce to opt 5 : opt na64 (as expected) and null : opt opt nat64 (unexpected). >This PR fixes the unexpected result to be opt opt 5 : opt opt nat64 (as expected).

Before the fix from this PR, 5 : int would coerce to null : opt opt nat. This PR fixes the result to be opt null : opt opt nat. The >motivation for this change is to enable using optional types as follows:

• outer option characterizes if a value (of any type) is provided;
• inner option expresses to optimistically deserialize the actual value into the given type (without failing to deserialize and without losing the information if a value of any type is provided).

This PR also adds multiple test cases covering the change to coercion (including coercion to nested optional types over null and reserved).

c.f.

• Rust fix fix: coercion of values into nested optional types dfinity/candid#679
• Candid spec PR, in particular this commit
  dfinity/candid@90af305

Furthermore (from #5619) fixes #5543.

The spec says that every Candid value is a subtype of opt T, and gives opt t when t <: T. But the null type has no subtypes. I.e. we only get out a null : Null when the binary Candid has a 0x7F there. In all other cases the coercion must fail, either trapping or indicating a coercion failure to the caller (when recoverable). However null : ?T can always be materialised when an argument is missing, similarly null : Null. So we have to treat missing arguments and coercion failures differently when the requested type is Null (or null respectively, in Candid).

This PR corrects the reading of (e.g. \x70, <reserved>) in the non-recoverable case and makes it a coercion failure.

Also adds improvements to the candid-tests runner to

• not ignore the case when a field type can be inferred, but not in the record type (eliminating M0164, ignored test)
• adds null values to top-level argument tuples where the type requests them
• adds null fields to record values (in argument sequences) where the type requests them

---

TODO Tasks:

• bump candid dependency to pick up revised test suite
• It would be cool if the generated Motoko would be output on stderr when --diag is passed to candid-tests
• potential simplifications suggested in fix: catch up with Candid spec changes (reserved ↛ null) #5619

[github-actions]

Comparing from 7012347 to b1b6869:
In terms of gas, no changes are observed in 5 tests.
In terms of size, no changes are observed in 5 tests.

[mraszyk]

Would it be possible to make the tests more compact (and better readable) using generics (analogously to the Rust tests)?