units

[joergfunger]

In our last meeting, we discussed the usage of units for the outputs, because we need to have a consistent representation in the KG, but that discussion was not finalized. We essentially have two options. The first one is that our metric file already expects units in a predefined format, such as discussed e.g. in #78.

{
  "number_of_dofs[-]": 10,

  "max_mises_stress[Pa]": 1.0,

  "displacement_error_l2[m]": 1.0,

  "max_displacement_error[m]": 1.0,

  "displacement_top_right_corner[m]": [0.0, 0.0]
}

The second option, which is the approach we have followed for the input parameters, is that values are always provided with units. For the metrics file, this would look like

{
  "number_of_dofs": 10,
  "max_mises_stress": {
        "value": 1.0,
        "unit": "MPa"
  },
  "displacement_error_l2": {
        "value": 1.0,
        "unit": "mm"
  },
  "max_displacement_error": {
        "value": 1.0,
        "unit": "mm"
  },  
 "displacement_top_right_corner": {
        "value": [0.0, 0.0],
        "unit": "mm"
  }

Both options are fine with me, the second option forces the user to think about units and potential conversion issues, but we would probably convert that to some standard units anyway when creating the KG to simplify the query (or at least have both results jointly in the KG, the result the user provides with user-defined units, and the SI-converted results). @M-Jafarkhani and @doigl , what would be your suggestions? (I'm happy for comments from anyone else as well such as @jpthiele @berndflemisch @div-tyg, @srosenbu and @Sarbani-Roy.

[M-Jafarkhani]

I see two main options for specifying units:

1. Define them within the semantic representation of the benchmark file, using an ontology such as QUDT to ensure unambiguous references, and feed the benchmark as an input to the simulation.
2. Specify them in the parameters.json file; however, in this case, the units would be more compatible with Pint rather than being semantically explicit.

In either approach, a mapping between ontology-based unit definitions and Pint-compatible representations is required.

[doigl]

If I remember correctly, we wanted to specifcy the units within the semantic representation of the benchmark (for the input configuration parameters and also for the resulting metrics) to avoid to have to specify them explicitely in the parameter and metrics files. As well as QUDT units, SI reference point unit expressions are also a good option; they offer a fair amount of flexibility.

But if I understood #78 correctly, the user should be free to output their metrics in the unit of their choice. Who should then be responsible to convert?

[joergfunger]

I think that we have a semantic representation of the units within the knowledge graph is generally agreed on - so both for the parameters in the benchmark definition as well as for the output metrics in the result ROCrate. The current discussion is related to when and how we do the conversion to the simulation tools that are generally without units. We have to enforce an interface such that this conversion is done without errors when user implement their own tool. As such, there are the two options that are discussed above that can be used both for the input (the parameter files, where we currently specify unit and value as two separate entries) and for the user-ouput (the metric.json file), where each entry also has a unit (currently this is implicitly assumed). For the input/parameter file, the user might just use the value field and manually (when interpreting the unit) do the conversion - or use Pint, but that is not for sure, in particular if the software tools are non-python based. And for the output, someone has to convert that to the semantic units as @doigl has pointed out. Either the user is doing that and specifies value and units, though I guess that is not the optimal choice since they are not QUDT affine, for standard examples it might work, but not sure what happens for more complex cases. We could let the user specify pint units and then convert that internally, but I'm either not sure how general the current approach is. The third option is that the user can also do the conversion to a prescribed unit that we define e.g. in the key (length[m]), such that - when we do the conversion in the semantic representation - we only have to interpret the actual value. IMO, the last option is actually my favorite, because it forces the user (when using the key) to think about the unit they are importing from/exporting to, and if changes of the unit in a new version of the benchmark are done, the simulation software developer would realize that (because e.g. parameter['length[m]'] does no longer exist (if it is e.g. changed to [mm]. And in the same way, the output is just done in a standard unit, and then everybody is forced to convert their numerical result into that, and we can easily do the conversion to QUDT, because it is just prescribed and thus known when creating the benchmark.

[jpthiele]

I think it is a valid assumption that the majority of tools don't handle units in any way.
Therefore, we should specify input units and have matching output units as if no conversion occured in between.

I think it would be a valid approach to use the units in the keys as long as they metch the semantic representation,
which we should probably check in our tooling then.

Changing the units in the input parameters from our side at some point should result in a new major version of the benchmark description as that is a breaking change according to semantic versioning and it is safe to assume that a breaking change can be breaking and so a developer would have to look at the Changelog of the benchmark and see what was changes.
That being said I don't see what would lead us to change the units in between versions.

[joergfunger]

so we change the parameter files (and the metrics.jon) to the first approach as outline above (units in keys).