111 create yaml template for problem submission #121
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # Please enter the relevant information. | ||
| # Fields that are not relevant can be left empty. | ||
| - name: | ||
| short: BBOB | ||
| full: Real-Parameter Black-Box Optimization Benchmarking | ||
| suite/generator/single: suite | ||
|
Collaborator
There was a problem hiding this comment. I still think it would be best to enter problems separately, but I understand the necessity of streamlining this process. So if we want to allow entering suites, maybe we have different templates? Because some values might only make sense for a suite? Otherwise, for a suite, a lot of the input would be "varies" for number of objectives, variables, constraints, dynamic, noise, etc?
Collaborator
Author
There was a problem hiding this comment. Yes, it would be nice to have problems separately, I have also prototyped how it might work to have both the suite and component problems, see here: Line 1107 in 84b34b6 I am not sure about separate templates for suites or problems, but we can think about this and discuss what makes sense. I would imagine, e.g., the BBOB sphere still has a bunch of "varies", because it is scalable in the number of variables for example. For a suite, I would want an exhaustive list (e.g., noise: yes, no / optional, or something like that) of the options, rather than a vague "varies". I will describe this in a comment in the template.
Collaborator
Author
There was a problem hiding this comment. TODO: create a new issue for this. |
||
| objectives: | ||
| number: '1' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| types: single | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| variables: | ||
| types: continuous | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| conditional: 'no' | ||
| dimensionality: scalable | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| constraints: | ||
|
Collaborator
There was a problem hiding this comment. Do we want an enumeration of constraints? Then we can have one or many soft constraints, one or many box/boundary constraints etc. It would also allow us to add linear or quadratic constraints in a natural way. constraints:
- type: box
soft: yes # Defaults to no
number: 2 # Number of variables that are box constraint
- type: linear
soft: no # Default but doesn't hurt
number: 10
- type: permutation
number: 1No entries in the
Contributor
There was a problem hiding this comment. Sounds like a good idea |
||
| present: 'no' | ||
|
CIGbalance marked this conversation as resolved.
Outdated
|
||
| soft: '0' | ||
| hard: '0' | ||
| boundary/box: 'yes' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| permutation: 'no' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| dynamic: | ||
| present: 'no' | ||
| types: '' | ||
| noise: 'no' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| modality: | ||
|
There was a problem hiding this comment. It's complex ... I agree that modality is important but there is way more to it than just "uni" or "multi" and just this doesn't really help IMO. |
||
| types: 'unimodal, multimodal' | ||
| evaluations: | ||
| multi-fidelity: 'no' | ||
| partial possible: 'no' | ||
| independent objectives: 'no' | ||
| reference: | ||
| links: | ||
| - https://doi.org/10.1080/10556788.2020.1808977 | ||
| authors: '' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| contact person: '' | ||
|
There was a problem hiding this comment. a possible value could be "any" in the sense of "any of the above" but leaving this field empty is confusing. (please don't contact us?)
Contributor
There was a problem hiding this comment. Contact person does not need to be an author, could also be maintainer,... or noone if something is abandoned, but we still want to have it in the database. But it should not be under reference to avoid confusion, rather be its own top-level field |
||
| implementations: | ||
|
There was a problem hiding this comment. Plural? Can people leave multiple implementations? Do we want this?
Contributor
There was a problem hiding this comment. Yes, it can be useful for example if suites are implemented in different software for different languages
Collaborator
There was a problem hiding this comment. Yes, there can be multiple independent implementations and I think that's fine. |
||
| - name: COCO | ||
| link: https://github.com/numbbo/coco | ||
| languges: 'C, Python' | ||
|
CIGbalance marked this conversation as resolved.
Outdated
|
||
| evaluation time: 'less than a second' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| specific requirements: 'no' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| source: | ||
| real-world: | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| degree: '' | ||
| open/closed: '' | ||
| artificial: 'yes' | ||
| other: 'no' | ||
| textual description: | ||
| general info: '' | ||
| motivation: 'evaluate algorithm performance for typical difficulties that occur in continuous problems' | ||
| challenage/key characteristics: '' | ||
|
kvdblom marked this conversation as resolved.
Outdated
|
||
| limitations: '' | ||
| other info: '' | ||
Uh oh!
There was an error while loading. Please reload this page.