Covering Specs and Code

Covering Specs and Code#

During input generation, Fandango attempts to cover as many behaviors as possible. There are three ways to do so:

  • Grammar coverage - that is, covering alternatives in the grammar;

  • Code coverage - that is, covering alternatives in the program under test; and

  • Constraint coverage - that is, covering alternatives in constraints.

Fandango is set to satisfy all these goals in unison.

Grammar Coverage#

During input generation, Fandango favors individual inputs that have a high grammar coverage - that is, they cover as many alternatives as possible. It does so after construction, as part of evaluating the fitness of individual inputs.

During protocol testing, Fandango already produces such interactions by construction – that is, with every new iteration, it tries to cover parts of the grammar (= states, messages, and transitions) that have not been seen yet.

Grammar coverage is determined and achieved using the \(k\)-path metric by Havrikov and Zeller [HZ20].

Added in version 1.2: For version 1.2, during input generation, Fandango will also achieve grammar coverage by construction.

Code Coverage#

As of version 1.2, Fandango will provide experimental support for guidance by code coverage. Details will be added here.

Added in version 1.2: This feature is planned for Fandango 1.2.

Constraint Coverage#

Future versions of Fandango will aim to achieve diversity by fulfilling different alternatives in constraints. Once this is complete, details will be added here.

Added in version 1.x: This feature is planned for a future Fandango version.

Contents