Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -623,8 +623,8 @@ profiles/lines.vectors.json the vectors: {"feature": ..., "cases": [...]}
```

The vectors file and the profile are in the conformance corpus's case
format, a public spec (version 0.1, early days), documented in
ComlineProject/runtime's `runtime/conformance/README.md`. Every case in the
format, a public spec (early days, versioned in its `VERSION`), documented in
ComlineProject/specification's README. Every case in the
file is run, and each profile fact's `proven_by` names cases in it.

The implementation is a process speaking the conformance **ops protocol**,
Expand Down Expand Up @@ -658,17 +658,17 @@ unproven fact makes the exit code `1`.
- `--impl <command line>`: the implementation.
- `--encoding <name>`: only this encoding (repeatable). Default: every
encoding the package declares.
- `--corpus <dir>`: run a corpus laid out like ComlineProject/runtime's
`runtime/conformance` (`profiles/<name>.json`, `vectors/*.json`) instead
of the package's declarations: the official encodings (`packed@1`,
`datagram@2`, ...), whose vectors live there. Each encoding runs every
- `--corpus <dir>`: run a copy of ComlineProject/specification
(`profiles/<name>.json`, `vectors/*.json`) instead of the package's
declarations: the official encodings (`packed@1`, `datagram@2`, ...),
whose vectors live there. Each encoding runs every
case of the features its profile's facts name.
- `--timeout <secs>`: how long the implementation may take for all the
cases together.

```bash
comline conformance --impl ./target/release/acme-lines-conformance
comline conformance --impl comline-reference --corpus ../runtime/runtime/conformance --encoding packed@1
comline conformance --impl comline-reference --corpus ../specification --encoding packed@1
```

### `comline clean`
Expand Down
7 changes: 3 additions & 4 deletions src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -460,14 +460,13 @@ proven and which cases failed, with byte diffs; the exit code is 1 on any \
failure. `--json` prints the same report as JSON on stdout instead \
(`format`, `implementation`, `encodings`, `totals`, `verdict`).

`--corpus` runs a corpus laid out like ComlineProject/runtime's \
`runtime/conformance` (`profiles/`, `vectors/`) instead, for the official \
encodings.
`--corpus` runs a copy of ComlineProject/specification (`profiles/`, \
`vectors/`) instead, for the official encodings.

Examples:
comline conformance --impl ./target/release/acme-lines-conformance
comline conformance --impl \"node conformance.js\" --encoding com.acme.lines@1
comline conformance --impl comline-reference --corpus runtime/conformance --encoding packed@1
comline conformance --impl comline-reference --corpus specification --encoding packed@1
comline conformance --impl ./target/release/acme-lines-conformance --json")]
Conformance {
/// The implementation's command line: a program and its arguments, split on whitespace
Expand Down
12 changes: 6 additions & 6 deletions src/commands/conformance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@
//! beside the profile, at `profiles/lines.vectors.json`: the profile's path
//! with `.json` replaced by `.vectors.json`. The file is a corpus vectors
//! file (`{"feature": ..., "cases": [...]}`), in the conformance spec's case
//! format (ComlineProject/runtime `runtime/conformance/README.md`). Every
//! format (ComlineProject/specification's README). Every
//! case in it is run, and the profile's `proven_by` names cases in it.
//!
//! **The official encodings** (`packed@1`, ...) aren't declared by a package
//! and their vectors live in the runtime's corpus: `--corpus <dir>` runs a
//! corpus laid out like `runtime/conformance` (`profiles/<name>.json`,
//! and their vectors live in the specification: `--corpus <dir>` runs a
//! copy of ComlineProject/specification (`profiles/<name>.json`,
//! `vectors/*.json`) instead of the package's declarations.
//!
//! **The implementation** is a process speaking the ops protocol, as
Expand Down Expand Up @@ -417,16 +417,16 @@ fn from_package(work_dir: &Path, names: &[String]) -> Result<Vec<Subject>> {
if declared.is_empty() {
return Err(miette!(
"the package declares no encodings (`encodings` in config.idp); to run an official \
encoding's vectors, pass `--corpus <runtime/conformance>`"
encoding's vectors, pass `--corpus <specification>`"
));
}
declared
} else {
for n in names {
if !declared.contains(n) {
let hint = if comline_core::encodings::is_bare(n) {
"; an official encoding's vectors are in the runtime's corpus: pass \
`--corpus <runtime/conformance>`"
"; an official encoding's vectors are in the specification: pass \
`--corpus <specification>`"
} else {
""
};
Expand Down
Loading