Mid-flight inspection

A run is going and you want to know what it is doing. These verbs say where every plate is, what is in it, what each labware thread is up to, who holds which device, and what is waiting on a person.

All read-only. None of them change runtime state. To act on what you find, go to Operator interventions.

Ids accept a prefix of four characters or more when it matches exactly one entity. Two matches exit 21, zero exit 20, and a prefix shorter than four characters exits 2 without looking anything up. Anywhere an execution id is taken you can also pass the literal last, meaning the most recent submission from this CLI session.

Where is every plate?

orca labware list                          # every instance the runtime knows about
orca labware where <id-or-prefix>          # current location of one plate
orca labware history <id> [--limit 20]     # location history for one plate
orca labware journey <id> [--kind move]    # moves + actions merged (full UUID required)

labware list

 id         template        barcode   location
 7a3f1c0e   cor96_falcon    PLT-001   shaker_01
 b29e0411   cor96_falcon    PLT-002   stacker_01 (expected)
 9d8c33ee   tip_300_l                 ml_star.deck.A1

The location column says how settled the placement is. (expected) means the plate is headed there and has not arrived. (removed) means it has been retired off the platform. A bare location means it is there now. Labware covers how an instance comes into being in the first place.

labware where

Takes an id, a full barcode, or an id prefix. Resolution order is exact id, exact barcode, then prefix. On a miss against the in-memory catalog it retries the same backend's store-first lookup, so a plate that survived a runtime rebuild is still reachable.

$ orca labware where 7a3f1c
labware 7a3f1c0e
  template: cor96_falcon
  barcode: PLT-001
  location: shaker_01

labware history

Every location this plate has occupied, in order. --limit keeps the last N rows (default 20).

 seq   location
 1     stacker_01
 2     shaker_01
 3     sealer_01

labware journey

history plus the actions that ran at each location, merged in timestamp order. --kind move or --kind action filters; repeat the flag for both. Works against both backends. Needs a full labware UUID, because the cloud backend has no list-instances endpoint to resolve a prefix against.

 kind     timestamp     where_or_op
 move     1780000012    shaker_01
 action   1780000045    shake

What is in the plate?

orca labware contents     <id>    # the answer, and which layer produced it
orca labware get-volumes  <id>    # per-well volumes (uL) from the ledger
orca labware get-tips     <id>    # tip positions the ledger has present

contents is the one to reach for. It prints a headline plus a layer table showing what each view says and whether the layers agree, so there is nothing to cross-check by hand. get-volumes and get-tips are the raw single layers, kept for diagnosis.

Every one of these three says how much the record actually knows. That is its provenance:

Provenance	Meaning
known	There is a value and nothing has gone unobserved since.
unknown	Nobody has ever stated it. This is not the same as empty.
stale	There is a value, but a restart, a reconnect or an error pause went by with nobody watching. Still the best value there is, and the only one worth asking an operator about.

A stale read is marked. contents appends worth a look to the headline; get-volumes and get-tips add [STALE] to the table title. Settle it with labware confirm-volumes / confirm-tips, or correct it with set-volume / set-tips. State explains where these values are kept and why a gap in the watching matters.

What is each execution doing?

orca execution list                        # every execution
orca execution detail  <exec-id>           # one execution, plus any error-paused threads
orca execution threads <exec-id>           # the thread instances inside it
orca execution thread detail <exec> <tid>  # one thread's full snapshot

execution list

 id         workflow    status      paused                        error
 7a3f1c0e   smc_assay   accepting   paused (operator), abort armed
 9b2e4108   smc_assay   draining

A pause does not change status: a paused execution still reads accepting or draining. The paused column is the latch. It says who set it, operator for a person's stop and system when the runtime paused itself (a stall, an orphaned backlog, a deadlock, a recoverable timeout), and whether a second confirmed stop would abort.

execution detail

A summary plus the thread counts. For every thread paused by an error it also prints the block an operator needs to decide what to do:

thread 4c1a9d20 (sample_journey) PAUSED (error)
  last_error: shake command failed: drive overcurrent
  pause_message: shake_step.shake raised on shaker_01
  action: shake: hold the plate at 500 rpm for 30 s
  pause_site: DEVICE_OP
  honoured_decisions: RETRY, RETRY_OP, CONTINUE, ABORT_ACTION, ABORT_METHOD, ABORT_THREAD
  paused_device_command: shake
  recover: orca execution thread recover 7a3f1c0e 4c1a9d20 <retry|retry-op|continue|abort-action|abort-method|abort-thread>

The pause site names where the thread stopped. honoured_decisions lists the only recovery decisions that site will accept; anything else is refused and the thread stays paused. paused_device_command names the device call the thread is suspended inside, which is what makes retry-op meaningful. See Recover.

--json returns the full structure, including per-thread method and action progress.

execution thread detail

The same fields for one thread, whether or not it is error-paused, plus current_location, current_method, current_action and the list of completed methods.

--json additionally carries waiting_for, the specific subject of a thread's wait: the device or transporter lock it is queued for, the target location of a move, the co-labware names it is missing, or the method it cannot resolve. A lock wait has no status of its own (an arm queued for a busy handler still reads MOVING), so waiting_for is the only place it shows.

Who is holding what?

orca reservation list                          # the latest execution in this CLI session
orca reservation list --execution <exec-id>    # one execution
orca reservation list --all                    # every tracked execution

Reservations are the engine's locking mechanism. Every device, location, transporter and method-scope shared resource is reserved before use and released after.

 id         thread_id   location      holding
 8f2a1c0e   4c1a9d20    shaker_01     assay_plate-1
 1c94b220   a1b2c3d4    stacker_01    sample_plate-3 (inbound)
 5e70b119   9d3c1b7e    pad_2         reagent_plate-1 (awaiting placement)
 33de7741               robotic_arm

The holding column says what the hold is for, because a held position reads the same however it is held and the ways mean different things to whoever wants it next. A bare name is labware standing there. (inbound) is labware on its way. (awaiting placement) is a person being asked to put that labware there. An empty cell is a hold that rests nothing at the position.

An empty thread_id is a reservation no thread owns (system-held or manual). --all prepends an execution column, drops holding, and is mutually exclusive with --execution. With neither flag the verb uses the last execution from this session and prints a one-line hint saying so.

Long waits are normal. A shaker running a two-hour incubation holds its reservation for the full two hours. A long-held reservation is not evidence of a deadlock. Cancelling one is on Operator interventions.

What are the devices doing?

orca device list                       # every device with its runtime snapshot
orca device info <name> [--capabilities]
orca device capabilities <name>        # driver introspection: interfaces, methods, params
orca device registry list              # topology card + connection card per device
orca device registry show <name>

device list:

 name         type            busy    initialized   mode       fault
 shaker_01    Shaker          False   True          LIVE
 ml_star      LiquidHandler   True    True          LIVE       aspirate (no_response)

The fault column names the command that did not come back clean and how it ended. The workflow will not drive a faulted device, so a row showing only busy and initialized would read healthy on a machine nothing may touch. Clear it with device clear-fault once you have looked at the instrument.

device registry list adds the two connection cards side by side:

• client is whether the on-prem client is reachable.
• device is whether the link is open on whichever driver is being driven.
• link names the mode of the driver that answered. A - means nobody could answer, which is a bridge-held device whose bridge has gone quiet; device then reads false for want of anywhere to put "unknown", not because a link is known closed.
• modes is the per-mode eligibility matrix (pure_sim, device_sim, live).

They differ on purpose: a released device sits under a client that is still heartbeating.

Liquid handlers

orca device mounted-tips <name>     # what the record says the head is carrying
orca device compare-deck <name>     # does the driver's deck agree with the ledger?

mounted-tips reads only. unknown means nothing has ever said, which is different from a head known to be carrying nothing. A channel marked channel number inferred was counted from zero because the pick named none; correct it with device set-mounted-tips.

compare-deck moves nothing and files nothing. Run it when something touched the robot outside orca, before reconcile-deck, because reconcile writes the ledger's answer over the driver's and there is nothing left to compare afterwards. A plate the ledger has in a gripper's jaws always disagrees, because the jaws are not a deck site. A driver reporting an empty deck is not a disagreement: its session was rebuilt and the deck is waiting to be re-declared.

What is waiting on a person?

orca manual-step list [--execution <id>] [--all]   # emitted-but-unconfirmed manual steps
orca state unsettled                               # everything nobody has stated

manual-step list shows every ctx.manual_step(...) pause that no one has confirmed yet, across all executions by default. Release one with manual-step confirm. Writing one into an action is on Context API.

 execution   step_id        instruction                    emitted_at
 7a3f1c0e    seal-check-1   Check the seal and confirm.    2026-09-08T14:22:03+00:00

state unsettled is the worklist for walking up to a paused system. Each line names the subject, says why it is unsettled, and names the verb that settles it:

$ orca state unsettled
'tip_rack_2' was known, then a stretch passed with nobody watching it
  settle with: confirm-tip-state
'assay_plate-1' was known, then a stretch passed with nobody watching it
  settle with: set-well-volumes
an action that has not finished did things to what 'sample_plate-3' holds, and the record only hears about them when the action ends; settle the action (retry, continue, or abort) rather than confirming or restating this

Subjects are labware and liquid-handler heads. It reports unknown (nobody ever said) separately from stale (it was known, then a stretch passed with nobody watching). An ordinary destination plate that nothing has happened to yet is left out, because listing every one of those is what makes the worklist unreadable.

The verb follows why the row is unsettled, not what the row is about. confirm-* agrees with what the record holds, so it is what a row gets when the record is still the best value there is. Three cases name set-* instead: nobody has ever said (there is nothing to agree with), an aborted action lost work it had really done (the record is short by an amount nothing can state, so agreeing would freeze a wrong number and mark it checked), and a record with nothing the confirm verb can back. A record that was merely contradicted at one position still gets confirm-*: the pick that proved it wrong is folded, so an operator who looks can find the record right and agree with it.

A row can name no verb at all. When an unfinished action is holding operations, confirming freezes a number that is behind and stating one is worse, because the action's own operations are folded on top of it when it ends. Only ending the action helps, so the line prints the detail and no settle with:.

See State.

Is the runtime up?

orca runtime status      # is a SystemRuntime built?
orca status              # backend health

runtime status works against both backends. It prints runtime built, or runtime NOT built plus the last build failure type, message and recovery hint. The local daemon has no submission pipeline, so it never reports a build error. The full verb is on Code deployment.

orca status against the local daemon prints the daemon snapshot: pid, port, uptime, whether a system is loaded, the loaded spec, the runtime state and the sim flag. Against a cloud backend it prints the runtime snapshot plus one list of blockers, everything stopping the run, worst first, each row carrying the command that clears it:

 kind                 severity   what is wrong                          what to do
 DEVICE_FAULT         error      ml_star faulted on aspirate            orca device clear-fault ml_star
 THREAD_ERROR_PAUSE   error      sample_journey paused on shake         orca execution thread recover ...

An empty list is the only thing that means nothing is in the way, and only when the backend actually reported the field. When it could not be read the row says so, which is not an all-clear.

What variables are in scope?

orca var list                                  # the latest execution in this session
orca var list --execution <exec-id>            # one execution
orca var list --all                            # every tracked execution
orca var get <name> --execution <exec-id>      # one value and the layer that answered
orca var explain <name> --execution <exec-id>  # what each submission of the execution resolves

A variable is a named value a workflow reads at runtime. var list prints name and value:

 name          value
 plate_count   4
 shake_speed   500

--all prepends an execution column and is mutually exclusive with --execution. With neither flag the verb uses the last execution from this session and says so.

var get requires --execution. It prints the value and the layer that produced it:

$ orca var get plate_count --execution 7a3f1c0e
var plate_count
  value: 4
  source: execution

Precedence, highest first: a named submission's own value (--submission <id>), then the execution scope, then the workflow or global layer underneath. A name starting with global. resolves down the global chain (global value, then computed, then the global definition's default); every other name resolves down the workflow chain (computed, then the workflow's VariableDefinition default). Sources are reported as submission, execution, global, computed, global-default and workflow-default. See Variables.

When a submission holds its own value for the name you read, var get prints a line saying which submissions shadow it and points at var explain. var explain gives the per-submission breakdown:

$ orca var explain shake_speed --execution 7a3f1c0e
var shake_speed (7a3f1c0e)
  value: 500
  source: execution
 submission   value
 a1b2c3d4     600

The merged per-execution listing has one slot per name, so two submissions holding different values show there as submission-diverged. explain is where you see both.

Pitfalls

• A variable defined nowhere exits 20 (NOT_FOUND). There is no string fallback.
• --all on var list spans executions, not scopes. It does not merge in globals.
• Reading a value is not the same as reading what a thread resolved. A submission's own layer outranks the execution scope, and only --submission or var explain shows it.

See also

• Operator interventions: act on anything you find here.
• Recover from errors: the decisions an error-paused thread will accept.
• State: where the labware record lives and what makes a read stale.
• Variables: the layers var get reports a source from.
• Forensics: the same history after the run has ended.