Operator interventions

This is where you step into a running lab: pause or stop an execution, operate on one labware thread, correct where the system thinks a plate is, send a command straight to a device, and change the numbers a move is made with.

These verbs change runtime state. Read Confirmation and --reason in the overview before running any of them in anger, and read Mid-flight inspection first to see what you are about to act on.

Two separate gates protect them, and they are not the same thing:

• A confirmation prompt. Most execution-level, thread-level, incident and variable verbs ask y/N (default N) before acting. Answering no exits 40 and changes nothing. --force (--yes, -y) skips the prompt, which is what a script needs: there is no stdin to answer it with. The labware override verbs do not prompt; their gate is the required --reason.
• A required --reason. The verbs whose engine action declares one take --reason STR as a required option, so --force cannot skip it: there is no way to make the change without saying why. The reason becomes the audit-trail entry.

The verbs that require --reason:

Verb
labware edit-location	labware reset-location
labware set-volume	labware set-tips
labware mark-tips-used	device release-hold
reservation cancel	incident recoverable-timeout abort
incident recoverable-timeout mark-complete	every execution thread surgery verb

--reason is accepted but optional on execution pause, execution resume, execution thread pause, execution thread resume, device take-control, device set-mounted-tips, device confirm-mounted-tips, labware confirm-tips and labware confirm-volumes.

Execution-level

orca execution pause   <id> [--reason "..."]   # pause every non-terminal thread
orca execution resume  <id> [--reason "..."]   # resume manually-paused threads
orca execution close   <id>                    # ACCEPTING -> DRAINING
orca execution stop    <id>                    # pause + arm abort (first call)
orca execution stop    <id> --confirm          # abort (second call)
orca execution remove  <id>                    # drop a terminal exec from the registry

pause

Pauses every non-terminal thread. Threads already paused by an error stay error-paused. A pause does not change the execution's status. It sets the pause latch, which execution list shows in its own column.

resume

Resumes manually-paused threads and reports what it did:

$ orca execution resume 7a3f1c0e
resume-all (7a3f1c0e)
  resumed: 3
  pause_cancelled: 1
  error_skipped: 1
  completed_skipped: 0

error_skipped counts threads it left alone. Error-paused threads need thread recover with a decision, not a resume.

close an execution

Moves the execution from ACCEPTING to DRAINING. JOIN_EXISTING submissions against it are rejected from then on; a STANDALONE submission for the same workflow boots a fresh execution. Live threads continue to completion, and the execution becomes terminal once the last one finishes. See Submissions.

$ orca execution close 7a3f1c0e
execution 7a3f1c0e -> phase=draining

stop an execution

Two calls, confirmed abort. The first call (no --confirm) pauses the execution immediately and arms the abort. Nothing is aborted yet and execution resume disarms it. A second call with --confirm runs the abort: threads are cancelled, reservations released, the execution ends at aborted. A stray --confirm on a not-yet-armed execution never aborts; it pauses and arms, the same as the first call.

$ orca execution stop 7a3f1c0e
execution 7a3f1c0e paused and abort armed; run `orca execution stop 7a3f1c0e --confirm` to abort, or `orca execution resume 7a3f1c0e` to disarm

$ orca execution stop 7a3f1c0e --confirm
execution 7a3f1c0e aborted (phase=aborted)

A stop against an execution that had already finished is a no-op and says so.

A confirmed stop ends a thread parked on an error as well as a running one. It resolves that pause with the same unwind an operator's abort-thread gives, so the action releases its device and records the operations it is dropping rather than abandoning both.

Pitfalls:

• stop --confirm is a hard abort. Plates in transit can end up somewhere the runtime does not expect. Afterwards, read labware where for every plate in the execution and reconcile with labware edit-location.
• An abort taken while a plate is genuinely in an arm's jaws leaves a real hold on the mover, and a mover holding one refuses every later pick. Clear it with device release-hold.
• To wind down without aborting, use close.

remove

Drops an execution from the runtime's registry and clears its events from the event bus. It is refused with 30 (CONFLICT) unless the execution is already completed, failed or aborted.

It does not erase history. The durable execution record survives, so execution list still shows the run and execution detail still answers for it, and the operations archive is append-only and untouched. See Audit and history.

Thread-level surgery

Thread surgery works on paused threads. If the thread is running, pause it or the execution first. How a method and an action are written is on Methods and Actions. What a thread is made of is on Threads.

orca execution thread pause          <exec> <tid> [--reason "..."]
orca execution thread resume         <exec> <tid> [--reason "..."]
orca execution thread spawn          <exec> <template-name> [--labware-id <id>]
orca execution thread skip-method    <exec> <tid> --method-name <name> --reason "..."
orca execution thread abort-method   <exec> <tid> --method-name <name> --reason "..."
orca execution thread insert-method  <exec> <tid> --where <head|tail|before|after> [--anchor <name>] (--template <name> | --from-file <path>) --reason "..."
orca execution thread replace-method <exec> <tid> <target-method> (--template <name> | --from-file <path>) --reason "..."
orca execution thread skip-action    <exec> <tid> (--action-command <cmd> | --action-id <id>) --reason "..."
orca execution thread insert-action  <exec> <tid> <action-file.py> --where <head|tail|before|after> [--anchor <name>] --reason "..."
orca execution thread replace-action <exec> <tid> <target-command> <action-file.py> --reason "..."

Methods are targeted by name, not by index. Method names are not unique within a lane, so skip-method and abort-method take the first match and you re-run for a later occurrence. Inserts require --where; before and after also require --anchor. Method inserts and replacements take exactly one of --template (an already-registered method template) or --from-file (a .py file holding a single @orca.method). Action inserts and replacements always read a .py file holding a single @orca.action, passed positionally, because action templates have no global registry.

skip-method

Skip a pending method. The thread resumes at the next method and the reservations that method would have taken are never acquired. For a method that has already started, use abort-method.

abort-method

Abort the in-progress method: abort the running action, skip the rest of the method, advance to the next one.

insert-method / insert-action

Splice a method or action into a paused thread's lane at the position --where names.

replace-method / replace-action

A pending target is spliced in place (skip plus insert), and the CLI says the replacement is queued and applies when the target is reached. If the target is the method or action the thread is error-paused on, the replacement is staged to run next and the CLI prints the next step: recover the thread with abort-method / abort-action (after making the cell physically safe) to drop the failed step. See Recover.

spawn

Start an extra thread from an existing template inside the execution. The template name is positional; --labware-id attaches an existing labware instance instead of creating one. The usual reason is an AUTO_SPAWN_FAILED incident where the engine could not find a matching template. Once you have identified the right one, spawning it releases the waiting contributor thread.

$ orca execution thread spawn 7a3f1c0e sample_journey
spawned thread 9d3c1b7e (sample_journey) status=running

Variables

orca var set    <name> <value> --execution <id> [--submission <sid>]
orca var set-global <name> <value>
orca var unset  <name> --execution <id> [--submission <sid>]

Name and value are positional; the execution is a required --execution flag (id or prefix). set-global takes no execution.

A variable read inside a running thread resolves through the layers described in Monitor and in Variables. Setting mid-flight takes effect on the next read. It does not retroactively change a value a thread has already read.

$ orca var set shake_speed 600 --execution 7a3f1c0e
shake_speed = 600  (execution 7a3f1c0e)

--submission is what changes an already-submitted thread. A submission's own layer outranks the execution scope, so setting at execution scope while a submission holds its own value changes nothing for that submission's threads. The CLI warns when that happens and names the submissions to look at. unset --submission <sid> clears one submission's override so its threads fall back to the execution value.

Labware

These verbs correct the runtime's record of labware. How a workflow declares the labware it needs is on Labware.

Where a plate is

orca labware edit-location  <id> <location> --reason "..."
orca labware reset-location <id> <location> --reason "..."
orca labware register       (<template> | --labware-type <type>) [--barcode <bc>] [--location <loc>]
orca labware edit-barcode   <id> <barcode>          # or --barcode <barcode>

Verb	When to use
edit-location	The runtime thinks a plate is at X and it is physically at Y. No transporter is invoked; you are asserting the plate is already there. A thread carrying this labware throws away the move it had planned and plans a fresh one from the new position, so you may put the plate anywhere reachable rather than back where the run left it. Refused only if another plate is already there, or a thread has reserved the spot.
reset-location	Clear the location history and set a new starting location, with no audit link to any prior move. Only safe before any thread has moved this labware; at runtime it corrupts the owning thread's location assumptions.
register	Record labware you have just put down. See below.
edit-barcode	The recorded barcode is wrong (mis-read, manual relabel). Positional or --barcode; conflicting values are rejected. In-flight actions that already captured the old barcode do not see the change.

register names the labware one of two ways, and exactly one: a template argument for labware the deployment package declares, or --labware-type for a catalog definition it does not, which derives an ad-hoc labware template on the spot. The second is what lets an operator introduce labware the workflow author never anticipated, instead of editing code. Passing both, or neither, is a usage error. Browse the catalog first with orca labware catalog.

--location may be any deck site, device site, storage position or mover gripper. It does not physically place anything.

register does not always mint a new instance. When a thread is parked waiting for labware of this template at this location, the operator has just placed that labware, so the expectation is adopted and its existing instance is what arrives. Only a slot nothing is waiting on mints a new instance.

Every refusal lands before anything is written, so state never lands half-registered:

Refusal	Exit	Cause
slot awaited for a different template	30	A thread is waiting at that slot for something else. You learn at your own call rather than through a thread failing behind you.
location_reserved	30	Another thread has claimed the slot. The refusal carries what the claim is for and whether that labware exists yet: on its way, or a person being asked to place it.
slot_occupied	30	Something else is already there. The refusal names the occupant.
LabwareHasNoModel	2	cheshire-drivers exposes no model for that labware type, so no instance can be made. The refusal names the type. This is not an edge case: most seeded catalog rows cannot be built, so try a --labware-type register before you need it. The check runs before the derived template is constructed, so a refused register leaves no template behind.

Tips and volumes

orca labware set-volume      <id> --well A1=100 --well B1=50 --reason "..."
orca labware confirm-volumes <id> [--reason "..."]
orca labware set-tips        <id> --position A1 --position A2 --reason "..."
orca labware mark-tips-used  <id> -p A1 -p A2 --reason "..."
orca labware confirm-tips    <id> [--reason "..."]

orca device set-mounted-tips     <name> --tip 0=rack_1:A1 --tip 1=rack_1:A2 [--reason "..."]
orca device confirm-mounted-tips <name> [--reason "..."]

set-* states a new truth. confirm-* agrees with what the record already says. Both clear the stale mark; use confirm-* when the physical state matches and set-* when it does not.

A confirm-* verb is refused when there is nothing worth agreeing with. All three confirms refuse in three cases, and set-* is the verb in each:

Case	Why confirm is wrong
Nothing is tracked	There is no value to agree with. A read of unknown is not a read of empty.
An in-flight action is holding operations	The read is behind. Confirming would freeze a number that is about to change when the action ends.
An aborted action lost work it had really done	The record is short by an amount nobody can look up, and the labware looks exactly as the record describes it to anyone glancing at the shelf. Confirming would freeze a wrong number and mark it checked.

A record that was merely contradicted is not refused. Something was proved wrong at one position at one moment, and the pick that proved it is already folded in, so an operator who looks can find the record right and agree with it. It confirms like any other stale row.

orca state unsettled names the right verb per row, so read it rather than guessing. State explains why one row gets confirm-* and another set-*.

• set-volume asserts absolute per-well volumes in uL. Repeat --well WELL=VOLUME. It overrides tracked volume, seeds the driver tracker if the labware is on-deck, and rejects an overfill beyond a well's capacity.
• set-tips asserts the rack's absolute layout: tips sit at exactly the positions named, and nowhere else. Passing no --position asserts an empty rack.
• mark-tips-used is the narrow version: it marks the named positions empty and moves the rack's next pick past them, leaving every other position's tracked state alone. Reach for it when a pick found air or a hand took a column.
• set-mounted-tips is absolute per head: any channel you do not name is recorded as carrying nothing. This is what the head is seeded from after a restart, so a wrong answer here makes the next pick wrong.

How a plate is carried

orca labware carry          <id> [--z-offset 4] [--jaw-opening 82] [--clear z-offset]
orca labware carry-normally <id>

carry is the narrowest move-parameter layer there is: it is about this one physical object, not a type, a place or an arm. It exists for what the other layers cannot say, such as a plate that came out of the sealer with a lid on it and must be carried higher until the lid is off. It wins over the labware type's grip profile, the position's numbers and the arm's defaults. Name at least one field to set, or one to --clear. carry-normally drops the whole override.

Anything true of the labware type belongs in grip-profiles set, where the next plate of that type inherits it.

Taking labware off the platform

orca labware clear-submission <submission-id> [--force]
orca labware discharge        <labware-id>    [--force]
orca labware clear-all        [--force] [--yes]

These clear the runtime's record. They dispose of nothing physically; that is the operator's job.

All three, and edit-location with them, reach every deck world the labware could be on, not just the one an operator write resolves to. A plate put down by a PURE_SIM run sits on the sim driver's deck, and a clear aimed only at LIVE used to leave it there, so the next run was refused that slot with no operator surface able to free it. Which world a run mode resolves to is on Sim hierarchy. The stores are written first and the driver half runs afterwards and cannot raise, so a discharge still works with the handler offline, which is often exactly why you are reaching for it. A deck that could not be reached keeps its record of holding the labware, so a later sweep tries again rather than believing a clear that never happened, and names the deck in the log.

• clear-submission is the ordinary end-of-run tidy-up. The run's plates are closed out, and anything a thread declared LEAVE_IN_PLACE stays, because a resident's identity is what carries its volume and remaining tips into the next run. The CLI reports how many were cleared and names the ones preserved.
• discharge removes one labware. Any thread still tracking it fails on its next reference. --force bypasses the active-thread refusal for every thread; it is not needed to release a manual-remove park, which is already exempt.
• clear-all is the panic button: every labware in the runtime, deck residents included. On an interactive TTY it prompts. On a non-TTY it refuses without --yes and exits 2, so a typo in a script cannot wipe a live deck.

--force on all three bypasses the active-execution refusal. None of them take --reason.

Devices

Device types and where a driver comes from are on Devices. Read the current state of one with device list and device info before you change it.

Lifecycle

orca device connect    <name> [--mode LIVE|DEVICE_SIM|PURE_SIM]
orca device initialize <name> [--mode ...]
orca device disconnect <name> [--mode ...]

--mode says which world the verb means. It defaults to LIVE out of a run, and the topology's sim_override ratchet applies on top.

• connect opens the link and nothing else. It moves nothing and readies nothing.
• initialize brings the device up and resets driver state. It can drop the labware the driver was tracking, reset calibration, and interrupt an action holding the device lock. It asks for no motion: homing is its own command. A clean run clears any fault standing on the device.
• disconnect ends the session. Anything mid-action on that device fails, and hardware that holds motor power while connected (a PF400 arm) drops it here.

connect and initialize also re-seed the driver's deck from the world model.

Faults and holds

orca device take-control    <name> [--reason "..."]
orca device release-control <name>
orca device clear-fault     <name>
orca device release-hold    <mover> [--to <location>] [--force] --reason "..."

take-control is a standing claim: the workflow will not touch the device, and a running thread that tries to use it or to move labware into or out of it fails until you release it. Without it, a workflow can start a move into the device between two of your own commands. release-control gives it back, so be clear of the device first.

clear-fault says a device has been looked at. A command that fails, times out, or is cancelled after it reached the device faults it, and the workflow stops driving it until the fault is cleared. Look at the machine and put it right first: this clears the record, not the trouble. It is usually not needed. Recovering a thread paused on that fault with retry, retry-op or continue clears it on the way, and so does a clean initialize or home. Reach for it when the machine is fit and neither of those applies.

release-hold frees a mover the record says is holding a plate. Give --to the position you actually put the plate: the jaws are freed and the thread carrying it plans a fresh move from there. Leave --to off when the jaws are empty and the record is wrong, and the labware is discharged instead. --force allows the discharge while a live thread still carries the labware, and is ignored when --to is given, because a carrying thread is told to plan again rather than stranded. No mover is driven either way; push the corrected world out afterwards with reconcile-deck.

Deck and world sync

orca device compare-deck   <name> [--mode ...]   # read-only, see Monitor
orca device reconcile-deck <name> [--mode ...]

reconcile-deck re-dispatches the deck layout and re-declares every labware the ledger places on that liquid handler's deck. It is a state push, and it moves nothing. Use it after anything that rebuilt the driver's session outside orca (a run cancelled at the touchscreen, a driver-internal recovery) so a retry can find its labware again.

It runs the comparison first and hands back what it is about to overwrite, because the push destroys the evidence. Every difference is filed as an incident, so it reaches the operator queue whether or not anyone read the answer. See State.

Sending a command by hand

orca device send   <name> <command>    [key=value ...] [--params <json>] [--mode ...] [--confirm]
orca device invoke <name> <capability> [key=value ...] [--params <json>] [--mode ...] [--confirm]

Both take arguments either as positional key=value pairs or as one --params JSON object. --params wins if both are given. Values in key=value form are typed best-effort: true/false to bool, then int, then float, else string.

send reaches any command the device advertises, including the vendor commands a driver forwards to its hardware from a vendor surface. invoke takes the name as it appears in orca device capabilities <name>. An orca-side namespace (shaker.shake) is not part of the method name and is dropped. A vendor command's prefix names the object it lives on (gripper.ungrip) and is sent whole.

orca device send   shaker_01 set_temperature celsius=37
orca device invoke shaker_01 shake --params '{"duration": 30, "speed": 500}'
orca device send   ml_star send_command --confirm line="..."

--confirm is for one thing. The driver flags exactly one kind of command requires_confirm: the raw vendor console (send_command and the aliases a backend gives it). Its payload is a string nothing between the CLI and the instrument can read, so nothing can say what it will do. Every other vendor extra dispatches without a confirm, because it arrives with a signature and a docstring the operator surface can show first, and a prompt on everything is a prompt nobody reads. Free to dispatch is not the same as safe.

Pitfalls:

• These bypass workflow coordination. They take a short-term device reservation, run, and release. If a workflow thread is mid-method on that device, they queue behind it.
• They refuse a device under external control. A standing take-control hold blocks send and invoke on that device; release the hold before sending.
• Eight world-model writes are refused outright, whatever the driver advertises: add_deck_labware, remove_deck_labware, reset_deck_labware, reconcile_deck_occupancy, seed_position, ensure_seeded, unseed_position and reset_world. Each edits a driver's own picture of where labware is and nothing else, so sent by hand they move the driver while the ledger goes on calling the site empty, and the labware does not survive a rebuild. The refusal names the operator verb to use instead: register, discharge or clear-all, each of which writes the record and the driver together.
• An ad-hoc command does write the ledger. A liquid-handling call lands an operation record in the same fold get-volumes and get-tips read; an arm's own pick and place move the plate into and out of the gripper position. It records the operation, not the driver's answer, and a recording that cannot be made is logged rather than raised, because the hardware has already moved. That trail is readable with ops-history.

Manual steps

When action code calls ctx.manual_step(...), the thread parks until an operator confirms.

orca manual-step confirm <step-id> --execution <id>

The step id is positional and the execution is a required flag. Confirm is safe: it fires immediately with no prompt and no reason. List the outstanding ones with manual-step list.

Reservations

orca reservation cancel <reservation-id> --execution <id> --reason "..."

Forces release of a held reservation. The thread that held it gets a typed error and pauses with an incident. Use it only for a genuinely stuck reservation (a hung driver, a manual intervention that needs the device), and read reservation list first to get the id. Long waits are normal and are not a reason to cancel.

Calibration and move parameters

A move's numbers resolve through six layers, each strictly narrower than the one before:

Layer	Set with	About
seed	built-in	The starting point when nobody has said anything.
arm defaults	orca move-defaults set <transporter>	One arm, deployment-wide.
labware	orca grip-profiles set <labware-type>	One labware type, wherever it goes.
site	orca teachpoints update and the access config it references	One position.
this labware here	orca teachpoints labware set <device> <position> <labware-type>	One labware type at one position.
this one plate	orca labware carry <id>	One physical object, for as long as the override holds.

The site layer is a teachpoint, a position taught on one transporter. Each layer contributes a sparse patch and the merge is field-wise, so a layer sets what it knows and inherits the rest. That is what lets one number be corrected without restating the other nine. orca move-defaults show <transporter> prints every field with the layer that won it. See Move parameters for the full model, and Transporters for the arms themselves.

orca grip-profiles list
orca grip-profiles show  <labware-type>
orca grip-profiles set   <labware-type> [--resource-width 82] [--z-offset 3] [--clear jaw_opening]
orca grip-profiles reset <labware-type>

orca move-defaults list
orca move-defaults show  <transporter>
orca move-defaults set   <transporter> [--jaw-opening 90] [--speed 60] [--clear z-offset]
orca move-defaults reset <transporter>

The approach fields (access type, clearance, z above, grasp offset) are deliberately absent from both. A teachpoint supplies them for every move and resolves after those layers, so change them on the access config the teachpoint references, or on the per-labware override at that one position.

--clear hands a field back to the layer underneath rather than setting it to zero. Repeat it per field. Both set verbs refuse a call that names nothing to set and nothing to clear.

Teachpoints, deck layouts, access configs

Reads for all three are in Inspect. The writes:

orca teachpoints create <device> <position> --coord-type <cartesian|joint> (--coords <json> | --coords-file <path>) [--access-config <name>] [--gateway <name>] [--orientation <left|right>] [--taught-with <labware-type>]
orca teachpoints update <device> <position> (--coords <json> | --coords-file <path>) [--access-config <name>] [--gateway <name>] [--orientation <left|right>] [--taught-with <labware-type>]
orca teachpoints delete <device> <position>

orca teachpoints labware set   <device> <position> <labware-type> [--z-offset 2] [--clear clearance]
orca teachpoints labware clear <device> <position> <labware-type>

orca deck-layouts create <device> <name> --config-file <path>
orca deck-layouts update <device> <name> --config-file <path>
orca deck-layouts delete <device> <name>

orca access-configs create <name> --access-type <vertical|horizontal> [--gripper-offset 20] [--vertical-clearance 20] [--horizontal-clearance 100]
orca access-configs update <name> --access-type <...> [...]
orca access-configs delete <name>

Both create and update need exactly one of --coords (inline JSON object) or --coords-file (a path to one). Coordinate values must be numbers or strings; a bool, a null or a nested object is rejected. --orientation is required for cartesian coordinates.

update always rewrites the coordinates, so it cannot be used to change only the access config. Access config, gateway, orientation and taught-with change only when their option is passed, coord_type is preserved, and per-labware overrides are left alone (edit those with teachpoints labware).

A deck-layout create or update needs a runtime rebuild to take effect. access-configs delete is refused while the config is protected or still referenced by a teachpoint.

See also

• Mid-flight inspection: read the state before you change it.
• Recover from errors: the decisions an error-paused thread accepts, which are not these verbs.
• Move parameters: the six layers behind grip profiles, teachpoints and carry.
• State: why a record is unsettled and which verb settles it.
• Audit and history: reading back what an operator did, and when.