Recipe index
Choose the example closest to your task, then adapt its inputs, step IDs, and outputs. Each linked TypeScript example uses public package imports and is compiled and checked against the packaged library.
For an existing script or workflow, use tubeless-make-pipeline from the
agent skill pack to choose step boundaries before adapting
one of these recipes.
| Intent | Executable recipe | Main primitives |
|---|---|---|
| Run all steps with a default result | minimal-pipeline.ts |
definePipeline({ id, steps }) |
| Reuse deterministic step outputs across runs | step-output-cache.ts |
Versioned opt-in, checked codec, readable ages, cache controls and artifact receipts |
| Return one step’s exact result type | precise-result.ts |
finalize: step, required output, inferred result type |
| Omit empty inputs and identity child mapping | inherited-inputs.ts |
runOrThrow(), compatible parent inputs through fromPipeline |
| Run independent DAG branches concurrently | parallel-dag.ts |
maxConcurrency, plan() checks controls, dependency joins, stable final reports |
| Declare a static family of checks | parameterized-steps.ts |
Recipe-local tuple helper, literal IDs, explicit dependencies |
| CPU parallelism on Node worker threads | worker-threads.ts |
createWorkerThreadAdapter, fromRemote, structured clone, pool ownership |
| Sequential import or ETL | typed-import.ts |
createSteps, dependsOn, requireOutputs, targets |
| Define and compose pipelines in YAML or JSON | yaml-pipelines.ts |
compilePipelineDocument(document, registry), adapters, skips, child fan-out |
| Validate options, outputs, and results | validated-boundaries.ts |
Standard Schema, outputSchema, resultSchema |
| Inspect, plan, or graph a pipeline or command | typed-import.ts |
tubeless inspect, tubeless plan, tubeless graph, toMermaid |
| Safe write/publish preview | publish-with-gates.ts |
dryRun, optionalDependsOn, skipAfterFailureOf |
| Deliberately omit unnecessary work | conditional-step.ts |
step with skip, valued skip, skip-aware output typing |
| Preserve independent work after failure | best-effort.ts |
continueOnError, finalized result narrowing |
| Compose one reusable workflow | child-pipeline.ts |
fromPipeline, domain mapOptions, child controls, async mapResult |
| Repeat a child until a bounded finish decision | iteration.ts |
iteratePipeline, fresh state, next / finish, static controls |
| Run a bounded in-process agent | agent.ts |
defineAgentLoop, defineTool, ordered outcomes, immutable state; guide |
| Resume a workspace agent after a restart | agent-durable.ts |
SQLite checkpoints, stable execution keys, saved conversation and outcomes; contract |
| Run a model-backed workspace agent | agent-model.ts |
Default prompt, project instructions, native conversation and compaction; setup |
| Attach MCP server tools to an agent | agent-mcp.ts |
connectMcpServers, HTTP and stdio servers, allowlists, await using; contract |
| Chain many tools from one agent script | agent-codemode.ts |
defineAgentLoop({ codemode: true }), typed code-mode decisions, generated declarations, nested child runs, previews; contract |
| Run a typed agent over an extract step’s output | agent-extract.ts |
defineAgent with inputSchema, task, resultSchema; fromPipeline with dependsOn; contract |
| Run the workspace agent on Claude | agent-model-anthropic.ts |
anthropicModel(), Messages API, compaction; providers |
| Continue an application chat across prompts | Chat sessions | createAgentSession, captured model/tool grants/limits, persisted conversation, clear() |
| Prompt an agent repeatedly with live pipeline reporting | agent-repl-model.ts |
tubeless agent, retained conversation, /clear, shared reporter, Tab completion, model selection, --env-file, pluggable provider; CLI |
| Wait for user review before continuing | user-input.ts |
waitForInput, typed pluggable input, cancellation, dry-run preview; contract |
| Ask for choices, confirmation, or text in YAML | yaml-user-input.ts |
Registered waitForInput(...).run, output schemas, declined confirmation skip; guide |
| Let a real LLM choose tools and finish | agent-openai.ts |
Application-owned Responses callback, accumulated outcomes, validated finish; setup |
| Delegate to subagents and ordinary pipelines | agent-delegation.ts |
pipelineTool, private child state, shared call/decision/depth/concurrency limits |
| Embed an agent in an ordinary pipeline | agent-pipeline.ts |
fromPipeline, validated child answers, dependent work, credential-free previews |
| Supply local or remote workspace capabilities | agent-environment.ts |
AgentEnvironment, inherited capabilities, optional Node adapter |
| Read, edit and check a workspace with default tools | agent-workspace.ts |
Default read/write/edit/bash/list/search tools, custom additions, real filesystem and shell execution |
| Call a real HTTP service | remote-steps.ts |
fromRemote, fetch cancellation, validated HTTP output |
| Host a pipeline in a durable engine | host-embedding.ts |
runOrThrow, pass correlationId / parentRunId |
| Invoke an Airflow DAG and validate its result | airflow/remote.ts |
fromRemote, stable run identity, polling, XCom validation; setup |
| Run a pipeline inside an Airflow task | airflow/hosted.ts, Python DAG |
runOrThrow, host-owned retries, correlation, structured task logs |
| Run a pipeline inside a Temporal Activity | temporal/activities.ts |
runOrThrow, Activity retries, heartbeats, cancellation; setup |
| Run a pipeline inside an Inngest durable step | inngest/functions.ts |
step.run, retries, JSON results, local preview; setup |
| Materialize a Dagster asset with a pipeline | dagster/hosted.ts, Python asset |
Dagster Pipes, materialization metadata, data versions, traces; setup |
| Run a pipeline on AWS Step Functions | step-functions/handler.ts, SAM template |
Lambda hosting, retries, deadlines, correlated logs; setup |
| Deploy to your Cloudflare account | cloudflare/cloudflare.config.ts, cloudflare/worker.ts |
tubeless init cloudflare, createCloudflareApp, Access, R2 cache store; setup |
| Fan out over runtime items | fan-out-progress.ts |
forEachPipeline with skip, stable keys, concurrency, progress |
| Inspect keyed fan-out failures | fan-out-progress.ts |
error.fanOut, bounded diagnostics, caller-directed reruns |
| Show determinate progress | fan-out-progress.ts |
reportProgress, bounded live CLI rows, complete final trees |
| Watch the live TTY reporter | live-tui.ts |
named steps, nested details, pane-only logs; reporter.logPane |
| Retry and rate-limit remote calls | resumable-enrichment.ts |
withRetry, RateLimiter, injected sleep and signal |
| Resume durable long-running work | resumable-enrichment.ts |
dryRun, openCheckpoint, withCheckpointedBatch |
| Record typed artifact reads and writes | artifact-lineage.ts |
recordArtifact, loaders/savers, reuse; guide |
| Write local pipeline artifacts | node-artifacts.ts |
definePaths, atomic writeJson, context.cwd, dryRun |
| Turn a schema-backed pipeline into a CLI | automatic-cli.ts |
definePipelineCommand(pipeline), inferred flags, automatic reporting |
| Map custom command inputs to pipeline options | cli-job.ts |
definePipelineCommand, conditional mapOptions, tubeless run |
| Handle cancellation and deterministic testing | cancellation-and-testing.ts |
createPipelineTestRuntime, captured status/progress |
| Supply intermediate outputs in tests | step-output-overrides.ts |
overrideStep, validated outputs, override provenance |
| Export lifecycle events | tracing.ts |
app-owned JSON / OTel adapters, composition, onExporterError |
| Observe lifecycle without changing execution | Lifecycle hooks | per-callback metadata snapshots; treat domain result values as read-only |
| Watch many primitives in one run | peloton.ts |
delays, logs, children, fan-out, retry, gates, test runtime |
| Watch an advanced YAML pipeline | yaml-peloton.ts |
declarative graph, concurrent handlers, retries, progress, dry runs, gates |
| Expose a project to CLI and Studio | tubeless.project.ts |
defineProject, inferred flags, tubeless list |
| Register custom command adapters | project/tubeless.project.ts |
defineProject, explicit command adapters, custom mappings |
| Host Studio beneath a private admin URL | Gateway hosting | --public-url, server-only gateway token, application-owned authorization |
| Complete commands, IDs, and flags in a shell | Shell completion | tubeless completion bash|zsh|fish, project IDs, targets, command flags, scoped Cloud/auth flags |
| Recover from mistyped CLI input | Typos and suggestions | Did you mean ...? hints, tubeless list, unique run-ID prefixes |
The model-agent recipe explicitly requests high reasoning;
omit that option when selecting a model that does not support it. The recipe loads guidance from the physical
workspace path and uses that same cwd for all tool calls. The prompt requires nested guidance
before file operations or task commands in its scope. The Claude recipe
uses the same factory with anthropicModel(); both adapters accept baseUrl for
compatible APIs. Custom OpenAI tools need inline strict schemas; oversized result
batches use explicit previews while the harness keeps complete outcomes. See the
model-agent contract.
Record the delegation recipe with --trace or
--store, then use tubeless history <run-id> for typed agent/turn/call history
and child-agent links; add --json for the same structured projection.
For ownership and governance discovery, adapt graph-metadata.ts.
Use metadata, querySteps, and toMermaid({ query }); read graph metadata
for bounds, trace identity, and document compatibility.
Static step families
Use parameterized-steps.ts when several
statically known steps share an implementation. Its defineRangeChecks helper
accepts a readonly tuple of specifications and returns an ordered tuple of ordinary
steps. Each specification names a stable ID, description, source step, and bounds.
The example checks positive quantities, fulfillment limits, and catalog prices.
The helper expands the whole family synchronously at module load. Spread it into
steps and, when each check should be selectable, targets. Pass the tuple to
dependsOn to join all checks, or use an individual tuple member to depend on one.
Plans, traces, Mermaid diagrams, and Studio show each check as an ordinary step.
definePipeline still validates duplicate IDs and missing dependencies.
This is a recipe-local pattern, not a new package export. Adapt the concrete
factory and specification type to your domain. Every check in this example
returns a number; sources must publish numeric arrays. The mapped tuple type
preserves each position’s literal ID. Its one assertion accounts for Array.map
losing tuple positions, and is justified by the factory always using the supplied
ID and returning the same output type. Do not reuse that assertion for a factory
whose output type varies by specification. Prefer explicit declarations for
heterogeneous steps unless you can prove their mapping separately.
Pass an inline tuple as shown, or retain literal IDs with as const on a separate
specification list. An already widened array cannot recover exact positions or
literal IDs. The tuple is enough for ordered expansion and destructuring; no
second ID-keyed container is needed. Use forEachPipeline for items discovered
during execution. Step families only describe a graph known before a run starts.
The example has no side effects, so its checks also execute during dry runs. If
you adapt the factory to write data, declare its ordinary dryRun policy there.
Run the registered recipe:
bun run tubeless -- run --project examples/project/tubeless.project.ts order-checks -- --quantities 2 --quantities 3 --prices 10 --prices 5
bun run tubeless -- plan --project examples/project/tubeless.project.ts order-checks --target valid-prices
Node helpers
tubeless/node provides optional helpers for pipelines that use the local
filesystem, environment, and worker threads. It uses Node built-ins and adds no runtime package
dependencies. Importing core does not load these helpers.
| Helper | Behavior |
|---|---|
definePaths({ name: "relative/path" }) |
Returns a factory; pass context.cwd on each run. Keys are preserved and resolved values have type string. |
readJson<T>(path) |
Synchronously parses JSON; missing files and malformed JSON throw. The type parameter does not validate data. |
writeJson(path, value) |
Writes pretty JSON with a trailing newline, creates parent directories, and replaces the file by renaming a sibling temporary file. Serialization must succeed before disk changes. |
resetDir(path) |
Recursively deletes the directory and recreates it empty. Use for generated output only. |
requireEnv(name, usedBy) |
Reads the environment when called and throws a descriptive error for missing or empty values. |
createWorkerThreadAdapter |
Reuses a bounded Node worker pool for explicit module exports; validates results through fromRemote. The caller owns close(). |
openCheckpoint, withCheckpointedBatch |
Track completed work for resumable pipelines. |
writeJson throws when the value has no JSON representation, including top-level
undefined, functions, symbols, and values whose toJSON returns undefined.
Serialization failures leave existing files and directories untouched.
Each write uses its own temporary file, including across worker threads. Concurrent
writes replace complete files; the last rename wins. This does not merge checkpoint
updates or provide a lock for read-modify-write operations.
Filesystem helpers do not know whether a pipeline is in dry-run mode. Put writes and
directory resets in steps marked dryRun: "skip", as in
node-artifacts.ts. Validate untrusted JSON at
the boundary with a schema; readJson<T> alone is not validation. Read required
environment values during execution so importing, inspecting and planning a
pipeline does not require credentials.
Choose a pattern
-
Use an ordinary step for one unit of domain work.
-
Set
dryRun: "skip"or provide a side-effect-freedryRunhandler before exposing side-effecting work through CLI dry-run support. -
Add
skiponly for a successful policy decision, never to hide an error. -
Use a child pipeline when the child has value independently; use a normal helper function when it does not.
-
Use
forEachPipelinewhen every item needs child-pipeline lifecycle and reporting. AddskiptoforEachPipelinewhen policy may omit the whole fan-out and a skip should appear in reports. UserunConcurrentfor lightweight worker functions that should throw on the first failure. UserunConcurrentPartialwhen the caller needs completed results plus that failure without throwing. Both stop admitting work after the first observed failure or cancellation, then drain active workers without cancelling siblings.runConcurrentPartialreturnsConcurrentPartialResult<R>: branch onok, never onfailure. Success has dense input-orderreadonly R[]results; failure has sparse results and the first rejection or observed abort error.completedIndexesidentifies successful outputs, includingundefined; a rejection can also beundefined. Invalid concurrency still throws, including for empty input.runBatchedandrunBatchedPartialapply that same throw-or-partial split to fixed-size batches instead of individual items. -
Group application pipelines with
defineProject(id, [pipelineA, pipelineB])fromtubeless/project. Useproject.get(id)to retain the selected pipeline’s exact option and result types; call its existing methods directly. LetdefinePipelineinfer its generics to preserve literal IDs; annotate the finalizer’s return type when an explicit result contract is needed. UsePipelineInput<typeof pipeline>andPipelineResult<typeof pipeline>for wrapper contracts instead of restating the pipeline generics. The input type is the value accepted byrunbefore schema transformation. Pipeline commands retain the exact wrapped pipeline ascommand.pipeline. Put optionalnameanddescriptionon a pipeline when CLI and Studio need friendlier presentation.definePipelineCommandinherits both; command-level values override them without changing the pipeline ID. Add optional{ name, description }as the third argument for presentation; the project name defaults to its ID. Compile parsed documents withcompilePipelineDocument(document, registry), then register selected pipelines. Reusecompiled.metadatathrough ordinary object construction, adding explicit name/description overrides as needed. A child need not be a project entry to run. A checked-indefineProjectexposes its pipelines to the CLI and Studio directly; they infer flags from the options schema’s Standard JSON Schema input metadata. Without that metadata, supply explicit commandparamsusing adapters in the project’s entry list, even for pipelines with no domain inputs. Register each pipeline exactly once, directly or through its command. Mixed lists preserve exact lookup types; duplicate pipeline IDs fail regardless of entry form. Default-export the project to select it for CLI and Studio; without a default, multiple distinct projects are rejected. Run a uniquely exported schema-backed pipeline file directly when inferred flags are sufficient. Start customized scripts withdefinePipelineCommand(pipeline); an explicit command wins when both it and its pipeline are exported. Use--exportto resolve multiple commands or pipelines. Useoverridesfor presentation only; explicitparamsandmapOptionsare advanced options for type-only pipelines or custom input shapes. Non-stringenum/constconstraints and$refsiblings with validation constraints also require explicit parameters; see CLI inference. UsedefineCommandfrom the same entrypoint for standalone scripts. Projects share their explicit adapters between terminal commands and Studio. Preview selection withcommand.plan()ortubeless plan; do not simulate planning with--plan.--stepand--targetare argv flags;mapOptionsand hooks readstepIdsandtargets. Commands expose--resumeonly when an explicit adapter configures managedcheckpointsupport or setsresume: trueand handles the value itself. Automatic project commands omit the flag. -
Declare public goals with
targets: [step]on the pipeline, select their IDs for goal-oriented execution, and usestepIdsonly for an exact filter. The control type rejects using both selections together. Without run selection controls, every step is selected. Omitted definitiontargetsexposes the last step in execution order as a selectable goal;targets: []exposes none. These declarations do not select work for a run. Omittedfinalizereturns that step’s output orundefinedif absent. UserequireOutputswhen the final domain result is not meaningful without specific step outputs. Read planselectionReasonsinstead of recreating target-closure logic in a CLI or application. -
Use the file layout and export conventions in the project with custom adapters only when custom adapters are needed. Do not infer executable modules from run history or the filesystem. Cancel only a live launch owned by the current studio process; it is not crash-resume and does not abort sibling launches. Studio pages top-level history; search includes nested runs across all pages, and run links open recorded details directly. Run details can switch to a live step graph that expands nested runs, agent turns, and tool calls. See the Studio controls.
Use fromRemote when a step calls another execution system. If a dry run
contacts that system, both the adapter and remote worker must honor
context.dryRun. A worker or durable host can invoke the whole pipeline when
it needs to own job delivery and retries; see remote steps.
For dependency, failure, and selection behavior, read core concepts.
For workbench commands, read the CLI. For hosted execution, follow
Cloud authentication and deployed runs; list loaded
pipelines, then run one by exact name or listed slug with JSON input. For the local run UI, read
the studio. To connect historical runs to a compiled graph and handler
release, set implementationVersion as shown in tracing.
Studio’s definition history groups
runs by definition and compares observed versions. Definition metadata remains visible
while its run pages refresh. Complete recorded snapshots must
match their hashes, including recorded child implementation identities, to be accepted
by NDJSON or SQLite history.
Host Studio privately
Use the Studio gateway recipe to mount a long-lived CLI process beneath an application’s admin URL. Tubeless supplies prefix-aware pages/APIs and backend authentication; your application checks login and admin permission for every request and owns deployment/storage. Use an explicit deployed catalog, keep the backend token server-only, and never retry an uncertain launch. No framework adapter or server-internal imports are needed.