TUBELESS v0.2.2
Markdown

Recipe index

Every linked TypeScript example is compiled by the package typecheck. pack:verify also executes these modules from the published tarball. Start with the smallest example matching the workflow rather than assembling primitives from the API inventory.

Intent Executable recipe Main primitives
Sequential import or ETL typed-import.ts createSteps, dependsOn, requireOutputs, targets
Validate options, outputs, and results validated-boundaries.ts Standard Schema, outputSchema, resultSchema
Inspect, plan, or graph a pipeline or command typed-import.ts tubeless inspect, tubeless plan, tubeless graph, toMermaid
Safe write/publish preview publish-with-gates.ts dryRun, optionalDependsOn, skipAfterFailureOf
Deliberately omit unnecessary work conditional-step.ts step.skippable, valued skip, skip-aware output typing
Preserve independent work after failure best-effort.ts continueOnError, structured run result
Compose one reusable workflow child-pipeline.ts fromPipeline, mapOptions, resolved async mapResult
Call a real HTTP service remote-steps.ts fromRemote, fetch cancellation, validated HTTP output
Host a pipeline in a durable engine host-embedding.ts runOrThrow, pass runId / parentRunId
Fan out over runtime items fan-out-progress.ts forEachPipeline.skippable, stable keys, concurrency, progress
Inspect keyed fan-out failures fan-out-progress.ts error.fanOut, bounded diagnostics, caller-directed reruns
Show determinate progress fan-out-progress.ts reportProgress, bounded live CLI rows, complete final trees
Watch the live TTY reporter live-tui.ts named steps, nested details; persist with --store
Retry and rate-limit remote calls resumable-enrichment.ts withRetry, RateLimiter, injected sleep and signal
Resume durable long-running work resumable-enrichment.ts dryRun, openCheckpoint, withCheckpointedBatch
Expose and run a typed command-line program cli-job.ts definePipelineCommand, conditional mapOptions, tubeless run
Render plans and diagnostics rendering.ts renderPipelinePlan, renderPipelineError
Handle cancellation and deterministic testing cancellation-and-testing.ts createPipelineTestRuntime, captured status/progress
Export JSON or OpenTelemetry lifecycle events tracing.ts trace context, exporter composition, JSON / OTel, onExporterError
Persist, port, inspect, launch, or cancel runs local-observability.ts SQLite / NDJSON stores, tubeless history --pipeline <id>, studio projector
Watch many primitives in one run peloton.ts delays, logs, children, fan-out, retry, gates, test runtime
Project layout, IDs, and command manifest tubeless.project.ts pipelines/, scripts/, definePipelineProject, tubeless list

Selection rules

  1. Use an ordinary step for one unit of domain work.
  2. Set dryRun: "skip" or provide a side-effect-free dryRun handler before exposing side-effecting work through CLI dry-run support.
  3. Use step.skippable only for a successful policy decision, never to hide an error.
  4. Use a child pipeline when the child has value independently; use a normal helper function when it does not.
  5. Use forEachPipeline when every item needs child-pipeline lifecycle and reporting. Opt into forEachPipeline.skippable when policy may omit the whole fan-out and a canonical skipped disposition matters. Use runConcurrent for lightweight worker functions that should throw on the first failure. Use runConcurrentSettled when the caller needs completed results plus that failure without throwing.

Use fromRemote when a unit of work lives on another engine but the parent DAG still runs in this process. Omit dryRun only when the adapter and the remote worker are side-effect free under context.dryRun. Embed the whole pipeline in Temporal/Lambda/a worker when the graph must outlive the process.

  1. Use definePipelineCommand for pipeline scripts; use defineCommand only when the script is not centered on a pipeline. Preview selection with command.plan() or tubeless plan; do not simulate planning with --plan. --step and --target are argv flags; mapOptions and hooks read stepIds and targets.
  2. Declare public goals with targets: [step] on the pipeline, select their IDs for goal-oriented execution, and use stepIds only for an exact filter. Use requireOutputs when the final domain result is not meaningful without specific step outputs. Read plan selectionReasons instead of recreating target-closure logic in a CLI or application.
  3. Use tubeless/render when plans or diagnostics cross a human or JSON presentation boundary; do not duplicate selection-reason or error formatting. Use tubeless/reporter for optional TTY run reporters; do not import them from tubeless.
  4. Copy consumer file layout, export names, and IDs from the project manifest. Register project commands explicitly; do not infer executable modules from run history or the filesystem. Cancel only a live launch owned by the current studio process; it is not crash-resume and does not abort sibling launches.

For semantics behind these choices, read core concepts. For workbench commands, read the CLI. For the local run UI, read the studio.