Pipelines

3 min read 442 words Updated 3 October 2026

A pipeline is a named list of steps in .shipyard.yaml. Each run goes through the queue, executes the steps in order and writes every step's result to the run log.

Every step has an id and an optional timeout (default 2m, at most 15m); a whole run stops after 15 minutes. A failed step stops the pipeline. A pipeline has 1 to 20 steps.

1. Local project check — no AI credentials

pipelines:
  project-check:
    steps:
      - id: validate-config
        type: diagnostic
        timeout: 30s

Confirms the project's configuration and records the result. Needs no AI credentials.

2. Prometheus observation

prometheus:
  production:
    url: https://prometheus.example.com
    token_env: PROMETHEUS_TOKEN
    query: 'up{job="my-project"}'
pipelines:
  observe-production:
    steps:
      - id: validate-project
        type: diagnostic
      - id: collect-metrics
        type: prometheus
        source: production
        timeout: 15s

The query result is written to the run log. A failed query, an error status or a response with warnings fails the step.

3. Article draft with a different reviewer

pipelines:
  article-draft:
    steps:
      - id: write
        type: ai_text
        task: writing
        prompt: "Draft an article about ${input}. Use only supplied facts; list missing evidence."
        timeout: 2m
      - id: review
        type: ai_text
        task: verification
        prompt: "Review this draft and return a corrected version: ${previous}"
        timeout: 2m

writing and verification resolve through ai.tasks and the project's models override, so the draft and the review can use different models. ${input} is the run input; ${previous} is the previous step's output. They are filled once — placeholder text inside the input is passed on as written. Prompts go to the provider as text, never to a shell. AI calls can cost money.

Run a pipeline

In the console: New run → Pipeline → pick the pipeline and project. Or from the Pipelines view, Run pipeline. API equivalent:

curl -X POST http://127.0.0.1:8080/api/v1/runs \
  -H "Authorization: Bearer $SHIPYARD_API_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: my-project-check-001' \
  -d '{"project_id":"ssg","kind":"pipeline","task":"project-check","input":""}'

Reusing the idempotency key returns the run already queued. One run per project executes at a time; others wait in the queue.

If Shipyard stops mid-run, diagnostic runs are picked up again (up to three attempts). Pipeline and AI runs are marked failed instead, so a paid model call is never repeated without you — use Retry.

The log records each stage's start and output and ends with pipeline completed. Messages over 16 KiB are truncated.