Flows API

The Flow surface spans three routers in the product backend:

• /api/pipeline/* — running recipes, run history, observability, anomaly detection.
• /api/flow/* (flow_codegen) — visual ↔ SQL recipe round-tripping, zone management.
• /api/flow/* (flow_ai) — agent recipes and AI-driven node placement.

For the conceptual model — node types, zones, run modes, lineage — see Flow. The endpoints below cover programmatic access.

Run history

GET/api/pipeline/runs🔒 auth

List recent pipeline runs. Filterable by status (running, success, failed, cancelled), recipe_type, since. Default sort: created_at desc.

GET/api/pipeline/runs/{run_id}/details🔒 auth

Full details for a single run: parameters, compiled SQL, stdout, downstream lineage, exit status. Used by the run-detail panel in the UI.

Triggering runs

POST/api/pipeline/run🔒 auth

Trigger a run. Body: {target: {type, name}, scope: 'single'|'upstream'|'downstream'|'full'}. Returns the new run_id immediately; runs are async — poll /api/pipeline/runs/{run_id}/details for status.

The four scopes:

Scope	Builds
single	Just the target recipe.
upstream	Target + every upstream recipe needed to bring its inputs up-to-date.
downstream	Target + every downstream recipe that consumes its output.
full	Full lineage — upstream and downstream.

Observability

GET/api/pipeline/api-health🔒 auth

Per-router health summary. Returns each router's recent error rate and median latency. Used by the platform's status dashboard.

GET/api/pipeline/anomalies🔒 auth

Active data-quality anomalies — rule violations detected on the last run of each dataset. Filterable by severity, dataset_name, rule_id.

GET/api/pipeline/anomalies/types🔒 auth

Distinct anomaly types with counts.

GET/api/pipeline/anomalies/rules🔒 auth

Configured data-quality rules — assertion type, target column, threshold.

GET/api/pipeline/anomalies/tolerances🔒 auth

Per-rule tolerances (e.g. allow 1% null rate before flagging). Used to quiet noisy rules during a known issue.

PUT/api/pipeline/anomalies/tolerances/{check_id}🔒 auth

Update tolerance for a specific rule.

POST/api/pipeline/anomalies/{alert_id}/dismiss🔒 auth

Dismiss a single anomaly without resolving it (acknowledged, won't re-surface until conditions change).

POST/api/pipeline/anomalies/{alert_id}/resolve🔒 auth

Mark an anomaly as resolved (the underlying issue was fixed).

POST/api/pipeline/anomalies/bulk-dismiss🔒 auth

Dismiss many anomalies at once. Body: {alert_ids: [...]}.

POST/api/pipeline/anomalies/bulk-resolve🔒 auth

Resolve many anomalies at once.

GET/api/pipeline/row-audit🔒 auth

Per-row freshness audit — which datasets refreshed in the last N hours, row count delta, dbt run that produced the refresh.

GET/api/pipeline/dbt/status🔒 auth

Status of the most recent dbt run across all tenants — per-model success/failure, total duration.

GET/api/pipeline/freshness🔒 auth

Per-dataset freshness — last_refresh_at, freshness_threshold, and a derived is_fresh boolean. Drives the "stale data" badges in the Flow.

Visual ↔ SQL round-trip

POST/api/flow/generate-sql🔒 auth

Compile a visual recipe (Prepare / Join / Group By / Stack) to dbt SQL. Body: the recipe's structured config. Returns the compiled SQL plus a list of inputs and inferred output columns.

POST/api/flow/parse-to-visual🔒 auth

Inverse of generate-sql — parse dbt SQL into a structured visual recipe config when possible. Returns null if the SQL doesn't fit any visual pattern (e.g. a window function with multiple partitions).

POST/api/flow/visual-lineage🔒 auth

Given a visual recipe config, return the column-level lineage edges it would emit when compiled. Used to preview lineage before saving.

POST/api/flow/preview-sql🔒 auth

Compile + run the recipe against a sample of input rows. Returns the result rows without persisting anything. Used by the recipe editor's Preview button.

GET/api/flow/recipe/{model_name}🔒 auth

Fetch the structured config for a recipe (visual structure or SQL body) by its dbt model name.

POST/api/flow/save-recipe-steps🔒 auth

Persist a structured recipe config. Triggers SQL compilation server-side and writes the dbt model file.

Zones

GET/api/flow/zones🔒 auth

List zones in the current project.

POST/api/flow/zones🔒 auth

Create a zone. Body: {name, color}.

PUT/api/flow/zones/{template_id}🔒 auth

Rename / recolor a zone.

DELETE/api/flow/zones/{template_id}🔒 auth

Delete a zone. Nodes assigned to it become unassigned.

POST/api/flow/zones/assign🔒 auth

Assign a node to a zone. Body: {model_name, zone_id}. Used by drag-and-drop on the Flow canvas.

POST/api/flow/zones/auto-assign🔒 auth

Auto-assign nodes to zones using the platform's heuristics (cluster by upstream connector, group raw inputs together). Returns the proposed assignments without committing them; the client confirms.

AI / Agent recipes

GET/api/flow/ai-nodes🔒 auth

List AI-recipe nodes in the project. Filtered by recipe_type (embed, llm_enrich, agent, rag_search).

POST/api/flow/ai-recipes🔒 auth

Create an AI recipe. Body includes recipe_type (one of embed, llm_enrich, agent, rag_search), inputs, run_config. Idempotent — duplicate (project_id, model_name, recipe_type) is a no-op.

GET/api/flow/ai-recipes🔒 auth

List the project's AI recipes.

GET/api/flow/ai-recipes/{recipe_id}🔒 auth

Full config for an AI recipe.

PUT/api/flow/ai-recipes/{recipe_id}🔒 auth

Update an AI recipe's config or run_config.

DELETE/api/flow/ai-recipes/{recipe_id}🔒 auth

Delete an AI recipe.

POST/api/flow/ai-recipes/{recipe_id}/run🔒 auth

Trigger a run of an AI recipe. Returns the run_id; poll /api/flow/ai-recipes/{recipe_id}/runs for status.

GET/api/flow/ai-recipes/{recipe_id}/runs🔒 auth

Run history for an AI recipe.

Lineage

For lineage queries see Flow → Lineage and the /api/lineage/* surface in the OpenAPI spec.

Auth and access

All endpoints require a valid JWT. The legacy layer gates writes on require_role("editor", "admin"); the new layer (planned) will use object-level permissions on the recipe's output dataset.

Run lifecycle

A run progresses through these states:

queued → running → (success | failed | cancelled)

Run status updates are written to pipeline_runs and visible via the /api/pipeline/runs/{run_id}/details endpoint. The frontend polls every second; for high-volume programmatic access, consider polling once every 5–10 seconds and using the run's updated_at timestamp to short-circuit unchanged states.

There is no streaming run-log endpoint today. The full stdout becomes available once the run completes; partial output during a long-running recipe is not exposed via the API. (UI-side, the recipe editor reads the platform's internal log buffer directly — that's not a public surface.)