Skip to content

v0.0.15

Changed: Provenance Schema and Lazy IO

Rework of provenance models for correctness and scalability.

  • is_source / is_sink flags replace node_type TEXT on node_executions. A node can now be source AND sink simultaneously. Composite status is derived from child runs (no column needed).
  • sink_outputs_json added to dag_runs — consolidated sink outputs are persisted when a run completes.
  • IoPayload modelinput_json / output_json fields moved into a lazy-loaded io: IoPayload | None on both NodeExecution and DagRun. Inspection queries (inspect_run, get_recent_runs) no longer load JSON blobs by default.
  • load_io(dag_run, node_labels=...) — new method populates io on a DagRun and selected nodes in a single DB round-trip. Does not recurse into sub-DAGs.
  • Composite node input recordingDagRunner now records actual upstream data as input_json instead of the {"composite": "MapNode"} placeholder.

New: End-to-End Engine Test

  • test_lyth_e2e.py — full lifecycle test: starts lyth engine, waits for triggers, stops via SIGTERM, verifies provenance records and PID file cleanup. Marked @slow.

Documentation

  • Added posts section to website, converted Show HN post to markdown.
  • Restructured releases nav to show only 3 most recent in sidebar.