Hi everyone,
I've been working on a problem that I think many of us building AI in Swift are facing: how do you debug why an AI agent behaves differently between runs?
Traditional logging tells you what happened. But when a model skips a step, changes its reasoning order, or produces a different output with identical input — you're left staring at walls of logs with no clear answer to why.
I've released DProvenanceKit — a reasoning observability framework for Swift that lets you:
Record every reasoning step an agent takes (non-blocking, async-safe) Query for reasoning patterns ("find runs where X happened but Y didn't") Diff two executions to see structural differences Detect regressions automatically with rule-based validation Think of it as Git for AI logic.
Example:
Swift // Record an execution try await DProvenanceKit<MyAIDecision>.run(contextID: "case-123", store: store) { DProvenanceKit<MyAIDecision>.record(.documentEvaluated(documentID: "DocA", score: 0.95)) DProvenanceKit<MyAIDecision>.record(.conflictDetected(reason: "timeline_inconsistency")) DProvenanceKit<MyAIDecision>.record(.finalDecisionMade(approved: false)) }
// Query for suspicious patterns let suspiciousRuns = try await store.queryRuns( TraceQueryDSL<MyAIDecision>() .requiring(step: "conflictDetected") .missing(step: "documentEvaluated") // Find runs where conflict was reported but no docs evaluated )
// Diff two runs let diff = engine.diff(base: runA, comparison: runB) print(diff.changes) // See exactly which steps appeared, disappeared, or moved The design:
Built specifically for on-device AI (macOS/iOS) with Apple Foundation Models, MLX, or Core ML Non-blocking recording (touches only in-memory buffer) Durable, crash-safe persistence with SQLite WAL Works with async/await context propagation Status:
Experimental (core engine complete, actively evolving). Free for development/testing under BSL 1.1.
GitHub: https://github.com/Therealdk8890/DProvenanceKit
I'm curious if this resonates with anyone here building AI in Swift. What debugging/observability challenges are you facing with AI systems?