Debug and profile agentic app experiences with Instruments

Overview of how the Foundation Models Instruments template helps debug and profile agentic app experiences built with the Foundation Models framework, including Dynamic Instructions and tool call loops.

The three challenges unique to LLM app development: probabilistic output (non-deterministic responses that break standard unit testing), model-to-model communication (coordinating data flow across multiple models), and observability (knowing where things went wrong in a multi-model pipeline).

Introduction to the craft companion demo app — a journaling app with an interactive brainstorming feature that uses two sets of Dynamic Instructions: one for idea generation and one for tutorial creation, both backed by the server model on Private Cloud Compute.

How to start profiling with the Foundation Models template in Instruments — selecting the template, recording a session, and an important note about sensitive prompt data in trace files.

A walkthrough of the Foundation Models instrument layout: tracks and lanes on the timeline (including the instructions lane and model inference lane with yellow/orange bars), the detail view, and the inspector — and how to use the tree view to inspect sessions, requests, inferences, and tool calls.

How to measure and optimize LLM experience performance using three key metrics: time-to-first-token (reduce by shortening prompts), tokens-per-second (benchmark across configurations), and total latency (reduce perceived wait with streaming).

Summary of what was covered, requirements to get started (Xcode 27 and latest OS), and pointers to related sessions on the Evaluations Framework and Agentic App Experiences.