Mlflow

An agent answered a question about Infest with 2,326 tokens of memory in its final context. Getting to that answer required thirteen model calls and 19,831 input and output tokens. The verifier accepted it. One of its claims attributed evidence from the fourth song to the first. That execution contains the central problem of this study. A smaller context can conceal a more expensive decision process, and an accepted answer can conceal an unsupported interpretation. Neither becomes visible if the experiment ends at the final prompt or the aggregate score. ...

An album ends. An agent has read every lyric. We ask it whether the ending transforms something established near the beginning. The agent can produce a convincing paragraph with almost no memory. That is precisely the problem. Fluency gives us very little evidence that it remembered the right songs, preserved their differences, or resisted turning a collection of voices into one convenient story. In my previous post on MLflow, I argued that a prompt belongs in the release process because changing an instruction changes application behavior. Here I want to bring that argument into my research on musical narratives. Memory selection changes behavior too. It needs an evaluation contract. ...

A prompt can be three paragraphs long and still change the behavior of an entire application. It can decide whether an assistant answers, abstains, calls a tool, or invents the argument that the tool receives. Yet it is often reviewed with less discipline than a change to a configuration file. Let’s start there. If editing a sentence can change what your software does, that sentence belongs in your release process. ...

Complete MLOps Series: Part 1 (current) | Part 2: Deployment → | Part 3: Production → Anatomy of an MLOps Pipeline - Part 1: Pipeline and Orchestration Why This Post Is Not Another Scikit-Learn Tutorial Most MLOps posts teach you how to train a Random Forest in a notebook and tell you “now put it in production.” This post assumes you already know how to train models. What you probably don’t know is how to build a system where: ...

Complete MLOps Series: ← Part 1: Pipeline | Part 2 (current) | Part 3: Production → Anatomy of an MLOps Pipeline - Part 2: Deployment and Infrastructure 8. CI/CD with GitHub Actions: The Philosophy of Automated MLOps The Philosophical Foundation: Why Automation Isn’t Optional Before diving into YAML, let’s address the fundamental question: why do we automate ML pipelines? The naive answer is “to save time.” The real answer is more profound: because human memory is unreliable, manual processes don’t scale, and production systems demand reproducibility. ...