A prompt can be three paragraphs long and still change the behavior of an entire application. It can decide whether an assistant answers, abstains, calls a tool, or invents the argument that the tool receives. Yet it is often reviewed with less discipline than a change to a configuration file. Let’s start there. If editing a sentence can change what your software does, that sentence belongs in your release process. ...
September 2026 update: This article is retained as an earlier overview. Its evaluation and judge snippets do not reflect the current GenAI API. For current examples and the prompt lifecycle, read MLflow for AI Engineering: Prompts Are Release Artifacts. MLflow for Generative AI Systems I’ll start this post by recalling what Hayen said in her book Designing Machine Learning Systems (2022): ‘Systems are meant to learn’. This statement reflects a simple fact: today, LLMs and to a lesser extent vision language models are winning in the Data Science world. But how do we measure this learning? RLHF work is always a good indicator that perplexity will improve, but let’s return to a key point: LLMs must work as a system, therefore debugging is important, and that’s where the necessary tool for every Data Scientist, AI Engineer, ML Engineer, and MLOps Engineer comes in: MLflow. ...