The State of Agentic Data Science

Agents can write the code, run the analysis, and produce a convincing explanation. How do you check that they’ve answered the right question, used the data appropriately, and reached a conclusion you can act on? I joined Gaël Varoquaux (scikit-learn, Inria, and Probabl), Shipra Arora (Bain & Company), and Luca Fiaschi (PyMC Labs) to explore how data scientists are delegating work to agents and building verification into that process: combining programmatic checks, agents critiquing other agents’ work, and human judgment about the problem and its consequences. That changes where we spend our time, what we need to tell our agents, and how we decide whether faster analysis is actually better analysis. We examine the mistakes agents make, which checks can themselves be delegated, and how those workflows need to evolve as models improve. You can also find the full episode on Spotify, Apple Podcasts, and all other platforms. 👉 The next cohort of our Master Agentic Data Science course starts Oct 6. We’ll build workflows that investigate interventions, run experiments, challenge findings, and preserve what your team learns. You’ll leave with reusable notebooks, agent skills, and practical methods for taking on more ambitious analyses while keeping the statistical judgment that makes them worth acting on. Friends of Vanishing Gradients get 20% off with code MADSOCG20. 👈 In This Episode * Verification inside the workflow. Programmatic checks, agent critique, and human review play different roles. Shipra explains how the risk of a decision shapes which checks can be delegated and where a person needs to stay involved. * The illusion of velocity. Gaël distinguishes experienced data scientists getting more done from people convincing themselves they’re getting more done, and explains why statistical judgment still matters. * Where the human work moves. Shipra describes spending less time coding and more time specifying the problem, supplying business context, and turning model outputs into decisions. * The case for stakeholder access, mistakes included. Luca argues that business users can uncover insights a data team would miss because they know the problem domain, even when some of their analyses go wrong. * When a stronger model knows too much. General knowledge can creep into an answer that should be grounded in your particular dataset. Better models still need to know how a prediction will be used and which features will actually be available. * Instructions that survive a model upgrade. Problem definitions and production constraints remain useful; procedural instructions need evals to establish whether they still help or have started holding the model back. * Synthetic consumers grounded in real behavior. Luca describes PyMC Labs’ work with Colgate-Palmolive and why transaction histories matter when modeling consumer responses and price sensitivity. * What to measure beyond speed. The panel weighs accuracy, reliability, explainability, and cost, with different priorities for an individual analyst and a system serving hundreds of stakeholders. * Data products people can question. Agents can let stakeholders interrogate models and run simulations, while faster experimentation could open up the backlog of ideas teams never find time to test. Resources * The Agentic Data Science Playbook by Hugo Bowne-Anderson, Luca Fiaschi, & Thomas Wiecki. * PyMC Labs’ AI-based customer research with Colgate-Palmolive. * Synthetic Consumers: A Practical Guide. 👉 The next cohort of our Master Agentic Data Science course starts Oct 6. We’ll build workflows that investigate interventions, run experiments, challenge findings, and preserve what your team learns. You’ll leave with reusable notebooks, agent skills, and practical methods for taking on more ambitious analyses while keeping the statistical judgment that makes them worth acting on. Friends of Vanishing Gradients get 20% off with code MADSOCG20. 👈 How You Can Support Vanishing Gradients Vanishing Gradients is an independent podcast, workshop series, blog, and newsletter about what people are building with AI and what survives contact with real users. * Become a paid subscriber. * Share this episode with a friend or colleague who’s building with agents. * Subscribe to the Vanishing Gradients YouTube channel. * Read and subscribe to the Vanishing Gradients newsletter. * Join our livestreams and free workshops on Luma. * Browse upcoming workshops. Get full access to Vanishing Gradients at hugobowne.substack.com/subscribe