Batch Testing Arrives for Custom AI Prompts in Copilot Studio

Batch testing for custom AI prompts in Power Platform is a new preview feature that has recently arrived in Copilot Studio in my test environment.

Prompts are quickly becoming one of my favorite tools across Copilot Studio, Agent Flows, and Power Automate to add small AI-powered timesavers into other automations. In many scenarios they take something that might otherwise be accomplished with a big set of Power Automate action steps and deliver the same using AI-based thought and reasoning rather than multiple fully defined paths or loops.

There is one problem with this approach though. At what point have you tested any single prompt with enough samples of the input data to ensure you are getting good quality and reliable outputs?

That's where this new batch test functionality fits. You can quickly throw a wide range of possible inputs at your prompt and see how it performs or even generate potential test inputs with AI.

One place where there is a big opportunity for improvement is that right now this doesn't seem to work with file inputs. This would be a really interesting use case for test, particularly if you could hook it up to a SharePoint or OneDrive folder full of test files.

I can't wait to try this out more fully. It seems like a really solid addition to our toolset to make prompts really useful in lots of different Power Platform scenarios.

First posted on Linkedin on 10/03/2025 -> View Linkedin post here