microsoft/promptflow

★ 11,125PythonAudience · developerComplexity · 3/5LicenseSetup · moderate

Mindmap

mindmap
  root((repo))
    What it does
      LLM app development
      Flow pipelines
      Quality evaluation
      Deployment
    Tech stack
      Python
      OpenAI API
      Azure OpenAI
      VS Code
    Features
      Interactive testing
      Batch evaluation
      CI/CD integration
    Use cases
      Build chatbots
      Test LLM quality
      Deploy AI apps

mindmap root((repo)) What it does LLM app development Flow pipelines Quality evaluation Deployment Tech stack Python OpenAI API Azure OpenAI VS Code Features Interactive testing Batch evaluation CI/CD integration Use cases Build chatbots Test LLM quality Deploy AI apps

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Build a chatbot or LLM-powered app by connecting prompts, Python code, and AI models in a testable pipeline.

USE CASE 2

Run automated quality evaluations against test data to measure how well your LLM application performs before shipping.

USE CASE 3

Integrate LLM quality checks into a CI/CD pipeline so tests run automatically on every code change.

USE CASE 4

Deploy a finished LLM flow to a cloud platform or Azure AI for managed, collaborative infrastructure.

Tech stack

PythonOpenAI APIAzure OpenAIVS Code

Getting it running

Difficulty · moderate Time to first run · 30min

Requires an OpenAI or Azure OpenAI API key to run flows against a live model.

Use freely for any purpose, including commercial use, as long as you keep the copyright notice.

In plain English

Prompt flow is a set of development tools from Microsoft for building applications that use large language models (LLMs), which are the kind of AI systems that power chatbots like ChatGPT. The toolkit covers the full process: creating a working prototype, testing it, improving its quality, and deploying it so real users can access it. The central concept is a "flow," which is a pipeline that connects an LLM, prompt instructions, Python code, and other tools in a defined sequence. You build a flow to describe how your application should respond, then run it against data to see how well it performs. Flows can be tested interactively in a terminal or evaluated in batch mode against larger datasets, which is useful for catching problems before going live. Getting started involves installing two Python packages and running a few command-line instructions. The README walks through creating a chatbot from a template, connecting it to an OpenAI or Azure OpenAI API key, and running an interactive chat session in the terminal. A Visual Studio Code extension is also available for people who prefer a visual interface over the command line. Quality evaluation is a core part of the workflow. You can define metrics, run your flow against test data, and see scores that tell you how well the AI is performing before you ship anything. The results can be plugged into a continuous integration and deployment pipeline so that quality checks happen automatically whenever the code changes. Deployment options include sending the flow to a hosting platform of your choice or integrating it directly into your application's codebase. Microsoft also offers a cloud-hosted version through Azure AI for teams that want shared collaboration and managed infrastructure. The project is open-source under the MIT license.

Copy-paste prompts

Prompt 1

I want to build a simple chatbot using prompt flow and OpenAI. Walk me through installing the packages, creating a flow from the template, and running an interactive chat session in the terminal.

Prompt 2

How do I define a custom evaluation metric in prompt flow and run it against a test dataset to score my LLM application's responses?

Prompt 3

I want to add prompt flow quality checks to my GitHub Actions CI pipeline. Show me the configuration needed to run evaluations automatically on each pull request.

Prompt 4

How do I connect prompt flow to Azure OpenAI instead of the standard OpenAI API, and what configuration changes does that require?

Prompt 5

I built a flow locally and want to deploy it so real users can call it. What are my deployment options and how do I use the Azure AI hosted version?

Open on GitHub → Explain another repo

← microsoft on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.