Building Large Language Models Applications using Prompt Flow in Azure ML — Simple LLMOps

Balamurugan Balakreshnan

4 min readAug 5, 2023

Introduction

Build Large Language Models using Prompt Flow in Azure ML
Using Prompt Flow
Using Azure Cognitive search vector services
using Azure open ai service

Concept

First create a vector store and provide cognitive search service
Create a connections to open ai and also cognitive search
Create a prompt flow
Create a pipeline to run the prompt flow
i am using existing vector store and cognitive search service

Pre-requisites

Azure Cognitive search
Azure machine learning service
Azure Open AI service

Code

Go to Azure Machine Learning services
Create a connection for open ai in prompt flow sections

Now create vector index search connections
i am using existing azure cognitive search service
Start with uploading data to vector store

Give a name
Upload the pdf files
Select the cognitive search service for vector index
Select the vector index
Now Click Next
In the next screen select Azure Open ai services

Select compute as serverless

Review and create the vector store connection

Prompt flow or LLM application

Now create a prompt flow
Create a Standard flow

Input and output will automatically be created

Select the Azure Open AI connection
Now add flow to search the index

now generate prompt context

Code

from typing import List
from promptflow import tool
from embeddingstore.core.contracts import SearchResultEntity

@tool
def generate_prompt_context(search_result: List[dict]) -> str:
    def format_doc(doc: dict):
        return f"Content: {doc['Content']}\nSource: {doc['Source']}"
    
    SOURCE_KEY = "source"
    URL_KEY = "url"
    
    retrieved_docs = []
    for item in search_result:        entity = SearchResultEntity.from_dict(item)
        content  = entity.text or ""
        
        source = ""
        if entity.metadata is not None:
            if SOURCE_KEY in entity.metadata:
                if URL_KEY in entity.metadata[SOURCE_KEY]:
                    source = entity.metadata[SOURCE_KEY][URL_KEY] or ""
        
        retrieved_docs.append({
            "Content": content,
            "Source": source
        })
    doc_string = "\n\n".join([format_doc(doc) for doc in retrieved_docs])
    return doc_string