Azure Power App using Azure Cognitive Service Florence model.

3 min readMay 6, 2023

Let’s build a Power App to use Azure Vision Cognitive service Florence Model for various use cases

What’s needed

Create a Azure Cognitive Service resource for vision in Azure portal
Get the URI and keys to use

Create a Power App

To create a power app first need to create a power flow
Flow is invoked by a powerapp trigger
Images is passed to the flow
on the default screen assign values to variables
Add an Add media with image control
Text box to display the output
Button to invoke the flow

Set(JSONImageSample, Substitute(JSON(UploadedImage1.Image, JSONFormat.IncludeBinaryData), """", ""));
Set(outputtext,florencemodel4.Run(JSONImageSample));

Power Flow

Flow name is florencemodel4
Let go deep into the flow
Note we are passing image as binary URI in json format
we need to make sure the proper image format is sent to florence model api
First create a variable and assign the output from power app
Variable name is: imagedata

now add Compose action
This is a important step to make sure the image is sent in proper format

Function to convert dataurito bring

dataUriToBinary(variables('imagedata'))

Now bring the HTTP connector to connect to florence api

For URL: https://modelresname.cognitiveservices.azure.com/computervision/imageanalysis:analyze?api-version=2022-10-12-preview&features=objects&language=en
Content-Type: application/octet-stream
Ocp-Apim-Subscription-Key: xxxxxxxx
Body: @{body(‘Compose_2’)}
For response we need to parse the response
Bring ParseJson

here is the schema

{
    "type": "object",
    "properties": {
        "modelVersion": {
            "type": "string"
        },
        "metadata": {
            "type": "object",
            "properties": {
                "width": {
                    "type": "integer"
                },
                "height": {
                    "type": "integer"
                }
            }
        },
        "objectsResult": {
            "type": "object",
            "properties": {
                "values": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {
                                "type": "string"
                            },
                            "confidence": {
                                "type": "number"
                            },
                            "boundingBox": {
                                "type": "object",
                                "properties": {
                                    "x": {
                                        "type": "integer"
                                    },
                                    "y": {
                                        "type": "integer"
                                    },
                                    "w": {
                                        "type": "integer"
                                    },
                                    "h": {
                                        "type": "integer"
                                    }
                                }
                            }
                        },
                        "required": [
                            "name",
                            "confidence",
                            "boundingBox"
                        ]
                    }
                }
            }
        }
    }
}

Now set the variable

Now apply for Each
now compose the output with mulitple output
then assign the variable to output

{
  "Name": @{items('Apply_to_each_2')?['name']},
  "probability": @{items('Apply_to_each_2')?['confidence']},
  "boundingbox": @{items('Apply_to_each_2')?['boundingBox']}
}