Search for a command to run...
Principal Program Manager, Microsoft Fabric CAT helping users and organizations build scalable, insightful, secure solutions. Blogs, opinions are my own and do not represent my employer.
No comments yet. Be the first to comment.
Last week Microsoft released an open-source text embedding model called Harrier in three sizes- 270M, 0.6B and 27B. I have been testing it in my RAG pipeline and so far it has crushed all my metrics.
For Power BI Copilot and Data agents with semantic models, you must use Prep Data for AI configuration to ground the responses in the context added in Prep for AI. In this blog, I will show you how yo
At FabCon Atlanta last week, the updated notebook Copilot for data engineering and data science was announced. It brings agentic capabilities to the Copilot and is much more intelligent and Fabric-awa
Using Fabric data agent Python SDK
Use workspace monitoring to monitor MCP server use
If you follow me on LinkedIn or Twitter, you may have seen my posts about using AI to detect Power BI reports with errors, visualizations that lack accessibility etc. All those posts used Gemini models because of their strong multi-modal performance. Well, Fabric now has strong multi-modal LLMs from OpenAI (GPT4.1, GPT5). In this blog, I experimented to see if I can use those models to detect if a report page has any errors. This is not a comprehensive test by any means but shows the capability and the use cases.
You could use this to scan your reports and detect errors before your users find them.
Use semantic link labs to generate pdf of the report
Convert pdf to image and encode it as base64
Send the base64 to LLM and ask it to detect any errors
Optional : if you have sensitive data on the report, you could optionally add noise to the image or apply filter (e.g. gaussian blur etc.) and then encode it. I will leave that to you but can cover if you drop it in comments below.
In my test, I had a report with four pages. 3 pages had error like below and the last page had no errors.
Page1 : Has error
Page 4 : No error
Attach a lakehouse to the notebook and install semantic-link-labs and pymupdf4llm
Below code will save the specified report as a PDF to the Files section in a lakehouse
%pip install semantic-link-labs pymupdf4llm
from sempy_labs.report import export_report
export_report(
report="Sales (1)", #name of your report
export_format="PDF",
file_name="mySalesreport", # name of the pdf to save
workspace="d70c1ed7-b6e4-40f1-911f-3300a13145ff", # workspace name or id where report lives
lakehouse="mylakehouse", # lakehouse to save file into
lakehouse_workspace="d70c1ed7-b6e4-40f1-911f-3300a13145ff" # workspace for the lakehouse
)
Using GPT 4.1 to detect errors on each report page
from synapse.ml.fabric.service_discovery import get_fabric_env_config
from synapse.ml.fabric.token_utils import TokenUtils
import fitz
import base64
from io import BytesIO
from PIL import Image
import requests
import json
# Setup Fabric API
fabric_env_config = get_fabric_env_config().fabric_env_config
auth_header = TokenUtils().get_openai_auth_header()
openai_base_host = fabric_env_config.ml_workload_endpoint + "cognitive/openai/openai/"
deployment_name = "gpt-4.1"
service_url = openai_base_host + f"deployments/{deployment_name}/chat/completions?api-version=2025-04-01-preview"
auth_headers = {
"Authorization": auth_header,
"Content-Type": "application/json"
}
prompt= """
Analyze the image of a dashboard and determine if it contains any errors, error messages, or failed data loading.
Look for phrases like 'Error fetching data', 'See details', error icons (X in circles), or visual indicators of failure.
Respond with 'ERROR DETECTED' if errors are present, or 'NO ERROR' if the page appears normal. Then provide a brief explanation.
"""
def pdf_page_to_base64(pdf_path, page_num, dpi=150):
doc = fitz.open(pdf_path)
page = doc[page_num]
pix = page.get_pixmap(dpi=dpi)
img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
buffer = BytesIO()
img.save(buffer, format="JPEG", quality=85)
doc.close()
return base64.b64encode(buffer.getvalue()).decode('utf-8')
def check_page_for_errors(base64_image):
payload = {
"messages": [
{
"role": "system",
"content": "You are an error detection assistant analyzing Power BI dashboard and reports."
},
{
"role": "user",
"content": [
{
"type": "text",
"text": prompt
},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{base64_image}",
"detail": "high"
}
}
]
}
],
"max_tokens": 500,
"temperature": 0.1
}
response = requests.post(service_url, headers=auth_headers, json=payload)
if response.status_code == 200:
return response.json()["choices"][0]["message"]["content"]
else:
return f"API Error: {response.status_code}"
def analyze_pdf_for_errors(pdf_path):
doc = fitz.open(pdf_path)
total_pages = len(doc)
doc.close()
results = {}
for page_num in range(total_pages):
print(f"Processing page {page_num + 1}/{total_pages}...")
base64_image = pdf_page_to_base64(pdf_path, page_num)
result = check_page_for_errors(base64_image)
results[page_num + 1] = result
if "ERROR DETECTED" in result:
print(f" -> ERROR FOUND on page {page_num + 1}")
else:
print(f" -> Page {page_num + 1} OK")
return results
pdf_path = "/lakehouse/default/Files/mySalesreport.pdf"
error_results = analyze_pdf_for_errors(pdf_path)
error_results
Below is the JSON returned by error_results
{1: 'ERROR DETECTED\n\nExplanation: The dashboard contains an error message at the top center stating "Error fetching data for this visual See details" along with an error icon (X in a circle). This indicates that at least one visual has failed to load data. The rest of the visuals appear to be functioning normally.',
2: 'ERROR DETECTED\n\nExplanation: The dashboard displays the message "Error fetching data for this visual" along with an error icon and a "See details" link, indicating that data failed to load for the visual.',
3: 'ERROR DETECTED\n\nExplanation: The dashboard contains a visual with an error message stating "Error fetching data for this visual See details" along with an error icon (X in a circle), indicating that one of the visuals failed to load data.',
4: 'NO ERROR\n\nExplanation: The dashboard displays all visuals and data without any error messages, icons, or indicators of failed data loading. All charts and figures are rendered correctly.'}
For my report, LLM correctly identified the pages with errors and described the errors as well.
I did not test all types of errors so if you wanted to make it more comprehensive, you could give few-shot example of each error and what to look for.