Search for a command to run...
Using the PowerBIRestClient in Semantic-Link v0.5.0
Principal Program Manager, Microsoft Fabric CAT helping users and organizations build scalable, insightful, secure solutions. Blogs, opinions are my own and do not represent my employer.
Last week Microsoft released an open-source text embedding model called Harrier in three sizes- 270M, 0.6B and 27B. I have been testing it in my RAG pipeline and so far it has crushed all my metrics.
For Power BI Copilot and Data agents with semantic models, you must use Prep Data for AI configuration to ground the responses in the context added in Prep for AI. In this blog, I will show you how yo
At FabCon Atlanta last week, the updated notebook Copilot for data engineering and data science was announced. It brings agentic capabilities to the Copilot and is much more intelligent and Fabric-awa
Using Fabric data agent Python SDK
Use workspace monitoring to monitor MCP server use
Update : Mar 8, 2024
As of March 8, you can cancel the Fabric Dataflow Gen2 refresh in service using UI:
Programmatically cancel the refresh:
DFg2 can be resource intensive, and if the refresh takes longer than expected, it may consume a significant amount of CUs. Thankfully, you can use the Power BI Rest API to cancel it programmatically. My friend Alex Powers already has a PowerShell script that you can use. You can also use the Power BI VS Code extension by Gerhard Brueckl.
But I would like to show you how you can do this using the PowerBIRestClient in the latest version of Semantic-Link (v0.5.0). The API is similar to the FabricRestClient I blogged about in my last blog. The new version has many updates I love and I will write about it soon. But in the meantime, checkout the official documentation for the details.
In the below code:
I first get the list of all the dataflows in a workspace using the Get Dataflows API. You can pass either the workspace name or id. By default, the workspace id of the workspace hosting the notebook is used.
For each dataflow, get the transaction id and the status of the dataflow using Get Dataflow Transactions API.
Cancel the refresh, using the Cancel Dataflow Transaction API
Semantic Link makes calling APIs very easy. Authentication is handled automatically for you so no need to generate a token.
# Install semantic-link >= 0.5.0
!pip install semantic-link --q
import pandas as pd
import sempy.fabric as fabric
from sempy.fabric.exceptions import FabricHTTPException
# Instantiate the client
client = fabric.PowerBIRestClient()
def get_dataflow_transactions(workspace_id, dataflow_id):
try:
response = client.get(f"v1.0/myorg/groups/{workspace_id}/dataflows/{dataflow_id}/transactions")
if response.status_code != 200:
raise FabricHTTPException(response)
return pd.DataFrame()
transactions = response.json().get('value', [])
if not transactions:
return pd.DataFrame([{'status': 'No Status', 'transaction': 'No Transaction ID', 'refreshType': 'None'}])
first_transaction = transactions[0]
return pd.DataFrame([{
'status': first_transaction['status'],
'transaction': first_transaction['id'],
'refreshType': first_transaction['refreshType']
}])
except Exception as e:
print("Error getting transactions. Check workspace, dataflow id:", e)
return pd.DataFrame()
def get_dataflows(workspace):
try:
workspace_id = fabric.resolve_workspace_id(workspace)
except Exception as e:
print("Check workspace name/id:", e)
return pd.DataFrame()
try:
response = client.get(f"v1.0/myorg/groups/{workspace_id}/dataflows")
if response.status_code != 200:
raise FabricHTTPException(response)
return pd.DataFrame()
dataflows = response.json().get('value', [])
dataflow_df = (pd.json_normalize(dataflows, max_level=1)
.rename(columns={"name": "dataflowName", "objectId": "dataflowID"})
.dropna()
.reset_index(drop=True))
result = dataflow_df.join(dataflow_df.apply(lambda x: get_dataflow_transactions(workspace_id, x['dataflowID']).iloc[0], axis=1))
return result
except Exception as e:
print("Error in get_dataflows:", e)
return pd.DataFrame()
def cancel_dataflow(workspace, transaction_id):
"""
Cancel a specific dataflow transaction.
"""
try:
workspace_id = fabric.resolve_workspace_id(workspace)
response = client.post(f"v1.0/myorg/groups/{workspace_id}/dataflows/transactions/{transaction_id}/cancel")
if response.status_code != 200:
raise FabricHTTPException(response)
return None
return response.json()
except Exception as e:
print("Error:", e)
return None
Example:
Get dataflow transactions:
Cancel the dataflow:
This was one example . You can use this client to call any Power BI Rest endpoint and make GET, POST requests. If you are new to Python, note that the actual code is just a couple of lines, majority is for error/exception handling to make it robust. I love Semantic Link, and it's becoming indispensable day by day.