Programmatically Removing & Updating Default Lakehouse Of A Fabric Notebook

Using NotebookUtils to update notebook properties

I have written about default lakehouse of a Fabric notebook before here and here. However, unless you used the notebook API, there was no easy/quick way of removing all/selective lakehouses or updating the default lakehouse of a notebook. But thanks to tip from Yi Lin from Notebooks product team, notebookutils.notebook.updateDefinition has two extra parameters, defaultLakehouse and defaultLakehouseWorkspace which can be used to update the default lakehouse of a notebook. You can also use it to update environment attached to a notebook. Below are some scenarios how it can be used.

Get Notebook Definition:

Use the notebookutils.notebook.getDefinition to get the notebook in a ipynb format as below:

import json

nb = json.loads(
    notebookutils.notebook.getDefinition(
        "<notebook_name>", #nane of the notebook
        workspaceId='<workspace_id_notebook>' #workspaceid of the notebook
    )
) 
#returns a json with ipynb

Get the default lakehouse of a notebook:

import json

nb = json.loads(
    notebookutils.notebook.getDefinition(
        "<notebook_name>", #nane of the notebook
        workspaceId='<workspace_id_notebook>' #workspaceid of the notebook
    )
) ['metadata']

## Returns
"""
{'language_info': {'name': 'python'},
 'kernelspec': {'name': 'synapse_pyspark',
  'language': 'Python',
  'display_name': 'Synapse PySpark'},
 'kernel_info': {'name': 'synapse_pyspark'},
 'widgets': {},
 'microsoft': {'language': 'python',
  'language_group': 'synapse_pyspark',
  'ms_spell_check': {'ms_spell_check_language': 'en'}},
 'nteract': {'version': 'nteract-front-end@1.0.0'},
 'spark_compute': {'compute_id': '/trident/default'},
 'dependencies': {'lakehouse': {'default_lakehouse': 'af9de4fd-e3-772099b80f9d',
   'known_lakehouses': [{'id': 'af9de4fd--772099b80f9d'},
    {'id': 'ab14c0f6-4f284707'},
    {'id': 'c9c60586-dc95ac3c'},
    {'id': '63c748e6-7b34c7f3'},
    {'id': '1d289df0-b9585c'},
    {'id': '12414a55-cd3be'}],
   'default_lakehouse_name': 'workspace1',
   'default_lakehouse_workspace_id': 'd70c1ed700a13145ff'},
  'environment': {}}}
"""

As you can see above, the definition returns a list of all the lakehouses mounted in the notebook, including the default lakehouse.

💡
If you are a Semantic Link Labs fan like me, you can use labs to get the notebook definition as well. Check this out.

Remove All Lakehouses

To remove all lakehouses, we can remove all known and default lakehouses as below:

import json

def remove_all_lakehouses(notebook_name, workspace_id):
    try:
        nb = json.loads(notebookutils.notebook.getDefinition(notebook_name, workspaceId=workspace_id))
    except:
        print("Error, check notebook & workspace id")

    if 'dependencies' in nb['metadata'] and 'lakehouse' in nb['metadata']['dependencies']:
        # Remove all lakehouses
        nb['metadata']['dependencies']['lakehouse'] = {}

    # Update the notebook definition without any lakehouses
    notebookutils.notebook.updateDefinition(
        name=notebook_name,
        content=json.dumps(nb),
        workspaceId=workspace_id
    )

    print(f"All lakehouses have been removed from notebook '{notebook_name}'.")

# e.g.
remove_all_lakehouses("Notebook 1", "<workspaceid_of_notebook>")

Add A New Default Lakehouse

To add a new lakehouse and make it a default lakehouse without removing any existing alkehouses:

(notebookutils
    .notebook
    .updateDefinition(
        name = "<notebook anme>", 
        workspaceId="<worskpace id of the notebook>"
        defaultLakehouse="<lakehouse name of the new lakehouse>", 
        defaultLakehouseWorkspace="<workspace id of new lakehouse to be added>", 
        )
)
# content parameter is optional which keeps the existing content
💡
This will keep all existing lakehouses and make this lakehouse the default lakehouse. By design, you CANNOT update the default lakehouse of the notebook which is used to run above code. So if you need to update the lakehouse of “Notebook1”, you will have to do it from another notebook.

Remove All Lakehouses And Add New Lakehouse

To remove all existing lakehouses attached to the notebook and make a new lakehouse the default lakehouse:

# first rmeove the lakehouses
remove_all_lakehouses("Notebook_name", "workspaceid_of_notebook")

#add new lakehouse
(notebookutils
    .notebook
    .updateDefinition(
        name = "Notebook_name", 
        workspaceId="workspaceid_of_notebook"
        defaultLakehouse="<lakehouse name of the new lakehouse>", 
        defaultLakehouseWorkspace="<workspace id of new lakehouse to be added>", 
        )
)

Needless to say, when you make these changes, be sure to reference to the right paths in your code. You can use the .lakehouse for it (see here).

💡
I have tested to make sure it only updates the lakehouse and not the content of the notebook, but please test to ensure no other changes are made. Backup before making changes (you can use getDefinition to create a backup)

Did you find this article valuable?

Support Sandeep Pawar by becoming a sponsor. Any amount is appreciated!