Extracting Spark Event Logs in Fabric for Monitoring & Optimization
Get spark metrics by extracting and parsing spark event logs
#spark
Posts tagged #spark
Quick Note : Fabric Spark Resource Profile
Just an FYI on spark profiles in Fabric
Extracting Fabric Spark Driver Logs Using API
Retrieving the spark driver logs in Microsoft Fabric
Upgrade Fabric Workspaces To The Latest GA Runtime
Using Semantic Link & APIs
Delta Lake Tables For Optimal Direct Lake Performance In Fabric Python Notebook
In my last blog, I showed how to use Polars in a Fabric Python notebook to read, write and transform data. What I did not cover was how to write a Delta Lake table in a Python notebook that’s optimal for DirectLake performance. All engines in Fabric ...
Mutable vs Immutable Fabric Spark Properties
In Microsoft Fabric, you can define spark configurations at three different levels: Environment : This can be used at the workspace or notebook/job level by creating Environment item. All notebooks and jobs using the environment will inherit spark &...