Today, we’re going to talk about using Azure Data Factory to run a custom query in Databricks. Why we need dedicated article about it? Well, Azure Data Factory helps us move and change data and in some scenarios Databricks can be treated as a source from which we want to move data. When we use […]
Author: Adrian Chodkowski
Run multiple notebooks in parallel using runMultiple in Microsoft Fabric
Orchestration is about organizing and controlling many computer systems, apps, and/or services, linking together many tasks to carry out a bigger workflow or process. These processes can have many tasks that are automated and can include many systems. The aim of orchestration is to make regular, repeatable processes run smoother and faster, helping data teams […]
How to determine if Direct Lake dataset is falling back to Direct Query in Microsoft Fabric
When you create a Direct Lake semantic model, it typically operates in Direct Lake mode. This means it directly accesses the delta table from the OneLake. This is the preferred scenario because it helps queries run fast, sometimes as fast as import mode. In Power BI, when utilizing Direct Lake mode, the semantic models directly […]
Why should I ‘refresh’ Direct Lake models in Microsoft Fabric?
You’ve probably heard about a new mode for datasets/semantic models called Direct Lake. This mode combines the advantages of existing modes: the performance of Import mode and the lack of need to refresh or cache data from Direct Query. It works by directly accessing files from a one lake without requiring extra steps. This means […]