Latest Posts

Databricks, Delta Lake, DevOps

Elevate your Databricks development workflow with SHALLOW CLONE

In this post, I would like to present how the SHALLOW CLONE functionality can be utilized to streamline your development process by accelerating the creation of dedicated development environments and enabling comprehensive data testing in your CI/CD pipeline. One more time, what exactly is SHALLOW CLONE? Let’s start with a quick recap of Delta Lake […]

MicrosoftFabricNotebookRunMultiple_00
Microsoft Fabric

Run multiple notebooks in parallel using runMultiple in Microsoft Fabric

Orchestration is about organizing and controlling many computer systems, apps, and/or services, linking together many tasks to carry out a bigger workflow or process. These processes can have many tasks that are automated and can include many systems. The aim of orchestration is to make regular, repeatable processes run smoother and faster, helping data teams […]

DevOps

githooks and pre-commit framework in the daily work of a Data Engineer

Do you recall a situation when, at the beginning of the project, your team, after many days of intense discussions, finally established coding standards that everyone agreed to follow? You thoroughly discussed rules regarding capitalization, spacing, indentation, line lengths, etc. You nearly came to blows over whether the comma should be placed at the beginning […]