We all know what YAML is – it’s like JSON, just with indentation instead of brackets. Easier to write and read. That’s it, isn’t it? In most situations… yes. But if we look a little deeper, we’ll find features that many people have no idea exist. And let me emphasize right away, I’m not judging […]
Latest Posts
Setup Git credentials for Service Principal in Azure Databricks
Introduction Databricks Jobs can execute code stored locally (1 on the picture below) or stored in a remote Git repository (2). Second approach simplifies the creation and management of production jobs while enabling automated continuous deployment. It eliminates the need to create and maintain a separate production repository within Azure Databricks, reducing the burden of […]
Databricks: MERGE WITH SCHEMA EVOLUTION
Anyone who has ever designed an ETL process involving more than a few tables of data has likely encountered the need to build a metadata-driven framework. By ‘framework,’ I mean any solution that standardizes this process and allows for scaling through configuration changes. Regardless of whether it involved BIML, SSIS packages generated from C#, dynamic […]
Terraforming ADF: Shared Self-Hosted Integration Runtime
In one of our previous posts, we explained what self-hosted integration runtimes are and how to fully configure them using Terraform. Today, we’ll take it a step further by discussing the sharing mechanism that allows us to reuse the same runtime across multiple Azure Data Factories. Multiple Integration Runtimes Let’s consider the following scenario: our […]
Last comments