Anyone who has ever designed an ETL process involving more than a few tables of data has likely encountered the need to build a metadata-driven framework. By ‘framework,’ I mean any solution that standardizes this process and allows for scaling through configuration changes. Regardless of whether it involved BIML, SSIS packages generated from C#, dynamic […]
Latest Posts
Terraforming ADF: Shared Self-Hosted Integration Runtime
In one of our previous posts, we explained what self-hosted integration runtimes are and how to fully configure them using Terraform. Today, we’ll take it a step further by discussing the sharing mechanism that allows us to reuse the same runtime across multiple Azure Data Factories. Multiple Integration Runtimes Let’s consider the following scenario: our […]
Microsoft Fabric 101 Episode 3: Pausing and Scaling using portal and Powershell
Welcome to Microsoft Fabric 101 series – your comprehensive guide to mastering Microsoft Fabric. This series of articles and videos is designed to help you understand and effectively use Microsoft Fabric, whether you’re a beginner or looking to deepen your knowledge. We’ll cover everything from the basics of setting up and configuring your Fabric tenant […]
Terraforming ADF: Credentials & Linked Services
Azure Data Factory is almost never created as an isolated resource. In nearly every project where we have used this service, it was provisioned alongside a Key Vault and a Storage Account. In the following article, I’ll describe the standard method for authenticating between these services using managed identities. As indicated by the title, I’ll […]
Last comments