Blog about Microsoft Data Platform and Azure cloud. #SQLServer #PowerBI #Azure #New
Anyone who has ever designed an ETL process involving more than a few tables of data has likely encountered the need to build a metadata-driven framework. By ‘framework,’ I mean any solution that standardizes this process and allows for scaling through configuration changes. Regardless of whether it involved BIML, SSIS packages generated from C#, dynamic […]
In one of our previous posts, we explained what self-hosted integration runtimes are and how to fully configure them using Terraform. Today, we’ll take it a step further by discussing the sharing mechanism that allows us to reuse the same runtime across multiple Azure Data Factories. Multiple Integration Runtimes Let’s consider the following scenario: our […]
Azure Data Factory is almost never created as an isolated resource. In nearly every project where we have used this service, it was provisioned alongside a Key Vault and a Storage Account. In the following article, I’ll describe the standard method for authenticating between these services using managed identities. As indicated by the title, I’ll […]
Git and mandatory CI/CD have become widely accepted standards, and an increasing number of people are advocating for defining cloud resources using IaC tools. Many are coming to understand that quality, reliability, and scalability in modern cloud data projects are simply unattainable without embracing the DevOps culture and proper automation. The ClickOps resistance movement is […]