Latest Posts

Azure, Databricks, DevOps

Terraforming Databricks #1: Unity Catalog Metastore

Over the past two years, we have participated in numerous projects where Azure Databricks was implemented from the ground up. Each of these deployments allowed us to learn something new, verify previous solutions, and ultimately develop a methodology that allows us to deploy Azure Databricks in a standardized, enterprise-scale-ready manner. As a result, the newly […]

QuerySQLFromDevOpsUsingServiceConnectionCreds_00
Azure SQL Database, DevOps

Executing SQL queries from Azure DevOps using Service Connection credentials

Executing SQL queries on Azure SQL Database using Azure Devops can be a complex and challenging task, particularly when it comes to establishing a secure and reliable connection. This option can be valuable in numerous scenarios. For instance, you can insert pipeline metadata into the database to track deployment information or changes. This technique also […]

GenerateGITCredentialsForServicePrincipalInDatabrics
Databricks

Setup Git credentials for Service Principal in Azure Databricks

Introduction Databricks Jobs can execute code stored locally (1 on the picture below) or stored in a remote Git repository (2). Second approach simplifies the creation and management of production jobs while enabling automated continuous deployment. It eliminates the need to create and maintain a separate production repository within Azure Databricks, reducing the burden of […]