Delta Lake is becoming more and more popular. It is right now the default storage format used by Spark engine so when you will not specify it differently it will be used by Synapse Analytics Spark Pools or Azure Databricks. Why it become so popular? Before we will start let’s say what was the problem […]
Author: Adrian Chodkowski
Dynamic SQL Lineage in Microsoft Purview
Purview is one of the most needed services in the Azure cloud. It gives us the opportunity to scan, classify, and govern data assets within our organization. This service is constantly developing, so we can notice more and more valuable tools. One of the main questions that I hear from my customers is about data […]
Scanning Databricks local hive metastore from Microsoft Purview
One of the latest news related to Purview is the announcement of a new Databricks connector that will aid in gaining insights from the Hive metastore within the Databricks instance. In this article, I have prepared a short tutorial to demonstrate how to use the connector, what it looks like, and the benefits you will […]
Lakehouse – What is it, and why is it so popular?
Over the years, many different concepts related to systems dedicated to data analysis have appeared on the market. For those interested in the topic, concepts such as data lake, data warehouse, or data mesh are likely familiar. Some of these concepts disappeared faster than others, while others have become standards and have been implemented for […]