One of the latest news related to Purview is the announcement of a new Databricks connector that will aid in gaining insights from the Hive metastore within the Databricks instance. In this article, I have prepared a short tutorial to demonstrate how to use the connector, what it looks like, and the benefits you will […]
Author: Adrian Chodkowski
Lakehouse – What is it, and why is it so popular?
Over the years, many different concepts related to systems dedicated to data analysis have appeared on the market. For those interested in the topic, concepts such as data lake, data warehouse, or data mesh are likely familiar. Some of these concepts disappeared faster than others, while others have become standards and have been implemented for […]
Load Synapse Analytics SQL Pool with Azure Databricks
In many projects, we have different tools that must integrate with each other. These scenarios are common when we talk about Synapse Analytics and Azure Databricks. Integrating those platforms can be beneficial in several ways. By combining the capabilities of Azure Synapse’s SQL pool with Azure Databricks’ Apache Spark cluster, you can analyze and transform […]
Dealing with ParquetInvalidColumnName error in Azure Data Factory
Data Factory or Integrated Pipelines under Synapse Analytics suite can be very useful as an extracting and orchestrating tool. It is a common scenario when we extract data from the source system and save it in a dedicated landing zone located in Azure Data Lake Storage Gen 2. The only question that can appear is […]