This blog is the second post in a series around Microsoft Fabric, a relatively new Software-as-a-Service solutions for anything concerning data platforms and data warehouses. In the first blog post we explained the use of Link to Microsoft Fabric to seamlessly integrate Microsoft Dataverse and Microsoft Fabric, including Dynamics Finance and Operations.
In this blog post we want to focus more on the security aspects when using Microsoft Fabric and explain any differences with previous Microsoft solutions. This post can be used as a guideline for your own (data) solutions.
Generally, when thinking about security for any data platform we can think about the following topics, which we will discuss:
- Microsoft Fabric inbound traffic
- Microsoft Fabric outbound traffic
- Microsoft Fabric Data Security
Keep in mind that Microsoft is still improving and changing Microsoft Fabric so some topics could already be outdated when reading this blog. Topics like Data Governance, Regulations, Monitoring will not be discussed in this blog.
Microsoft Fabric inbound traffic
Inbound network security refers to incoming traffic towards the Microsoft Fabric environment, such as through the Fabric portal to the Fabric backend or via REST APIs to the Fabric backend. The Fabric backend, in this case, is the backend as provided by Microsoft which handles all the Fabric Workloads, e.g. Data pipelines is a service that is hosted in one of the Fabric workloads hosted by the Fabric backend. An example of inbound traffic is accessing the OneLake with the use of the Sql Server Management Server tool or the Azure Storage Explorer tool. The Microsoft Fabric backend is located in a Microsoft Vnet but is accessible via the public
internet from a number of endpoints. If you want to further scale up security, there are 2 recommended practices:
Figure 1. Source: Fabric Security Whitepaper, inbound traffic options
- Use Private Links to a Vnet within Azure (the Vnet needs to be setup in your own tenant) and disable public internet access. Both options are available in the Fabric admin portal. Note, however, that when using this option, certain functionalities will no longer be operational, these can be found in the security whitepaper. Also, this option is tenant-wide, meaning the entire Microsoft Fabric / Power BI environment will be behind a Vnet, which has a significant impact on the existing environment. Keep in mind that for example existing users cannot access the Microsoft Fabric / Power BI portal through internet, instead a VPN or other solution needs to be in place. Additionally, any application that wants to have access to the OneLake needs to be part of the Vnet.
- Entra Conditional Access: Use Entra ID in combination with security policies to further protect access. Examples of this include office locations, secured laptops, MFA, etc.
In short option 2 is usually already in place and has the least effort to setup, however public internet access is still active. Option 1 is the most secure option, however has a major impact on the way to connect to the Microsoft Fabric / Power BI environment. The Power BI environment already had the option to disable public access for some time, but protection of data was only applied to Power BI Datasets and files in Power BI Workspaces. If you have a requirement that your Fabric Data Platform needs to be behind a Vnet including the API endpoints, you need to have the entire tenant in the Vnet. If having the whole tenant behind a Vnet is technically not desirable, maybe a hybrid solution can be thought off with having storage still in Azure (Datalake Gen2 / SQL Database) in a Vnet while making use Fabric services like Dataflow Gen2, Data Pipelines and others. In this case, outbound traffic needs to be secured, which will be explained in the next part.
For more information about inbound traffic see the following: Fabric Security Whitepaper.
Microsoft Fabric outbound traffic
Outbound network security concerns traffic from Microsoft Fabric towards an on-premises or Vnet environment. The following options are available, listed in order of priority as we see it in terms of recommended practices:
- Trusted Workspace Access: If the Data Source is an Azure Storage Account in an Azure Vnet, you can choose Trusted Workspace Access within Fabric. This involves using a generated service principal at the workspace level, which is configured as an exception on the Storage Account. This is a low-effort solution but only works with an Azure Storage account. You can also make use of Shortcuts, or alternatively utilize Data Pipelines and Dataflow Gen2.
- Vnet Data Gateway in Fabric in combination with an Azure Vnet and resources: Here, the Vnet data gateway is hosted within Fabric itself, and only the Azure Vnet needs to be further configured. Currently, only DataFlow Gen2 can use this managed Vnet data gateway, Azure Pipelines cannot be used (yet). This is a limitation if you want to be able to copy unstructured files.
- On-premises/Vnet data using a Virtual Machine with the on-premises data gateway application: Here, Dataflow Gen2 and Data Pipelines (preview) are supported, but this does require additional resources in the form of an (Azure) VM. The downside of this option is there is another resource that needs to be maintained, however gives you the flexibility to also unlock data from a secure on-premises network.
- IP Allowlist: If no Vnet construction is present, a Firewall can also be configured to work with whitelisting. The list can be programmatically retrieved via REST APIs or CLI to keep the Firewall configuration up to date.
In short, if your Data Source is an Azure Storage account use option 1. If your Data Source is in a Vnet or on-premises and you only need to copy structured data then option 2 (managed gateway) can be used. If you need a Data pipeline because of unstructured data then option 3 (gateway on a VM) can be used. You will always have the option of IP Allowlist, however should be considered a last resort option.
For more information about outbound traffic see the following references: Fabric Security Whitepaper, Trusted
Workspace Access, Access on-premises Data sources, Install an on-premises data gateway.
Microsoft Fabric Data Security
In addition to securing network traffic, there are also options to secure the data in OneLake. The following list is not in any specific order but will depend on your own requirements if you need to implement them or not:
- Authentication / Authorization with Entra ID at Workspace and resource levels: the standard 4 roles can be chosen here (Admin/Member/Contributor/Viewer). This option is sufficient if you only have security requirements at the Lakehouse or Warehouse level, e.g. securing your Data Mart. If it needs to be more refined, you need to use multiple Lakehouses/Warehouses/Workspaces or make use of the next option.
- Column-Level and Row-Level Security in Warehouse or Lakehouse SQL analytics endpoint: make use of security policies to control access to rows in the database table. It is possible to perform this security also on a Lakehouse table, but you need to make use of the SQL analytics endpoint.
- Dynamic Data Masking in Warehouse: use masking of sensitive data in a Warehouse, this option is complementary to the column-level and row-level security. SQL permissions can be set on the tables do determine which users can see the masked data or not. In this case it is important to realize that only the Viewer role will have masking functionality enabled, all other roles will see all data! Again if more refinement is required here, it is better to split up the Lakehouse and Warehouse accordingly and use multiple Workspaces.
Currently, when having unstructured data in a Lakehouse it is only possible to secure this on the Lakehouse-level. Microsoft is working on enhancing this, in the future folder and file-level security will be introduced in OneLake.
For more information about data security see the following references: Fabric Security Whitepaper, Row-level security in Microsoft Fabric, Security for Data Warehousing, Implement row-level security, SQL granular permissions.
Conclusion
This guide provided some insights in the security options in Microsoft Fabric for inbound / outbound traffic and when securing data in your Fabric Lakehouse or Fabric Warehouse. We hope that this will help you out in your projects!