Microsoft Fabric: How to set it up for your organization

This blog is the fourth post in a series around Microsoft Fabric, a relatively new Software-as-a-Service solution for anything concerning data platforms and data warehouses. In the previous blog posts, we explained the use of Link to Microsoft Fabric to seamlessly integrate Microsoft Dataverse into Fabric, we discussed security options in Security Considerations when using Microsoft Fabric and we dived into the concept of CI/CD with Fabric.

In this blog post we are going to make use of the info gathered during all previous blog posts and describe the different architectural decisions you need to make for your organization and your environment. We are going to take a top-down approach and start with the Organization-Level and Domains, from there we will go into more detail with Workspaces, Capacities, and Licenses. Finally, we will talk about Governance for Fabric. During each step of the way we will use as an example a Data Warehouse solution, one of the most requested solutions with our customers.

Fabric from the organizational level

A lot of companies already have lots of legacy data either on-premises or in the cloud and it is expected that will stay growing exponentially. One of the most common requests from customers now is how to organize and manage that data properly, how to be able to perform efficient governance. The original data warehouse solutions tend to be more centralized, where data from all sources are ingested and transformed, after which it is projected out again by reports. For small-sized companies in terms of employees and data this works just fine, in Microsoft Fabric you could set up a small data warehouse solution as seen below.


In this case the Medallion pattern as recommended by Microsoft is used, where the Gold layer is represented
by the combination of Warehouse with Semantic Model. This is a very basic setup and could be sufficient when you only have several data engineers or BI engineers who only have to serve several different reports to
for example the Finance department.

If there are multiple departments involved, the scale of data to be ingested increases or multiple teams are involved in the whole ETL. tenant-level planning of the Workspaces should be done. Microsoft offers guidelines for tenant-level planning and workspace-level planning with checklists you can follow.  The guidelines are very detailed so there is no point in replicating it here, we do want to mention the following which was not very clear in the Microsoft documentation:

  • There can only be 1 Power BI App per Workspace, so for Workspace-level planning keep this in mind. Try to group your Power BI reports/dashboards/workbooks so you can serve your audience with 1 Power BI App. Alternatively you can also include Links in your Power BI App so they can easily navigate to another relevant Power BI App. 
  • It is possible to store multiple workspaces in a single Git repository, just make use of folders when creating the Git link. You can for example have a Domain with multiple Sub-Domains and Workspaces in a single Git repository.  We will also use it in our example later on.
  • When using Power BI Deployment pipelines keep in mind you can only have 1 Deployment pipeline per Workspace stage. If you want to be able to deploy multiple Workspaces to a new stage, you might want to use Azure DevOps (or other tooling) and Deployment REST Apis to orchestrate this.

Example setup with Domains and Workspaces in Microsoft Fabric

We will go back to our example Datawarehouse solution and add the following requirements:

  • There are 2 departments (Finance and HR) which both use the same data sources (Oracle) but also have some data sources which should only be visible for their own department.
  • There are now 3 teams working in Microsoft Fabric: 1 team responsible for ETL (Extraction, Transform and Load) of all the data and 2 BI teams (1 per department, together they manage the Company Data too). The ETL is allowed to work with all the data, the BI teams are only allowed to handle the data from their respective department. 
  • There should be 3 different Power BI apps, a general one for all departments, and 1 specific for each department.

If we take above requirements, we could setup the following Domain and Workspace strategy:


In this case we have the following:

  • On each Domain and Workspace level appropriate access roles can be set for each team to be able to contribute.  
  • Each Domain has a Domain Admin which manages the Workspaces within. 
  • The ETL team is responsible for Bronze and Silver Workspaces, the BI Team are responsible for the Gold and Reports Workspaces in their own domain and the General domain. 
  • End-users should not have access to any Workspace, but only access to the Power BI App and the underlying data. With the use of security groups appropriate access can be given. 
  • Additionally, HR also needs some data from the General Domain, a shortcut has been added as an example on how to accomplish this (and avoid data duplication).

DTAP deployments could be done on Domain-level or Workspace-level, where Domain-level requires multiple Fabric Pipelines or an Azure DevOps setup for orchestration. For more information
about this, you can read CI/CD with Fabric. As you can see from the above simple example, it can become very complex very quickly so try to start as early as possible with the domain and workspace planning.

Capacities and Licenses in Microsoft Fabric

You can find almost all information about licenses and capacities at the Microsoft Learn documentation, we will give some highlights we think are important to know:

  • Microsoft Fabric users and Microsoft Power BI users use different resources in Microsoft Fabric! If you don’t make use of Power BI reports as a user, it could be enough to use a Fabric SKU Capacity with a Fabric Free License. The same goes for only using Power BI semantic models (not DirectLake) and Power BI reports, you only need a Power BI license and no Fabric license or capacity.
  • If you are small to medium-sized companies with <250 Power BI users you don’t need a F64 (P1) license, instead you can work with the Power BI Pro or Premium Per User licenses. Most likely in this case, you should start with a small Fabric SKU capacity with Pro licenses first and scale up if needed.
  • You can also pause and resume Fabric capacities when not needed (e.g. development environments), just keep in mind that any outstanding charges due to carry-forward usage will be billed.
  • Capacities are linked to Workspaces, not Domains. There is an option in the Fabric Admin Portal though to Assign Workspaces to a Domain by selecting Capacities.


If we use the Microsoft documentation and above highlights and apply this to our use case we got the following options for Capacities:

  • 1 Capacity per DTAP environment: the downside of this approach is that it is much harder to govern costs based on department. You would have to do the calculations yourself (or program it).
  • 1 Capacity per department: this will make it easier for governance however you introduce the risk of too much consumption in the DTA environments which causes problems in your Production. Definitely something you want to avoid.
  • A combination of 1 and 2 could work, where you can also pause Capacities which are not in use. In this case you would have like 4-6 capacities, 3 of those would be used for the production environment (3 Domains), the remaining capacities can be paused when not needed to save cost.
As for the licenses for small to medium-sized companies it should be enough to use 1 or more Fabric Capacities, give Power BI developers and end-users a Pro license, and for the remaining users use Fabric free licenses. Make use of the subscription functionality for people who only need to read a report and don’t need direct access.

Governance in Microsoft Fabric

In the previous sections we talked about different aspects of Governance namely Domains/Workspaces/Capacities/Licenses. Governance monitoring in Microsoft Fabric is done with the use of Microsoft Purview Hub, for a detailed explanation go check out this documentation from Microsoft. This Purview hub and the Microsoft Purview portal are integrated, so often you will be redirected to Microsoft Purview. Keep in mind there that depending on your requirements the standard free account at Microsoft Purview is not enough and you need to make more costs, check out the purchase options here.

Screenshot of the Microsoft Purview hub admin view.

Conclusion

This guide provided some insights in the various architectural decisions you need to make for your organization and your environment, in the case of Microsoft Fabric it is all about Domains, Workspaces and Capacities. We hope that this will help you out in your projects!

Facebook
X
LinkedIn
WhatsApp
Email