What the just hell just happened to Power BI? What is Fabric? Why is there Fabric? Is Azure Synapse dead, is it going the way of Azure Data Lake Analytics? (U-SQL… huh, remember that).

Well, here is the overview of the new service, the data lake evolved, compute independent to the storage, built for the dash mesh design approach, and is for a mix of what Microsoft call citizen developers, mostly doing self service data and analytics and building their own stuff, and the enterprise developers, building things for the whole enterprise.

The Aim

You should note that this is Microsoft Fabric, not Azure Fabric, so it’s the M365 of Data, moving away from the complexity of Azure, to a more friendly interface and experience. One of the issues with Power BI, wasn’t the service, but the idea of self service and that it was a solution for all your data needs. Having done quite a bit of Power BI consultancy, there seemed to be an idea in the clients heads, that it was a sliver bullet to solve their data needs. ‘I’ve been told we don’t need a data warehouse, we can do it all in Power BI’ is mostly an accurate quote from some clients. For some clients and small organisations, it can be done, if you have limited data needs and use cases. Microsoft have tried to scale it up, with Dataflows and Datamarts, that do meet some of the demands of larger projects but there are still limits to those technologies.

One of the things I used to say to clients is ‘They didn’t build the pyramids from the top down’. A number of times I was called in to help out a client and see what the issue was with their Power BI implementation. Most of the time, they were trying to do everything in Power BI, which could have been solved with a move to a database, rather than try to shove everything into Power BI. Part of this, was that the user didn’t have access to Azure or databases, or the technical ability to implement those sort of solutions. Microsoft Fabric can now give you access to a suite of tools to do that. Access to a database, need to run a query… no need to pop into Azure or Azure Data Studio (Or SSMS), log into Fabric. Want one platform for your users and a place for your data to stop Excel files flying around your email inbox…log into Fabric and the OneLake.

The Data Mesh

What is the Data Mesh? I’ll give a brief overview here, but for a more in-depth discussion check out the blog of Paul Andrew for his series of posts on applying the data mesh architecture in Azure.

If you recall the old days of the data warehouse, you had a number of datamarts in it. Basically databases that served a specific use case, or organisational functions. If you needed to you could run cross database queries to mash up data.

With the move to the cloud, this is now Resource Groups, with dedicated items for that specific group. This can allow each of those groups be managed and developed independently and now have data as a product, and also allow data to be accessed across these resources using for example SQL Endpoints.

This data mesh approach, is quite tricky to implement correctly. You have to ensure that all the services are deployed, and secured correctly between the items in the resource groups, and between the resource groups themselves. You also have to ensure that the development approach is consistent across all the departments that are now doing their own thing. There are many other things you need to consider, but now, you have in Fabric, a massive reduction in the implementation, it is now just a turn key service. Data domains are now Workspaces, that can flow all the way from the OneLake to Power BI.

They called me… Mister Glass

Azure Synapse, was an attempt to bring together the Data Lake, SQL Database, Spark and Pipeline development all into one place, and for the most part it worked, there are some deployment issues using Azure Dev-ops, but it works. Power BI itself was also an attempt to bring together Power Query, Analysis Services and Reporting into one experience. Now this single plane of glass approach is now fully realised with Fabric. You have all the services, and the workspaces, and the reporting all under one service. Your user logs in and away they go, if they have the required permissions of course. All the data is in OneLake, ready for what ever compute type you need. SQL, sure, serverless and ready to go when you run the query. AI & ML, not a problem. Dynamic or paginated reporting, click here.

The Future

Fabric is currently in preview, but if Microsoft update it as often as Power BI, this service will grow rapidly in terms of users and functionality. The growth rate and engagement of Power BI has influenced this move to an improved data and analytics experience. Users have, and will always demand improved experiences, and the data skills that are needed to drive todays organisations can be used in one place. Fabric has reduced the complexity of learning on how to query and create items. It is one experience, not having to develop in one service then changing to another. Now self service users can build the data pyramid from the bottom up. It will be interesting to see how users and organisations embrace it over the next year, when all the hype settles down. Try it out here.