At Ricoh Digital Services (AKA the day job) we do a lot of Data & Analytics projects for a diverse range of customers. When the project is completed and handed over, we can let the customer run it, or we can support it. What we don’t get involved with is the changes that it can bring to an organisation. Sure, during the sales process, we talk about the single source of the truth, the outcomes, what the customer wants, but not how they use data internally. We assume that the Data Warehouse is going to be used, but what we don’t do is see, is if the organisation is using it effectively, we assume that it will be. However, what we see is a number of organisations reacting to information, rather that being lead by it. So here are some ideas I’ve gathered on turning that around, and moving from a reactive driven to a more proactive data driven and some of the benefits that it should bring.

The Problem

The above image is a picture of a hand loom, weaving thread into cloth by hand, a laborious time intensive manual process, prone to mistakes. Replace that hand loom with Excel, and that is a chuck of your organisation today. That is the problem, there is normally a small cottage industry of Excel users creating reports, running teams, departments, entire companies, and COVID-19 track and trace databases using Excel. In the wrong, and even the right hands, Excel can be dangerous. Did you know that Excel played its part in a $6 Billion trading loss at JP Morgan as one of the copied and pasted quantitative models had some basic Excel issues, as noted in the internal investigation:

“After subtracting the old rate from the new rate, the spreadsheet divided by their sum instead of their average, as the modeler had intended. This error likely had the effect of muting volatility by a factor of two and of lowering the VaR . . .”

task force’s report (shareholder.com)

For more Excel horror stories have a look at the European Spreadsheet Risks Interest Group that lists some of these blunders.

Excel is the most used business intelligence product on the planet and it has some pros and cons:

Pro: You can use it to do anything

Cons: You can use it to do anything

Just to show you can do anything with it, he’s some art work created in Excel using the vector graphics tools in it.

The usage of Excel has grown like crazy, but why? Partly with the age of the internet revolution and ‘disruption’, and with the advances that swept away the monolithic applications like the good old Enterprise resource planning (ERP) system. These big beasts have been hunted down, by more flexible and agile sets of next generation applications and platforms. These have improved functionality, service levels, and made the workforce mobile. But there has been another level of disruption, the lack of a clear picture of your organisation, as you cannot join up the services effectively. In most large companies (Or at least the ones I have come across) There will be a number of information silos, under departmental dominion, owned by sections of the business. Some of the worst I have come across the data owners refuse to allow access, or to share the data that they have. 

It is with these disparate systems and other information silos were Excel thrives. Extracting data from one, mashing it up with data from another. I would suggest that these data silos and Excel patterns, help to destroy innovation, prevent understanding and more importantly increase friction and reduce efficiency of the organisation. 

It is time to move beyond Excel, to value the data you have. To do this will require a change of mindset. Here are five steps to point you in that direction, to create a data driven culture that delivers value, information and insight.

Step One – A single source of the truth

Where is the operations data for last month?
Where can we get that data from?
That’s got to be the wrong figure, the other analysts shows a higher value, should ours be higher?

If the three questions above are similar or the same you ask every day. Your organisation has a problem.

The single source of the truth should be the central, managed and cleansed source of information for your business. If you do not have that source of truth, you will have people extracting data from systems and producing different values (See above comments on Excel above). This will create arguments, and a they-said-we-said scenario, with each person defending their data and protecting their reputation. A long time ago, in a company I shall not name, I was in a meeting were two very senior managers nearly came to physical blows over data, and some serious number issues they had with each others reports. Long story short, both their numbers where wrong. The business ended up using my figure!

Is more time spent discussing the data, than acting on it? Are some people using old stale data which is low quality and full of gaps, when you should have access to a better source. 

It is time to bring into your single source the data from the diverse applications and tools that you have, as when you have a single source of the truth, you add value to the data consumers, analysts and decision makers. Time is saved by not having to hunt for or request access to the data. The organisation moves from sunken time trying to find the data, to actually using it. The single source of the truth should be documented, defined, joined, cleaned and add a rich context to the areas that it covers. You move from data to actionable information. A single source can be a Data Warehouse or Data Lake or just a Database, it is not about technology, it is the approach and vision. It should be capable and flexible to answer the organisations questions. With it you move from hindsight seeing what happened, to insight, seeing why it happened. You can be proactive, not reactive to how you operate.

As much as it is the source is the truth in your organisation, it is the trust in the data it brings. When it is trusted, you can do more with it.

Step Two – Data Dictionary

Once you have set up that source of the truth, you need to tell people what is in it, and what the logic behind those metrics means. This does not mean a long list of formulas and functions. It should explain what is the source of the data, who owns it, what is it used for, as well as the understanding of the filters and formulas. With out this step, if you do not have a clear list of what is data is available, and how it works, people will make assumptions and question if is it right, then go looking for it elsewhere.

The data dictionary needs to be clear with defined and agreed definitions. This will require the buy-in of your key stakeholders and subject matter experts, so there is no working around with other reporting sources with the secret formula of a certain calculation that they have.

Once the core definitions have been have agreed on, you need to move to the edge cases. It is these edge cases were you may need a slightly different metric to the main one to capture and understand different perspectives and context.

One issue that many fall into is making the data dictionary too techincal. It should be aimed at it being read standards users, not technical experts or programmers, so always go for the following:

• Clear well chosen names of what the measure/metric/formula is or ambiguous definitions that may cause confusion
• Be descriptive in naming, have ‘Transactions Year to Date’ not ‘TR YTD’
• No strange acronyms in naming metrics ‘NIMROD Metrics’ I’m looking at you another company I can’t name

People have to come to use it, trust it and see it as invaluable to understand the origin and context of the data. Its aim is to stop questions being asked all the time of the where and why of the data. In an organisation it is estimated a business analysts time is divided into, 80% finding data and answering questions from the organisation with the other 20% trying to discover insights.

That 80/20 split should be reversed. More value is added to an organisation finding insight, that it is fielding questions of how do I get this. This data dictionary is your map to your data, it is time to let people explore it.

Step Three — Open Up Access To The Data

You have the single source of the truth. You have the trust in the metrics. The next step is to open it up to people.

Your data is ineffective if it is not being used. You will be replacing multiple silos with one huge one. To create a data driven culture there needs to be an inclusive approach to the data. This does not mean complete unrestricted access to it. This means understanding the needs of the users and roles that they have.

Data should be used and played with. Small side projects about data can transform the understanding of the business. Instagram, Slack, Groupon, Twitter, The Post it Note, are all examples of side projects that have changed the world. Your users only have to figure out one small thing that can save a company that works at scale significant money. Like Robert Crandall did. You may not know who he is, but I’m certain that his eye for detail may have influenced your business thinking. Who is he? He is manager who figured out that American Airlines could save $40,000 per year. How? By removing one olive from every salad served to passengers. How did he do it? He had access to the data.

In most organisations there is often an overhead in getting access to data due to auditing, governance, regulation, and internal politics. Implemented badly all they cause is friction for the user to get the data, sometimes to the point they think it is not worth asking for it. Agility in an organisation is critical, if it takes to long to do something the advantage is lost. Layers of bureaucracy and formality should be removed, and access should be quick. Review and understand the layer, then demolish any walls between the users and the data that you do not need. Advertise it to your users, we are now open for all your data needs!

Step Four — Data Literacy

Once users get the data, do they know how to analyse it?
What tools do they use?
How can they learn?

Excel has been called the 2nd best tool to use in most situations, maybe its time to show people to use something else. This stage requires people to learn and for experts and power users to help teach and guide new people through it. We have the worlds information at our fingertips, let us use to it build something useful. There are millions of training videos, blogs, books out there to help different sort of learning patterns. Help curate a standard set of training videos or use training sites like Pluralsight to help people learn new technology and ideas. With the disruption that automation will bring, the expectation is that people will require retraining. It is time to skill them up, by adding value to the organisation and to the people.

In your organisation you have to promote the idea that understanding this is to the advantage of the organisation and to the people themselves. Organise workshops and drop in sessions to help users get to grips and help answer questions. Outside the organisation there are technology user groups that run free of charge, for people to get involved.

When your experts help to answer the questions of the new users, you also get an understanding of what the organisation is looking for, and see what points of action you need to take. You see the problems that the user is facing, is it the data, the process, the pattern of inefficacy of ‘well we have always done it this way’, that you need to overcome.

Once users have the data in front of them you need the last mile of data literacy, and that is how to show it off. Data presentation is the tip of the iceberg the people see. You can have the worlds greatest single source of the truth, with experts in statistics, but it will fail if you can not articulate or show the use of it. Why should you use one chart type and not the another? How do you present the answer so people can see the insight? You need to think about the data and reports and the context in which they sit, and show the narrative of what is going on in your organisation. Carefully designing reports will be make information digestible, interesting and engaging. If it shows insight by choosing the right way to present it, then it will be used.

Step Five — Use the information to make decisions

Lets talk about the elephant in the room… well first it is not an elephant, its a HIPPO. HIghest Paid Persons Opinion. The data points one way, the Hippo’s gut feeling is the opposite. Opinion based decisions are based on poorly understood metrics or pure guesswork, and are coupled with the baggage of preconceived notions and bias. Data cuts through this to find the truth, or to show you what the truth is. Ultimately your data will only make an impact if it is used to make decisions. Opinions are the opposite of being data driven. The delivered insight needs to be seen that it is being used, that actions have been taken from it, so those that have built it, questioned it, used it, and showed the value it, can that the struggle has been worth it.

In the other four steps, we have outlined what needs to be done to drive you towards a data driven culture, however with out the buy-in of those that make the decisions, it will be all for nothing.

Starting a data driven culture will be harder that maintaining it. The change will not happen overnight, you need to understand the path before you. Find a use case as a test bed, research and prove the value in it, once you have confidence and certainty, then scale.

Start small, to deliver big.

Rounding it all up

The best start to becoming a data driven culture, is wanting to become one, the desire to drive you and your organisation forward is the start. The next step as mentioned can be small, trying to understand where all the data is coming from, and demanding that ‘we’ve always done it this way’ is removed from the company. As typically when someone says that, there is a better way of doing it. I hope it has outlined some of the steps and ideas you need to start to drive a data culture.

Further Reading

How to best present your visuals — storytelling with data

Creating a Data Dictionary — Data Dictionary: a how to and best practices | by Carl Anderson | Medium

How to manage a Hippo —  Data-Driven Decision Making: Beware Of The HIPPO Effect! (forbes.com)

Credits

All images from the https://unsplash.com/ website

Nine Koepfer, Priscilla Du Preez, Richard Balog, Romain Vignes, Xuan Nguyen, Marek Piwnicki, thanks for your images!