3 Steps for a Data-Driven Operation
First you measure, then you analyse, finally you automate. The three simple steps to bring your operation into the digital age.
Author: Daniel Stojanov | Published: 29 September 2019 29 September 2019 | Permalink
Many of the important decisions in your job, your business, or your other operation could probably be made perfectly and automatically without much or any direct human input. This idea may seem idealistic, but many business problems fall into a small number of optimisation tasks that have been well studied and have known solutions. To get there, a data-driven operation must measure by capturing available data, analyse that data to create models of how the operation behaves, and finally create solutions that are precriptive: algorithms and solutions that instructs on the decision to make which achieves the best possible result given available information.
A data-driven operation
Making a decision without data is like navigating a dark, unlit room. Even once you measure and obtain data there is a world of difference between a proven rational procedure to determine the perfect or highly optimised course of action, against the alternative of having to exercise judgement in decision making. Leaders of operations that are not data-driven will often resort to superstitions (often masked by calling them experience), guess-work, or rest at the mercy of random chance.
When decisions have important consequences they may be stressful. Have you made the right choice? Have you missed something? What should you be thinking about that you have not realised? How do you know if you are right about this? If the processes you used was irrational, that the rest of your emotions surrounding this decision be similarly irrational, only makes sense. If your problem has known optimised solutions and you followed the known, studied, provably optimum solution to your problem; then, there is no rational reason for concern. You took the correct decision arrived at by the best known procedure to solve your particular problem. The world is uncertain, but if you followed the correct procedure there was nothing else you could, or should, have done.
As previously discussed, many business operations and decisions are actually well understood and have known and perfect solutions. Solutions to optimisation problems had been proposed over recent centuries, but operations research as a discipline grew to wide use by the military during the second world war. As well as some of the more novel applications previously discussed—including sports strategies/player recruitment and fashion design—operations research can solve a range of problems throughout industry. These include planning problems, such as staff schedules; resource allocation (this might include the most profitable subset of jobs to run in a factory, the optimum order in which to run jobs, etc.); designing efficient and robust networks: including electrical, logistic, or communication networks; all the way through to the best way in which to stack palettes; and, many other applications.
Data is indispensable in making decisions. Although not enough alone to provide solutions, it is absolutely necessary to capture the data that will be used to calculate optimised solutions.
Anybody not measuring and capturing all of the information around their process is not making the best effort to inform their decision making or to optimise their operation. Data storage is essentially free. There is zero cost to ignore captured data, but once the window has closed to measure a given component or variable in your operation, that uncaptured data can never be accessed. The current Internet of Things (IoT) buzz reflects the present reality that computer hardware is very cheap. Deploying hardware to measure data around a factory, an office, a warehouse, is not an expensive task.
To be useful data must be digitised and structured in a way that makes it easily programmatically accessible. This often means storage in a database, spreadsheets, or other format that makes sense for a given application. This means that data stored only on paper can almost be considered to not exist.
The importance of creating digitally stored and accessible data is illustrated in this negative case. The US congress has long been noted as a hotbed of insider trading. For members of congress trading on insider information—something illegal and likely to attract long jail sentences for regular citizens—is legal, albeit perhaps an awkward topic for discussion. Representatives are required to annually report their personal investments… but records are kept in a paper-only system accessed by appointment from the Cannon House Office Building, in Washington DC. The paper-based system meets the minimum requirements the congress set for itself, while achieving the dual aim of essentially keeping the information inaccessible to outsiders.
Once captured, the next step is to analyse data. This helps to close the loop, but is only just one step above random guesses in the dark: perhaps random guesses in a dimly-lit room.
Data analysis—the transformation of data records into processed information, visualisations, and predictive analytics—certainly aids in running an operation and in making decisions. Good analytics gives you a picture into what is happening in your operation, it may even provide you alerts, graphics, and great visuals; but, the question still remains: so what!
Suppose you have the most accurate forecasts, that you know exactly the number of orders and of what type you will receive over the next period. Do you know precisely which jobs you need to accept to maximise your profits? Suppose you know which jobs you are going to work on, do you know the sequence in which to complete those jobs to maximise efficiency on your staff and equipment?
To answer the above questions you need to interpret the processed data. Although charts and visualisations help in this way people still have biases and are imperfect when reading reports. Bar charts, pie charts, box plots, etc. are useful in showing proportions, but our interpretation is at best an approximation. Illusions, errors in judgement, and other biases further complicate how we can perceive information. More importantly, once information is absorbed, what is the process to find the best solution to a decision? It makes little sense to meander to a solution, perhaps learning and getting closer over time, or developing heuristics (possibly superstitions) about the best action to take. If a problem has a known solution there should be no need for experience. You should not be learning and getting better over time if your problem falls into one of the categories of problems with known solutions. It is often misunderstood that there is a wide range of problems, from across industry and the economy, which have known methods to find optimal or near optimal solutions.
Reports and forecasts are just the next step, but still short of the best solution. Forecasts and processed data should instead provide inputs into a known optimisation method that then finds the optimal solution.
The end goal for a data-driven operation is automated and optimised decision making. Analysis helps people make sense of data but that does not close the gap to the end of the entire decision-making process. It still leaves the person with the task of interpretation and exercising judgement.
An optimised data-driven solution in a successful operation will likely feature two components: a model to simulate the system being optimised and the method to find the optimised solution. The model can be a set of equations describing the system, a geometric and mechanical computer model in the case of mechanical optimisation problems, or a simple spreadsheet. There are a range of well studied methods for solving optimisation problems.
As an example, solutions to linear programming problems—a very common category of optimisation problem that appears throughout industry—can be found for cases with millions of parameters using modern desktop computing hardware. Whether it be creating mixtures of chemicals for industrial use, preparing a diet/meal plan, selecting jobs to run in a factory, it is in each case the same mathematical problem.
It is likely that even today businesses are scheduling staff, planning operations, and making decisions without the best data, modelling, and solution methods to their problems. Those businesses that embrace a data-driven operation have the advantage of making perfect decisions and removing human guesswork, superstition, and errors in judgement from their decision making. To get there an operation can consider and implement procedures for each of the three steps described above: measure, analyse, and optimise.