Data can be a valuable tool for driving business growth and profitability. Being data driven can improve the understanding of various aspects of the business and better equip companies for success. Data can play a crucial role regardless of a company’s size. Although larger companies have more resources for hiring data professionals and obtaining the most sophisticated technology, smaller businesses can also benefit considerably from basing decisions on data.

As I’ve written previously, how a company interprets data and extracts meaning from it is a vital element in how much success the data can bring. Adept use of data helps improve business results. Nonetheless, companies do not necessarily use data as well as they could. They can be vulnerable to data misinterpretation. So, they must do what they can to guard against it.

This brings to light an example in recent media that illustrates the potential for data misinterpretation. The March 28, 2022 issue of the Wall Street Journal featured a review of the book “The Voltage Effect” by John A. List. Based upon the review, “The Voltage Effect“ appears to be a book that provides very useful insights to consider when evaluating whether or not a new business has good potential for successful growth. According to the review, the book suggests questions to ask for determining whether or not a business concept has the potential to scale up. As I see it, these questions are an excellent step to take when evaluating the potential to scale.

However, the review says, “ ‘The Voltage Effect’ is a fine business book, though in many ways it works better as a meditation on the shortcomings of our increasingly data driven world.” The review goes on to say, “Mr. List seemingly argues that good and helpful data analysis may not scale well.” The review points out that what works based on data collected in Kenya, may not necessarily work well in California. The implication is that a successful business concept based on data collected in one location won’t scale because what works there may not work elsewhere.

As I see it, especially for readers without a strong sense of what to do with data, the review can easily be misinterpreted. It can leave readers with the impression that data analysis cannot scale. It is true that data collected in one place may not apply elsewhere. But, data analysis can scale when used correctly.

A good example is how Amazon uses data to recommend additional items that customers might like. Granted, Amazon’s recommendations may not always work perfectly. After all, some people who buy a product may be very different from the typical purchaser of that product. So these non-typical buyers may not have even the slightest interest in what Amazon recommends. But, for the most part, Amazon’s recommendations are a reasonable fit with customers’ interests, even if the customer chooses not to buy the additional items. Thus, Amazon’s data analysis does scale and Amazon is doing this at a high volume.

So, rather than thinking that data analysis doesn’t scale, it is important to remember a basic principle that applies to all data analysis, whether it’s customer purchase data, questionnaire surveys, AI (artificial intelligence), focus groups, randomized controlled trials, or a simple count of something that did or did not happen. No matter what data collection and analysis methods are used, conclusions only apply to groups that are like the group from which the data is collected. If data is collected in Kenya, it may not apply in California. If data is collected from white males, it may not apply to females or blacks. This is not an issue of whether data analysis can scale, but it is an issue of where the data can apply. For example, data on white males can scale to a much larger number of white males because that’s where the data applies.

In research methodology, this principle is called validity. Research has validity if it measures what it is supposed to measure. If data is collected from one group, but information about some other group is what you really wanted to know, then the research lacks validity because it didn’t measure what it was supposed to measure.

These days, there are vast quantities of data available as well as sophisticated technology for analysis. Yet, conclusions based upon the most sophisticated techniques may not be valid if the data is being collected for some group or place that differs from the group or place you want to know about. Without validity, there may be inadequate information for drawing the desired kind of conclusions. This isn’t because data analysis doesn’t scale. It’s due to not collecting the right data, since what was collected describes some group or place that doesn’t apply to what you wanted to know.

So, in conclusion, remember that data analysis can scale. But, data validity is required for drawing appropriate conclusions. It’s important for the data to measure what it is you want to measure for those groups that you want to know about. Don’t fall into the misinterpretation trap.