You need reliable information to make decisions about risk and business outcomes. Often this information is so valuable that you may seek to purchase it directly through 3rd-party data providers to augment your internal data.

Yet, how often do you consider the impact of data quality on your decisions? Poor information can have detrimental effects on business decisions, and consequently, your business performance, innovation, and competitiveness. In fact, according to Gartner, 40% of all business initiatives fail to achieve their targeted benefits as a result of poor data quality.

Organizations that treat information as a corporate asset should engage in…

The advantage of using data to drive innovation and business results is often discussed, with mounds of evidence backing up its efficacy. However, data by itself has 0 value. It is people that dissect data (digital or analog) to extract the information required to solve tough problems. It makes sense, then, to talk about how the human component of our systems affects the quality of our solutions.

Among the human factors that lead to extraordinary performance, team diversity has been one of the least understood and highly under-utilized levers at our disposal.

I’m not writing this post to convince you…

Metrics & Measurement

One of the most precious pieces of information you can have to run your business effectively is to understand the revenue being generated as a direct result of some business activity.

Knowing which efforts are paying off allows you to double down and scale them up, while understanding where time and money are being wasted allows you to get more done by doing less.

This is the problem of attribution.

Marketers deal with this issue head-on because they must spend limited resources on multiple channels of influence. Prospects have many touch-points with the business during their customer journey. …

Metrics & Measurement

Conversion rates, or the percentage of people taking some desired action in your business, are among the most fundamental metrics you need to be aware of.

Track the conversion rates over a series of steps, and you have yourself a (conversion) funnel.

Looking at each step’s conversion rates can be critical for identifying bottlenecks and identifying issues with steps in your process; moreover, looking at the funnel as a whole gives you a mathematical engine that tells you how many outputs you can expect to derive from inputs into your process.

The latter is very useful when you know how…

Metrics & Measurement

Too often in business, we ignore measurements with small sample sizes because they are not “statistically significant,” but rarely, if ever, do we actually do the math to support that decision!

The fact is that it is precisely when we are highly uncertain about something that any data will greatly reduce our uncertainty.

I’m going to play out 2 scenarios from a fantastic book called “How to Measure Anything” by Douglas Hubbard so you can experience the “Aha!” moment for yourself.

By the end of this blog post, you’ll have an understanding of situations where small data can tell you…

Data Engineering

When I first started building the Data Vault at Georgian, I couldn’t decide what column data type to use as my tables’ primary key.

I had heard that integer joins vastly outperform string joins, and I was worried about degrading join performance as our data grew.

In the SQL databases of the operational world, this decision is pretty much made for you by giving you auto-incrementing integer primary keys out of the box.

However, in the Data Warehousing world, whether you’re building a Kimball or Data Vault or something else, you need to make this choice explicitly.

You can generate…

This article was originally published on the Show Me The Data blog and has been re-formatted for Medium.

If you’ve ever tried to build an enterprise data warehouse using BigQuery, you’ll quickly realize that the auto-incrementing primary keys you were so fond of in operational databases are not a thing in BigQuery.

This is by design. It’s not the job of your data warehouse to ensure referential integrity as your operational databases do, but rather to store and process tons of historical data.

However, this shift in perspective does not make the problem of finding a way to represent unique entities go away. …

Measurement & Metrics

There is a class of metrics where you simply want to know “How many of thing X are there in total?” but it is utterly impractical to actually count them because there are too many of them to count, or the space over which you’re measuring is too sparse.

Consider the following measurements:

• Number of homeless people in a city
• Number of endangered species in the wild
• Number of illicit drug users

There’s no doubt that quantifying these values is useful in understanding the scope of a problem and devising an effective plan to deal with it.

These questions can…