But too often, organizations only have a loose understanding of the thing they want to measure. Think about it: what are the measurable criteria for “employee engagement”? How would you measure “a motivated sales force”? Until we know how to define the things we’re planning to measure, we’re basically guessing.
In the last two years, I have worked on four projects in which the client has wanted to measure employee engagement. In each case, the client had an Employee Satisfaction survey which explicitly asked the employee if they felt engaged with their job, but they had no other criteria for defining an “engaged” employee. The clients were willing to accept that the self-report for engagement was accurate (not a great idea), but had no idea what went into an employee feeling “engaged”. We gathered a few benchmarks and found some I/O psych knowledge that provided possible frameworks. But the task of truly identifying the behaviors and influences which create engagement was entirely guesswork.
The Dangerous Promise of Data Mining
In each case, when I asked questions about engagement, the organizations showed an earnest and intense belief in its importance. They wanted their employees to be engaged, even if they didn’t know exactly what that meant, or how to measure it. And I’m not mocking that situation; it’s far more common – in and out of the workplace – than we probably care to admit. But if we’re going to show the impact of our efforts, we need to be able to define both the inputs and the results… and show the obvious correlational relationship between the two.
Unfortunately, all of these organizations had buoyed their hopes on the false promise of data mining. They believed that if they simply gathered enough data on learner behavior, it would eventually yield some insight. The data could be used to identify engaged employees (which – in the best instance – consisted of a few self-reported questions on an annual survey), and we could mine their behavioral data to find out what behavioral patterns lead to employee engagement. It’s not a foolish methodology, and one that they’ve certainly been sold. (I admit, I’ve been guilty of promoting it in at least one book in which I was a contributor.) But gathering random data with the goal of defining a broad concept is like buying an entire grocery store with the hope of creating a delicious recipe for dinner. Except data sets are rarely robust and clean enough to perform the kind of detailed analysis to find these results. To gather enough clean data can be a time-consuming and intensive effort. As far as methodologies go, these efforts usually outweigh the reward.
Gathering random data with the goal of defining a broad concept is like buying an entire grocery store with the hope of creating a delicious recipe for dinner.
The Power of Hypotheses
The most practical way to define a broad concept like engagement is through hypothesis. Hypothesis is often overlooked as a process because it feels so much like guesswork, but nothing could be further from the truth. Hypothesizing requires stakeholders to take an honest consideration of their organization and craft a measurable, testable definition. Hypothesis may begin from a place of statistical uncertainty, but it’s the opposite of guesswork. It’s a process of narrowing our possibilities down to a concise definition with less and less uncertainty.
A.D. Detrick is a strategy and measurement consultant, human capital analytics expert, project manager, instructional designer, and trainer. He's also a self-confessed comic book geek and a believer in using humor and humanity to teach complex concepts.
Request a Consultation
Contact us to discuss how MetriVerse can wield the power of data science to help your organization.