Putting People Into the Big Data Equation


Over the last few years we have witnessed the popular rise of Big Data. That is, organizations of all sizes and agendas are now looking to use large datasets to guide their strategies. Using data to grow and improve organizations makes complete sense as digital technology has permeated every corner of our society. The sheer size and quality of these new data sources has enabled us to build large statistical models that can predict and understand some of the greatest challenges facing mankind (i.e. climate change, the economy & democracy). Yet the question remains, what makes a good data analyst?

The ability to understand and manipulate Big Data in a way that can produce both meaningful and actionable insights for leaders is something that is not widely understood, nor discussed. It is well established what personality characteristics are predictive of success at work, but given the relative novelty of Big Data, do leaders know what skills and characteristics to look for when hiring a new data analyst, aside from technical knowledge and experience? Setting up data warehouses and systems is an expensive and costly process, therefore organizations must ensure that they are recruiting and developing the right talent. If the success of Big Data is equally dependent on technical and psychological factors, what then makes a good data analyst?

  1. They are motivated by Big Data: The psychologist John Holland found that our vocational interests are closely related to our personality and largely overlap with what motivates and drives us to seek out certain careers and strive for certain goals. Accordingly, there is evidence that demonstrates individuals who value logical and rational thinking are not only more likely to seek out jobs that encourage such a mentality, but are also more likely to succeed at them. In the context of Big Data, people with these values seek to answer questions using solid evidence and understanding. In essence they embody the Big Data ethos and uphold best scientific practice. This is crucial given that the technicalities of Big Data can easily produce false positives.
  1. They are curious: In light of the countless scientific papers describing the role of personality at work, its obvious that personality characteristics influence our behavior, and in turn, our success at work. Given this, the best data analysts are those who have a curious mind — they are always seeking out new information, viewing problems from different perspectives and questioning what is accepted knowledge. It is therefore no surprise that such individuals have higher levels of IQ and are more proficient with understanding mathematical problems. Given that one of the many selling points of Big Data is the identification of relationships across a variety of datasets and sources, ensuring your data analysts have this mindset will be a great strength as they will only be satisfied once their results are conclusive and accurate.
  1. They appreciate the devil in the details: Ask any programmer or analyst and they will tell you that modeling data requires absolute attention to detail, as simple mistakes can quickly turn into huge errors. Therefore, the best analysts are not only experienced; they are also prudent and judicious. In fact, evidence has demonstrated such psychological qualities to positively support the relationship between an individual’s Human Capital (i.e. education & experience) and job performance. Put plainly, the best analysts apply their knowledge carefully and pragmatically. Although this may produce a degree of rigidity, leaders are likely to find it a worthwhile trade-off when looking to build a team of reliable data-analysts.

It should now be evident that organizations looking to adopt Big Data into their operations and strategies need to take a minute to stop and think about what the job requires and whether they have the right talent. Unfortunately, the only stories you hear about Big Data are successful ones — you rarely hear about the large amount of resources invested into data warehousing that bring little to no value due to incompetent staff. The time has now come for leaders to stop thinking of Big Data as just a technical problem, and start to also view it as a people problem.

The key to any success is down to talent, personality and expertise, and that remains true for Big Data.