I have known many smart guys in my life, but John Holland, who died last month, was the man who most influenced my career. John was part of a small group of famous psychologists (John Flannagan, J. P. Guilford, Robert Thorndike, Harrison Gough, Hans Eysenck, Alphonse Chapanis, and B.F. Skinner) whose interest in applied psychology and assessment was sparked by their experiences in World War II. If you have ever seen Warren Buffett interviewed on TV, you will have a good sense of John, who was also from Omaha. He was mild mannered, unpretentious, overtly modest, wickedly funny, ferociously competitive, and very hard working—he was one of the five most published scientists of the 20th century. He was a keen football fan (the Kansas City Chiefs), a fine pianist, and catnip for women—his wife Elsie (also from Omaha) could have had a career in the movies. And he created intense admiration and loyalty in everyone who worked with him because he was so smart, so perceptive, so principled, and so very funny. The same seems to be true for Warren Buffett.I learned five lessons from John that are worth repeating. The first concerns the importance of assessment for guiding peoples’ lives. As Freud said, the two biggest problems in life concern choosing a mate and choosing an occupation and people never do either for rational reasons—and he was right. The process of choosing a mate—male/female relations—is bound up in biology and hormones and is utterly irrational. The process of choosing an occupation is bound up with one’s relationship with one’s parents; nonetheless, it is in principle possible to make rational career choices and the key to doing that is valid assessment and competent feedback. The second lesson that I learned is utter disdain for the rules of the psychometric (or academic testing) establishment—e.g., what we learned in graduate school. One of John’s best lines was, “Forget everything you learned in graduate school”; he meant it and he was right. Mainstream psychometrics concerns measuring entities (i.e., determining “true scores”). But applied assessment has a job to do, and that is to predict outcomes. The psychometric establishment is only concerned in principle with how their methods apply to real world; in reality, they don’t care. Real test development is an intellectual and scientific activity that requires careful thought and some creativity; there are no formulae for good test development, there are no cook book recipes for developing meaningful assessment, and the best tests have been developed by mavericks.The third lesson I learned was that it is essential to have a sound, well thought out, conceptual basis for your measurement model. John was an avid student of philosophy of science. He was greatly concerned about, and immensely pleased with, the way in which his measurement model was theory-based and his research was consistent with the best precepts of real science: have an idea, test the idea, refine the idea based on the test, then start over. This lesson is quite unique and none of our competitors understand it. The fourth lesson I learned is that if you develop your assessments correctly, they will sell. John made a lot of money from his Vocational Preference Inventory and his Self-Directed Search. The same is true for Raymond Cattell and Harrison Gough. At some point, the market can distinguish between valid assessment and psychometric garbage. Finally, I learned from all of these guys (Holland, Cattell, Gough, etc.) that you have to retain control of your intellectual property. They all found it difficult to manage the test sales by themselves (John’s garage was stacked full of paper products), they sold their rights to “real” business people, and spent the rest of their lives bitterly regretting their decision. In 1982, Joyce and I sold the rights of the HPI to National Computer Systems. They were as incompetent as all big public companies seem to be; it was a terrible decision, and when we got the rights to the HPI back in 1992, we began making money immediately, and we vowed never to make that mistake again. It is important to remember John’s approach to assessment because, after the wind blows away all the psychometric schlock, his work will still be standing. There is a strong tendency in our discipline to forget the past, to assume that the research that is going on today is qualitatively better and has somehow superseded the earlier research. That, of course, has to do with the vanity and arrogance of youth. Virtually all of us could profitably reread the early work of Binet, Spearman, Strong, Allport, Murray, and of course, John Holland. And when we do, we will be surprised to discover how little progress has been made in the fundamentals of assessment over the past 100 years.
What is the Most Significant Change Coming for the HR Profession?
The following is a column by Dr. Robert Hogan, that recently appeared in the "2009 Forecast" edition of Human Resources Executive. Dr. Hogan was asked to comment on what he sees as the most significant change affecting the HR community in the future.
By now, everyone is aware of the coming demographic tsunami which will be defined by the retirement of the baby boomer generation. On the one hand, this means that a lot of talent and institutional memory will walk out the doors. On the other hand, the replacement pool—composed of the young, the inexperienced, and the untried—will grow steadily smaller. The generic answer to dealing with this looming problem is called talent retention, and a number of talent retention models are available for commercial consumption.
Talent retention can be broken down into two generic strategies. The first concerns how to retain older workers past their normal retirement date. The second strategy concerns how to attract and retain talented replacements for the retirees. Standard talent retention solutions involve special training, on boarding, compensation, and career pathing packages, all of which are sensible structural solutions. However, what is missing from most talent retention packages is a careful consideration of the human factor. The critical insight comes from the Gallup research, which shows quite clearly that people don’t quit organizations, they quit their immediate bosses. Unless and until talent retention programs take this crucial generalization into account, they will not be effective tools in the coming war for talent.
Research data gathered over the past three decades clearly indicate three conclusions.
Read More »
George Bernard Shaw and the Concept of Faking It
Dr. Hogan Interviewed on The Doug Noll Show
Doug Noll interviewed Robert Hogan about the dark side of leadership on Thursday, October 16. Links to the audio segments appear below.
Excerpted from the Doug Noll Show's website:
Defective and dark leadership is the single most pressing problem facing humanity. In Corporate America, over 65 percent of the managers and leaders are incompetent, defective, or badly flawed. A higher percentage exists in government. The costs are staggering and one only has to look at the financial market melt down of the past months to understand the enormity of the problem.
Doug and Robert begin by understanding leadership through the lens of evolutionary psychology. Leadership evolved in humans as a way to come together for a short time to accomplish a common goal. Thus, humans became hard wired genetically to form social, hierarchal groups with leaders in charge. The most effective leaders were humble, supported the group and its goals, and was not self-aggrandizing. With the development of agriculture about 12,000 years ago, Robert describes the rise of the kleptocracy, which persists today. This is a class of leaders that rose to high status through power grabs, political maneuvering, technical competence, and raw luck. Once high leadership status was achieved, this class ahd no difficulty stealing from the groups it was leading. Leadership, as Robert sees it, is the ability to build and maintain a high performing team. Over time, this team will compete well against other like-minded teams.
Segment 1
Segment 2
Segment 3
Segment 4
Personality and Financial Management
The Shoe Fits
In this cover story from Canadian publication Advisor's Edge, Dr. Robert Hogan discusses the validity of personality assessment in the selection process, as well as his pioneering role in the history of personality testing.
Dr. Robert Hogan, an international authority on personality assessment, recalls facing stiff resistance from academics and lawyers when he and his wife pioneered personality testing in the United States in the early ’70s. “The furor was like Galileo saying the earth revolves around the sun. It was a big career risk.”
Read the full text of the article by downloading the PDF here.
Validity
For test developers and test users, validity is the most fundamental concept in psychological assessment. It is also a surprisingly vexed notion. A review of the literature on validity composed by the “great minds” (e.g., Lee Cronbach, Jane Loevinger, Paul Meehl) will give you a case of vertigo. The definition of validity found in the AERA Standards for Educational and Psychological Testing is a statement by a committee—in the same way that a camel is a horse designed by a committee.
The confusion about validity is the result of the way psychological measurement was conceptualized at the outset, beginning with Charles Spearman’s research on intelligence. Spearman taught at a private boys’ school; he noticed that his students’ scores on their various academic examinations were correlated. Based on this, he derived two conclusions. First, he proposed that there was a single, general factor underlying performance on all the exams. And second, he proposed that that factor was (or reflected) “intelligence”.
Spearman set the framework and the terms of the discussion for all subsequent assessment research. The framework consists of two assumptions that follow from his two conclusions. Since Spearman’s time, virtually all psychological assessment has been based on these assumptions, neither of which is necessary or necessarily true. The first assumption is that individual differences in human performance depend on, or are related to, or reflect individual differences in the strength or magnitude of a corresponding underlying trait or propensity. The second assumption is that the goal of assessment is to measure individual differences in the strength or magnitude of the underlying trait. In this view, the goal of assessment is to measure traits. This view is never questioned, and it has implications for understanding validity. In this standard model, a test or measure is valid to the degree that it accurately measures the underlying trait.
But this view of validity makes it impossible ever to determine validity. The problem is that the actual existence of traits is questionable on genetic and neuropsychological grounds. There are no anatomical or neurological structures corresponding to any of the many traits that have been proposed. Consequently, it is by definition impossible to determine whether a test accurately measures an underlying trait—because the existence of trait is in doubt.
The confusion about the meaning of validity can be resolved fairly easily with a simplifying assumption. If we assume that the goal of assessment is to predict outcomes, then validity can be defined in terms of the ability to predict those outcomes. A measure of sales performance is valid if it predicts sales performance; a measure of customer service potential is valid if it predicts ratings of customer service, and so on. This view makes no assumptions about the existence of underlying traits the strength of which causes performance. It simply stipulates that assessment has a job to do, and that job is to predict non-test outcomes. This definition of validity satisfies the requirement of Occam’s razor, which states that one ought not multiply causal entities unnecessarily; the definition also satisfies the aesthetic of the Bauhaus movement which states that less is more.
Flawed Leadership: Is it Pressure or Personality?
The Secret Life of Organizations
The HDS: Which Animal(s) Are You?
We've enjoyed a lot of positive feedback lately regarding our Hogan Development Survey marketing campaign, which features a number of different animals that "personify" the 11 derailer scales measured by the assessment. These potentially-derailing characteristics aren't necessarily bad (in fact, they can be very *good* depending on your line of work), so we developed a method of presenting them that makes the concept a bit more accessible, and actually injects some fun into the realm of "strategic self awareness."
From the print ads associated with this campaign:
Every leader has personality traits that threaten his or her success. The Hogan Development Survey is the only business-related assessment that measures performance risks that impede work relationships, hinder productivity and limit overall career potential. The HDS provides valuable feedback for strategic self-awareness, which is the key to overcoming these tendencies and achieving success in the workplace.
We discuss the HDS in depth on our website, so be sure to read more about it and tell us... Which animal are you?