Blog

I have known many smart guys in my life, but John Holland, who died last month, was the man who most influenced my career. John was part of a small group of famous psychologists (John Flannagan, J. P. Guilford, Robert Thorndike, Harrison Gough, Hans Eysenck, Alphonse Chapanis, and B.F. Skinner) whose interest in applied psychology and assessment was sparked by their experiences in World War II. If you have ever seen Warren Buffett interviewed on TV, you will have a good sense of John, who was also from Omaha. He was mild mannered, unpretentious, overtly modest, wickedly funny, ferociously competitive, and very hard working—he was one of the five most published scientists of the 20th century. He was a keen football fan (the Kansas City Chiefs), a fine pianist, and catnip for women—his wife Elsie (also from Omaha) could have had a career in the movies. And he created intense admiration and loyalty in everyone who worked with him because he was so smart, so perceptive, so principled, and so very funny. The same seems to be true for Warren Buffett.I learned five lessons from John that are worth repeating. The first concerns the importance of assessment for guiding peoples’ lives. As Freud said, the two biggest problems in life concern choosing a mate and choosing an occupation and people never do either for rational reasons—and he was right. The process of choosing a mate—male/female relations—is bound up in biology and hormones and is utterly irrational. The process of choosing an occupation is bound up with one’s relationship with one’s parents; nonetheless, it is in principle possible to make rational career choices and the key to doing that is valid assessment and competent feedback. The second lesson that I learned is utter disdain for the rules of the psychometric (or academic testing) establishment—e.g., what we learned in graduate school. One of John’s best lines was, “Forget everything you learned in graduate school”; he meant it and he was right. Mainstream psychometrics concerns measuring entities (i.e., determining “true scores”). But applied assessment has a job to do, and that is to predict outcomes. The psychometric establishment is only concerned in principle with how their methods apply to real world; in reality, they don’t care. Real test development is an intellectual and scientific activity that requires careful thought and some creativity; there are no formulae for good test development, there are no cook book recipes for developing meaningful assessment, and the best tests have been developed by mavericks.The third lesson I learned was that it is essential to have a sound, well thought out, conceptual basis for your measurement model. John was an avid student of philosophy of science. He was greatly concerned about, and immensely pleased with, the way in which his measurement model was theory-based and his research was consistent with the best precepts of real science: have an idea, test the idea, refine the idea based on the test, then start over. This lesson is quite unique and none of our competitors understand it. The fourth lesson I learned is that if you develop your assessments correctly, they will sell. John made a lot of money from his Vocational Preference Inventory and his Self-Directed Search. The same is true for Raymond Cattell and Harrison Gough. At some point, the market can distinguish between valid assessment and psychometric garbage. Finally, I learned from all of these guys (Holland, Cattell, Gough, etc.) that you have to retain control of your intellectual property. They all found it difficult to manage the test sales by themselves (John’s garage was stacked full of paper products), they sold their rights to “real” business people, and spent the rest of their lives bitterly regretting their decision. In 1982, Joyce and I sold the rights of the HPI to National Computer Systems. They were as incompetent as all big public companies seem to be; it was a terrible decision, and when we got the rights to the HPI back in 1992, we began making money immediately, and we vowed never to make that mistake again. It is important to remember John’s approach to assessment because, after the wind blows away all the psychometric schlock, his work will still be standing. There is a strong tendency in our discipline to forget the past, to assume that the research that is going on today is qualitatively better and has somehow superseded the earlier research. That, of course, has to do with the vanity and arrogance of youth. Virtually all of us could profitably reread the early work of Binet, Spearman, Strong, Allport, Murray, and of course, John Holland. And when we do, we will be surprised to discover how little progress has been made in the fundamentals of assessment over the past 100 years.

George Bernard Shaw (1856-1950), the Irish-born playwright and novelist, was the only writer to win both the Nobel Prize for Literature and a movie Oscar. Shaw was an ardent socialist who, nonetheless, founded the London School of Economics (LSE)—a hotbed of British capitalist thinking. But most importantly, as an Irishman, Shaw was a gifted and intuitive psychologist, and his most famous play, Pygmalion, contains an important practical lesson for the critics of personality assessment.The standard criticism of personality assessment is that people can and do “fake” their responses to the items on personality inventories. The concept of faking is deeply problematical in social life. It rests on the assumption that there is a “real” you and that faking involves pretending to be someone other than the real you. In contrast, for social constructionists, there is no “real” you, there is just the you that you have chosen to be; you then use social interaction to tell other people who you are and how you would like to be treated. More specifically, there is the real you, which is the you that you were as a child—an unsocialized horror, then there is the you that, as an adult, you (more or less self-consciously) pretend to be in public—the socialized you. The maturation process involves learning to fake, to hide the real you. In this way, the answers that you provide on a personality questionnaire would reflect the socialized you, not the real you, the unsocialized horror—unless you avoided the maturation process.In Shaw’s play, the linguist Henry Higgins bets a friend that he can train a cockney flower girl to speak “proper” English and pass her off as English gentility. Higgins selects Eliza Doolittle, and with some effort, trains her to drop her normal way of speaking and, instead, speak “proper” English. Higgins is successful, Eliza Doolittle is transformed, Higgins falls in love with her, and the play ends with her rejecting Higgins and marrying an impoverished English gentleman.Shaw’s play contains at least three lessons for personality psychology. First, there is no real you, there is just the you that you pretend to be in public. Every social performance is an act. Only people who are na?ve, unsophisticated, or psychologically obtuse insist on being authentically themselves. But as the existentialist philosopher Jean Paul Sartre remarked, “Authenticity is the mark of a person who has been taken in by his own act.”Second, with some concentration and practice, most people can change their social performances; like Eliza Doolittle, they can learn to fake, except that they are not faking, they are just playing a different role from the one they usually play.And third, some roles lead to greater social acceptance, popularity, and success than others. People who insist on being rude, opinionated, and abrasive aren’t simply being themselves, they are behaving in a self-defeating manner.Returning finally to personality assessment, people present themselves differently using the items on personality questionnaires—this is the essence of individual differences—and some of these presentations will lead to better outcomes than others. But no one is faking when they respond; how they present themselves is a choice, whether it is conscious or not.

Doug Noll interviewed Robert Hogan about the dark side of leadership on Thursday, October 16. Links to the audio segments appear below.Excerpted from the Doug Noll Show's website:Defective and dark leadership is the single most pressing problem facing humanity. In Corporate America, over 65 percent of the managers and leaders are incompetent, defective, or badly flawed. A higher percentage exists in government. The costs are staggering and one only has to look at the financial market melt down of the past months to understand the enormity of the problem.Doug and Robert begin by understanding leadership through the lens of evolutionary psychology. Leadership evolved in humans as a way to come together for a short time to accomplish a common goal. Thus, humans became hard wired genetically to form social, hierarchal groups with leaders in charge. The most effective leaders were humble, supported the group and its goals, and was not self-aggrandizing. With the development of agriculture about 12,000 years ago, Robert describes the rise of the kleptocracy, which persists today. This is a class of leaders that rose to high status through power grabs, political maneuvering, technical competence, and raw luck. Once high leadership status was achieved, this class ahd no difficulty stealing from the groups it was leading. Leadership, as Robert sees it, is the ability to build and maintain a high performing team. Over time, this team will compete well against other like-minded teams.Segment 1Segment 2Segment 3Segment 4

People in the world of finance tend not to be very thoughtful about personality—and there are overwhelming data to support this generalization. But personality affects the decisions of financial managers just like everyone else. In addition, the success or failure of every organization is the sum of the decisions that the leaders make on a daily basis. Ultimately, then, the success of an organization depends to a significant degree on the personality of the leaders. Consider the case of Lehman Brothers, a 150 year old Wall Street investment bank with 25,000 employees world wide. It was the fourth largest investment bank in the world, and it declared bankruptcy on September 15th, 2008. The CEO of Lehman Brothers is Richard Fuld. Starting in the summer of 2007, Fuld made a series of decisions that ultimately doomed his company. As the subprime mortgage business slumped, Fuld decided that money could be made by investing as others retreated, so he “doubled down” on mortgage backed derivatives. As they continued to decline in value, so did Lehman’s holdings. But Fuld seemed not to notice. As Joe Nocera remarks in the New York Times (9/15/08), “And yet, even as they lowered the value of their mortgage-backed securities, firms like Lehman had still priced them too high.” David Einhorn, a hedge fund manager, described Lehman’s mark-to-market pricing as “dishonest”; Nocera called it wishful thinking.For at least a year, Lehman Brothers needed to sell parts of itself to remain solvent. Over the past several months, a series of interested buyers approached Lehman, but in each case, Fuld thought Lehman was worth more than the price offered. He maintained this position right into bankruptcy. Why did Richard Fuld make these decisions? What sort of person is he? The question is hard to answer because the only information we have is in the financial press, which is not insightful about psychology. Fuld received a BS from the University of Colorado in 1969 and an MBA from NYU in 1973. He joined Lehman Brothers in 1969 as a commercial paper trader, and Lehman is the only place he has ever worked. He rose rapidly, and has been the CEO since 1993. The Institutional Investor magazine named Fuld the #1 CEO in the industry in 2006. Fuld’s salary in 2007 was $51.7 million dollars and in March, 2008 he received a $22 million dollar bonus for 2007. Over the past five years, Lehman paid him $311.9 million dollars in compensation.Here are some other facts about Richard Fuld:

In this cover story from Canadian publication Advisor's Edge, Dr. Robert Hogan discusses the validity of personality assessment in the selection process, as well as his pioneering role in the history of personality testing.Dr. Robert Hogan, an international authority on personality assessment, recalls facing stiff resistance from academics and lawyers when he and his wife pioneered personality testing in the United States in the early ’70s. “The furor was like Galileo saying the earth revolves around the sun. It was a big career risk.”Read the full text of the article by downloading the PDF here.

For test developers and test users, validity is the most fundamental concept in psychological assessment. It is also a surprisingly vexed notion. A review of the literature on validity composed by the “great minds” (e.g., Lee Cronbach, Jane Loevinger, Paul Meehl) will give you a case of vertigo. The definition of validity found in the AERA Standards for Educational and Psychological Testing is a statement by a committee—in the same way that a camel is a horse designed by a committee.The confusion about validity is the result of the way psychological measurement was conceptualized at the outset, beginning with Charles Spearman’s research on intelligence. Spearman taught at a private boys’ school; he noticed that his students’ scores on their various academic examinations were correlated. Based on this, he derived two conclusions. First, he proposed that there was a single, general factor underlying performance on all the exams. And second, he proposed that that factor was (or reflected) “intelligence”. Spearman set the framework and the terms of the discussion for all subsequent assessment research. The framework consists of two assumptions that follow from his two conclusions. Since Spearman’s time, virtually all psychological assessment has been based on these assumptions, neither of which is necessary or necessarily true. The first assumption is that individual differences in human performance depend on, or are related to, or reflect individual differences in the strength or magnitude of a corresponding underlying trait or propensity. The second assumption is that the goal of assessment is to measure individual differences in the strength or magnitude of the underlying trait. In this view, the goal of assessment is to measure traits. This view is never questioned, and it has implications for understanding validity. In this standard model, a test or measure is valid to the degree that it accurately measures the underlying trait.But this view of validity makes it impossible ever to determine validity. The problem is that the actual existence of traits is questionable on genetic and neuropsychological grounds. There are no anatomical or neurological structures corresponding to any of the many traits that have been proposed. Consequently, it is by definition impossible to determine whether a test accurately measures an underlying trait—because the existence of trait is in doubt.The confusion about the meaning of validity can be resolved fairly easily with a simplifying assumption. If we assume that the goal of assessment is to predict outcomes, then validity can be defined in terms of the ability to predict those outcomes. A measure of sales performance is valid if it predicts sales performance; a measure of customer service potential is valid if it predicts ratings of customer service, and so on. This view makes no assumptions about the existence of underlying traits the strength of which causes performance. It simply stipulates that assessment has a job to do, and that job is to predict non-test outcomes. This definition of validity satisfies the requirement of Occam’s razor, which states that one ought not multiply causal entities unnecessarily; the definition also satisfies the aesthetic of the Bauhaus movement which states that less is more.

Business literature typically characterizes the people at the top of organizations in remarkably positive terms. Senior executives are described as visionary, acute, charismatic, decisive, creative, etc.—i.e., as being substantially more talented than the average person. They are also usually described as being well motivated, as caring about their staff, their colleagues, their organizations, and their shareholders. The reality is somewhat less flattering.With regard to the talent level of senior executives, there are no reputable data to support the claim that they are more capable than a group of younger but equally well educated managers. The data do indicate, however, that, as one ascends the corporate hierarchy, the demands of jobs change, and the data show that derailment is a function of an inability to make the needed adjustments to changing job demands. We can conclude that the senior people in most organizations differ from their younger colleagues primarily in terms of age and political experience, and little else. But what about the claim that the senior people are better motivated than the rest of us? The example of Sprint Nextel, a telecommunications company headquartered in Overland Park, Kansas is instructive.Daniel R. Hesse, who formerly ran the wireless business at AT & T, became CEO of Sprint Nextel in December, 2007. Consider the situation he faced and with which he is valiantly trying to cope. First, the management team that he inherited had participated in the merger of Sprint with Nextel, a merger that is charitably described as “ill fated” because the two corporate cultures were so poorly aligned. Second, the same management team presided over a massive defection of customers; at 2.5 percent, Sprint’s client churn is the highest in the industry; in the first quarter of 2008, 1.1 million of Sprint Nextel’s 59.3 million customers defected to other companies. They left because they were tired of surly customer service and lackluster cell phones.At his first meeting with his senior management team, Mr. Hesse asked the group who was responsible for customer service and no one raised a hand. At his next meeting he asked senior managers how they had arrived at their earnings projections and the response was blank stares. Mr. Hesse has moved quickly to put some accountability in place for his senior managers, but that is not the point. The situation he encountered is actually quite common, the reason the situation is common is important, and has to do with the secret life of organizations.The fundamental dynamic in every organization is the individual search for power. It is a Darwinian process, which means some people are more successful at it than others, and they rise to the top. Perhaps the most important single challenge for top managers is to try to persuade their staff to spend less time trying to advance their careers and more time trying to enhance the performance of the organization that employs them. This is a challenge for two reasons. On the one hand, American culture, with its symbiotic ties to capitalism, is inherently individualistic and typically misreads Adam Smith’s concept of the “invisible hand”. Many people (who should actually know better) believe Adam Smith argued that if each person pursues his/her self-interest, the general welfare will take care of itself. Smith had no illusions about the rapacious tendencies of capitalism; he understood that, left to their own devices, most people will behave badly. Smith also understood how hard it is to get people to understand that their welfare is tied to the welfare of the collective.On the other hand, managers are usually evaluated and promoted based on how much their boss likes them. They are rarely evaluated and promoted based on their contribution to the maintenance of the entire organization, or to the team for whose performance they are responsible. And this was exactly what was going on at Sprint Nextel before Daniel Hesse arrived. His challenge is to damp down the individualistic excesses of his managers and find leaders who are willing to think about and work for the good of the organization rather than trying to enhance their personal careers. His biggest asset in this process is his own stature as a role model.