A Quick and Dirty Guide to Validity & Reliability

Choosing the right assessment for selecting or developing employees can make or break the success of a talent initiative. Why bother using assessments that don’t predict performance, or that fail to resonate with your business leaders?

When deciding on the right assessment for your valuable talent, pay attention to the scientific rigor with which the instruments have been tested. Any good tool should have concrete data demonstrating its validity and reliability. Validity and reliability can tell you two general things: 1) that the assessment is measuring what you want it to, and 2) that it will reliably assess the same thing each time – ensuring that the results you get aren’t a one-off.

An easy way to think about this concept is with a bullseye metaphor: The very center of the bullseye is exactly what you want to assess.

 

Val_Rel_Target_Model_2_800

Reliable but not valid means that you are consistently testing the same thing over and over again, but it’s not testing what you want to test.

Valid but not reliable means that the average scores align with the goals of the test, but individual scores are inconsistent.

Both reliable and valid means that the test will consistently measure what it is supposed to over a period of time – it’s consistently hitting the bullseye.

Read More »

Podcast: Hogan CEO Discusses His New Book

Too often, HR practitioners and talent management experts rely on intuition when it comes to identifying and developing key talent, even when armed with an overwhelming amount of data and objective tools that contradict their instincts. The problem is that the data they ignore is right in almost every case.Talent Delusion

When these “experts” are wrong, it costs organizations all over the world billions of dollars each year in turnover and wasted development expenses. Unfortunately, all of that money could have been salvaged if organizations instead relied on data over intuition in their selection and development practices.

In a recent collaboration with National Business Review Radio and Spotify, Hogan CEO Dr. Tomas Chamorro-Premuzic spoke with Andrew Patterson to discuss his new book The Talent Delusion – Why Data, Not Intuition, Is the Key to Unlocking Human Potential. The book aims to education HR practitioners and leaders on how to measure, predict, and manage talent.

All organizations have problems, and they nearly always concern people. Psychology, the main science for understanding people, should be a pivotal tool for solving those problems.

 

Hogan X Launches Consumer-Facing Platform

 

Hogan X, the innovation division of Hogan Assessments, has launched a new consumer-facing personality platform that helps people understand themselves, their motivations, interests, strengths and challenges.

HoganX Updated LogoThe interactive assessment, which takes just minutes to complete, provides users with the answer to the key question everyone asks themselves at some point in their lives: “Who are you?” The new tool, Hogan X’s most advanced to date, dynamically generates over 2,000 different personality descriptions.

“At Hogan X, we are passionate about helping people understand themselves and others, so they can make their lives better,” says Dave Winsborough, the head of Hogan X. “That’s why we were adamant about making this tool completely free. We believe that self-knowledge is a right and should not come with a price tag.”

The assessment leverages Hogan Assessments’ four decades of personality research and data, which is widely accepted as the most reliable personality science in the world. It is the first time Hogan X and Hogan Assessments have moved away from the world of work to focus solely on people.

“By combining Hogan personality science with cutting-edge technology, we are quickly positioning ourselves at the forefront of our industry in both reliability and technology,” says Uri Ort, Hogan X’s Manager of Innovation. “And, this is only the beginning. We have numerous projects well underway, which will continue pushing the boundaries of personality science.”

To take the assessment or for more information, visit HoganX.io or connect with Hogan X on Facebook.

Distributor Spotlight: metaBeratung Focusing on Innovation in Leadership

It’s no secret that technology is affecting all facets of our lives. Social media has affected how organizations attract new customers. Smartphones have put the internet at our fingertips, which has changed the way we communicate, how we access information, and even how we order lunch. But, how will it affect leadership?metaBeratung Logo

metaBeratung, a Hogan Distributor in Germany, Switzerland, and Austria, is staying at theforefront of this issue as it is hugely impactful to the global workforce. Organizational landscapes have changed and continue to change. Therefore, what it takes to be an effective leader has also changed. metaBeratung, in conjunction with IMD Business School, recently examined the qualities and competencies that translate to effective leadership in today’s workforce.

Agile Leadership in an Age of Digital Disruption

In 2017, metaBeratung became an official partner of IMD Business School in Switzerland, and conducted a study that revealed why leaders need to adapt their approach to leadership.

“This is a win-win collaboration for us,” says Nicole Neubauer, managing director of metaBeratung Switzerland. “On the one hand, will we be working together on joint research, on the other, we are able to offer IMD executive programs to our clients based on individual development needs. A perfect match.” Read More »

Podcast: Dr. Hogan Discusses Future of High Potential Programs

HRD Connect LogoIt’s no secret that identifying high potentials (HIPOs) is a hot topic in the global discussion on talent management, and for good reason. Those who hold leadership positions today will inevitably move on or retire. Thus the need to identify and develop successors is vital to the future of organizations worldwide.

Numerous organizations across the globe have implemented HIPO programs to groom the next generation of leaders. Unfortunately, most organizations are not very good at identifying HIPOs. Instead of using data and objective measures that are backed by science, companies tend to promote those who emerge, such as the confident, charismatic, and politically savvy individuals. However, science tells us those who emerge are often ineffective.

In an interview with HRD Connect in Amsterdam, Dr. Hogan addressed the HIPO topic, and offers his top tips for assessing employee potential.

If HIPOs are critical to the success and sustainability of your organization, contact a Hogan consultant or visit hoganhipo.com.

SIOP 2017: Collected Wisdom from Orlando

The 2017 SIOP Conference was the largest in history, and several Hogan representatives presented on a wide variety of topics. For those of you who were unable to attend or for attendees interested in accessing the Hogan presentations, here is a recap with links to many of the Hogan presentations at this year’s conference.

*Please note that symposium titles will differ from linked content. The content provided is to a paper within the symposium.

Conceptual Foundations of Personality Assessment in Organizations: “Useful” to “Optimal”

Interest in personality assessment in organizations continues to grow.  However, criterion-related validities are only at the “useful” level currently.  Three personality models are presented that have organizationally relevant labelling and are more differentiated at the primary factor level.  These models should help personality validity reach “optimal” levels.
Jeff Foster, Steve Nichols

Development of Empirically Based Short Form Personality Assessment

Short form personality assessments are nothing new, but many existing forms are constructed with an emphasis on internal reliability rather than predictive utility. A short form was developed using an empirical approach that simultaneously optimizes reliability and criterion validity. Evidence supporting the utility of the short form is provided.
Brandon Ferrell, Blaine Gaddis Read More »

Hogan Launches New Online Post-Certification Course

In response to overwhelming customer demand, we have launched a new online Level 1 Hogan Certification course that will serve as a post-certification “refresher” program for Hogan-Certified users. The program will be geared toward the review and reinforcement of basic Hogan interpretation skills gained in the Level 1 Hogan Certification Program.Certification Banner

“With approximately 50,000 Hogan-Certified professionals across the globe, it became clear that our users were hungry for a way to hone their skills and continue to learn more about the Hogan Assessment Suite,” says Jackie Sahm, Hogan’s Director of Global Learning. “This new program gives them that opportunity at their fingertips.”

The web course provides Hogan users with a thorough review of all 28 scales across Hogan’s three core assessments: Hogan Personality Inventory (HPI); Hogan Development Survey (HDS); and the Motives, Values, Preferences Inventory (MVPI). The course begins with a diagnostic assessment of the learner’s current knowledge of the Hogan scales, and offers appropriate tutorials to strengthen and reinforce interpretation of the Hogan Assessment Suite.

The course will include challenging exercises to both assess and teach scale meanings and their performance implications. Additionally, informational feedback is provided throughout the course to guide interpretive thinking versus rote recitation.

“We’re really excited to launch this new program, but we’re even more excited that we have thousands of Hogan users who already see the value in it,” says Sahm. “It confirms that our users understand the importance of personality assessment and its role in the future of talent management and leadership development.”

The price of the course is $195 USD, and is available via Hogan’s On-Demand Learning Course Catalog. For more information, contact training@hoganassessments.com.

 

Hogan Forms Partnership with Talent Quarterly

We’re excited to announce that Hogan has partnered with Talent Quarterly, the leading voice for science-based, practical advice on hiring, developing and retaining exceptional talent.

Talent Quarterly, published by The Talent Strategy Group, makes organizations more successful by providing evidence and insights to help leaders make higher-quality talent decisions. With that mission, and a commitment to improve the global discussion around talent management, a partnership with Hogan made perfect sense.TQ Cover

“I’ve been a fan of Hogan for quite some time, and their views on talent management align perfectly with what we’re trying to accomplish at Talent Quarterly,” says Marc Effron, Founder & Publisher of Talent Quarterly and Founder & President of The Talent Strategy Group. “I started this publication because the current dialogue around talent management simply wasn’t sufficient. Hogan has been battling the status quo around personality assessment and leadership development for decades, so this partnership was a natural fit.”

With contributions from some of the world’s leading management authors, consultants, and scholars, Talent Quarterly offers readers a fresh perspective and allows them to view the world’s talent issues through a new lens. Each issue focuses on a particular theme around talent management, including culture, female talent, performance, and engagement. The latest, The Boss Issue, covers such hot topics as curing incompetent leadership, how to become a superboss, and how smart bosses build smarter employees.

The partnership will allow both organizations to continue their mission of improving the global workforce.

“We are quite excited about this partnership,” says Hogan CEO Dr. Tomas Chamorro-Premuzic. Marc is not only a superstar in the world of talent management, but he is also on a quest to educate professionals and leaders in order to make talent practices more data-driven and effective. Talent Quarterly embodies this mission and we are honored to play a part in making it an even bigger success.”

Talent Quarterly is a quarterly print magazine accompanied by a daily digital operation. Print and digital subscriptions, complete issues, and individual articles are all available at talent-quarterly.com. Also, follow them on Twitter: @TalentQuarterly.

Distributor Spotlight: After a Decade of Success, Mobley Group Pacific Continues to Deliver Stellar Results

MGP Logo_Big Size

When American psychologist, Dr. William H. Mobley, founded Mobley Group Pacific in 2004 in Shanghai, he had a vision to revolutionize the development of China’s business community. In 2007, he made the decision to put his entire staff through Hogan Certification, and MGP became an official member of the Hogan Distributor Network. Fast forward 10 years, it is quite apparent that his vision has become reality.

Focusing on executive assessment for senior executives, leadership teams, and emerging leaders, MGP has used the Hogan assessment suite to enhance the effectiveness of organizations across the Greater China region, covering Mainland China, Hong Kong, Macau, and Taiwan. The diversity of their client portfolio, including industries such as automotive, healthcare, manufacturing, and food & beverage, is a true testament to the versatility of Hogan products. In 2015, MGP deployed a Hogan-centric coaching program for a large, international pharmaceutical company, and recently shared how they implemented it:

“A leading multinational in the pharmaceutical industry initiated an organizational change in the beginning of 2015. To facilitate such a change, the client wanted to increase its leaders’ awareness of what such changes may bring. The entire process started with a Hogan-based development feedback and coaching session. MGP worked with the top-200 leaders in China over a period of 18 months. The feedback we gathered from such a process was that we successfully kicked off the change process through cultivating an in-depth self-understanding. On top of the individual sessions, we also conducted (over different cohorts) Hogan-based group analyses, helping the senior executive and the program champion to form a high-level understanding of whom do they have, how their leaders lead, as well as the implications to such a big change.” 

The implementation of such a comprehensive leadership development program is a daunting task for any organization, regardless of industry or company size. With the help of MGP, organizations across China have seen significant outcomes, and the region can expect many more years of superb service and stellar results.

Hogan Assessments has experienced a great deal of success through the years, and this is largely due to Bob Hogan’s emphasis on generating results that help a company’s bottom line. When partnering with organizations across the globe like MGP, those results are delivered frequently, and all parties involved benefit greatly.

 

 

Hogan to Speak at 32nd Annual SIOP Conference

SIOP17_email_plainI-O experts from Hogan’s Research and Consulting divisions will showcase advances in personality research during an impressive 30 sessions, symposia, panel discussions, practice forums, and poster sessions at the 32nd Annual SIOP Conference in Orlando, April 27-29. #SIOP17 #HoganatSIOP


Thursday, April 27

10:30-11:50
Asia 2
Conceptual Foundations of Personality Assessment in Organizations: “Useful” to “Optimal”Interest in personality assessment in organizations continues to grow.  However, criterion-related validities are only at the “useful” level currently.  Three personality models are presented that have organizationally relevant labelling and are more differentiated at the primary factor level.  These models should help personality validity reach “optimal” levels.
Jeff Foster, Steve Nichols

10:30-11:50
Australia 3
Giving Top Performers the Star Treatment: Is Meritocracy Overrated?
Should organizations especially single out—and disproportionally recognize and reward—star performers? Do stars really contribute disproportionately to organizational success? What are unintended negative consequences of meritocratic practices? These and related questions will be debated by a distinguished panel representing both sides of the meritocracy question.
Tomas Chamorro-Premuzic

12:30-1:20
Atlantic BC
Development of Empirically Based Short Form Personality Assessment
Short form personality assessments are nothing new, but many existing forms are constructed with an emphasis on internal reliability rather than predictive utility. A short form was developed using an empirical approach that simultaneously optimizes reliability and criterion validity. Evidence supporting the utility of the short form is provided.
Heather Hayes, Brandon Farrell, Blaine Gaddis

1:30-2:50
Asia 4
Millennials Versus Millennials Debate (and No One Gets a Trophy)
As scientist–practitioners, industrial-organizational psychologist must understand each side of the Millennial debate. The current session presents the different viewpoints through an interactive scripted debate and panel session. Panelist will “argue” both sides of the Millennial debate, then present an overview of their own experiences managing generational issues in the workplace.
Amber Smittick

1:30-2:50
Asia 1
What a Difference a Boss Makes: Leveraging Leaders to Enhance Employee Engagement
Given the impact that leaders have on employee engagement, this panel will focus on techniques to identify, select, develop, and evaluate engaging leaders. Panelists will share benefits and challenges associated with current practices used in their organizations and provide recommendations for other organizations to enhance employee engagement through improved leadership.
Jocelyn Hays

1:30-2:50
Hemisphere E2
Will Technology Make Assessment Obsolete?
How will assessment evolve in the next decade? Industry experts will explore how trends such as big data, artificial intelligence, virtual and augmented reality, social media, and the Internet of Things will force assessment to change. The session will offer guidance to those working to push forward.
Ryan Ross

3:30-4:50
Hemisphere A3
One Type to Rule Them All? Debating Predictors in Selection
In this presidential-style debate format, 4 panelists, each representing a common selection method including cognitive ability, personality, situational judgment, and simulation, will respond to judge panelists’ questions on how they can help organizations in 2 fictitious business scenarios. Fun and engaging conversation is assured to invigorate the audience.
Steve Nichols

4:30-5:20
Atlantic BC
Derailers Versus Personality Disorders: What are the Differences?
There remains little consensus regarding the structure and meaning of personality derailers. This research aims to fill this gap by comparing items from the HDS and the PID-5.  Results support the concept of derailers as personality constructs that align with disorders but are not clinically debilitating.
Heather Hayes, Jeff Foster, Blaine Gaddis

5:00-5:50
Hemisphere II
The Use and Utility of Big Data in I-O Psychology
Recent SIOP conferences have seen a surge in sessions on big data, most of which highlight future possibilities of using big data techniques in the field.  In contrast, this session will focus on active projects within the field and the real benefits of big data for I-O psychology today.
Brandon Farrell, Jeff Foster, Blaine Gaddis

Friday, April 28

8:00-9:30
Hemisphere A4
The Hungry Mind: Why Companies Hire and Promote for Curiosity
This panel examines the growing importance of intellectual curiosity in different organizational settings. Six experts from renowned global organizations discuss how they go about selecting and nurturing individuals with a “hungry mind,” why curiosity is critical for employability and career success, and how it impacts on leadership and organizational effectiveness.
Tomas Chamorro-Premuzic

10:00-11:30
Asia 4
Factors to Consider in 360-Degree Feedback Ratings
The goal of this symposium is to discuss factors that can affect 360 performance ratings. Four studies on the effects and relationships of (a) culture, (b) gender and personality, (c) gender and importance performance, and (d) personality and self–other discrepancies in 360-ratings are discussed.
Karen Fuhrmeister, Derek Lusk

1:00-2:20
Americas Seminar Room
Identifying Grit in Existing Personality and Other Individual Differences Taxonomy
Proponents of grit treat it as a completely new construct, though a few studies suggest it is indistinguishable from other well-established constructs like Conscientiousness. This session focuses on examining relationships between grit and constructs from personality, positive psychology, and interests.
Brandon Farrell, Robert Hogan

1:00-2:20
Hemisphere E4
Best Practices in Personality-Oriented Job Analysis
This session explores how to best identify which personality trait scales will be job related using expert judgments. The panel discussion will feature brief presentations from the participants, answers to a series of structured questions, and a question-and-answer discussion with the audience in order to identify cutting-edge job analysis methodologies.
Jeff Foster

3:00-3:50
Atlantic BC
Differences in Judgment and Decision-Making Across Job Levels
Leadership judgments drive corporate performance. However, it is common for leaders to make poor decisions. Therefore, it is necessary to look at judgment tendencies and the ability to learn from past mistakes. This study examines judgement at different job levels to identify differences in key judgment tendencies.
Michael Tapia, Blaine Gaddis

4:00-4:50
Atlantic BC
Job Analytic Comparisons of Critical Competencies Across Industries
Using archival job analytic data, authors examined the degree of consistency in competencies required for effective job performance across 10 industries. Findings suggest that characteristics identified as important for jobs in one industry are likely to generalize across industries with few exceptions.
Matt Lemming

4:00-5:00
Atlantic BC
Validation of an Off-the-Shelf Competency Solution for Nine Job Families
Many organizations develop competency models to guide HRM efforts, but models may be based in business trends more than science, making validity evidence scarce. the authors developed and validated an off-the-shelf competency solution to help organizations identify individuals with personal characteristics aligned with critical competencies for 9 job families.
Blaine Gaddis

Saturday, April 29

8:00-9:20
Hemisphere A4
Assessments on Mobile Devices: Our Opportunities at Digital Speed
This session will discuss current issues in the use of mobile devices for completing candidate assessments. The IGNITE sessions will cover topics ranging from prevalence of test completion on mobile devices, using mobile assessment technology as a recruitment tool, and key elements of a successful mobile assessment app.
Amber Smittick, Jennifer Lowe

8:00-9:20
Hemisphere V
Limits of Engagement: A Panel Discussion
Employee engagement is the amount of behavioral, affective, and cognitive energy dedicated by employees in their work. Although there has been widespread agreement about the meaningful positive consequences of an engaged workforce, this panel discussion will address if, when, and how there can be too much of a good thing.
Darin Nei

10:00-11:20
Hemisphere E3
High-Potential Identification: You’re Doing it Wrong
Although many of the numerous difficulties surrounding high-potential identification and development are well-publicized, this diverse panel seeks to explore high-potential issues and insights that are not commonly addressed in practice. Panelists will answer questions about the nomination procedures, assessment strategies, data inputs and development programs used in high-potential management.
Jackie Sahm, Scott Gregory

10:00-11:20
Americas Seminar Room
Recent Advances in Personality Assessment and Validation: Beyond Self-Reports
Personality assessments are a mainstay in personnel selection, yet applications to organizational research and practice have been dominated by self-report methods. This symposium showcases 4 papers demonstrating the usefulness of observer reports of personality. Papers advocate for the greater use of multiple sources of personality ratings in the organizational sciences.
Robert Hogan

11:00-11:50
Atlantic BC
Beyond Cognitive Ability: Using Personality to Predict Student Retention
This study examined the relationship between personality and student retention. Conscientiousness predicted student retention across 3 years and eventual graduation rates. Researchers and practitioners can use these findings to develop personality-based interventions to increase student retention and reduce costs for colleges and universities.
Matt Lemming

11:00-11:50
Atlantic BC
Personality Assessment as a Supplementary Predictor of Tenant Behavior
Current tenant screening methods lack thorough research support and may be subject to adverse impact. This study proposes the use of personality assessment as a supplementary tool and provides evidence for the use of personality measures to predict tenant behavior, including payments, vacating, maintenance, cleaning, landlord interactions, and causing damages.
Michael Tapia, Brandon Farrell, Matt Lemming

11:00-11:50
Atlantic BC
Improving Prediction Through Personality and Criterion ABC Alignment
Authors hypothesize that by aligning the affective, behavioral, and cognitive content of personality and workplace criteria, prediction will be improved. This hypothesis is tested in 2 datasets that have both personality and performance data. The results generally support the hypothesis; there was better prediction of performance on average.
Michael Tapia, Kimberly Nei

11:00-11:50
Atlantic BC
Dark Personality as a Causal System: A Network Approach
The origin, development, and nature of maladaptive traits are discussed as latent traits (e.g,. narcissism) causing sets of interrelated behaviors (e.g., arrogance). An alternative, network perspective is proposed for conceptualizing dysfunctional tendencies as causal systems of interlocking strategies. Item-level networks of Dark Triad are demonstrated and reproducible R code offered.
Jeff Foster

11:30-12:20
Hemisphere A3
Using Personality Assessment to Predict Valued Outcomes in Healthcare
The healthcare industry significantly affects people’s lives but relies on objective data. As such, assessments of “softer” individual differences have been underused despite evidence that these constructs predict health-related outcomes. This symposium allows professionals to demonstrate how personality assessments predict a range of outcomes for healthcare providers and recipients alike.
Blaine Gaddis, Kimberly Nei, Derek Lusk

11:30-12:20
Hemisphere E4
Maladaptation: Building the Nomological Net of Derailing Traits and Behaviors
Despite increasing interest in derailing traits and behaviors, there remains much to be learned about their construct space. By presenting quantitative and qualitative research results, authors seek to deepen understanding of how derailing traits and behaviors relate to other individual differences and work-related outcomes, expanding insight into maladaptation at work.
Brandon Farrell, Blaine Gaddis

11:30-12:20
Globalizing Selection Systems: What You Need to Know for Success
Hemisphere A1
Panelists will share their experiences related to planning, implementing, monitoring, and validating international selection systems. They will discuss strategies as well as lessons learned from these implementations. In particular, unique considerations important for global launches will be identified.
Steve Nichols

12:30-1:20
Hemisphere A4
Team-level Interventions: Using Personality Data to Enhance Team Effectiveness
Panel members will discuss their successes and lessons in using personality data for team-based applications. Topics will include best practices and ethical considerations for presenting personality data to teams, examples of creating and using team personality profiles, and other practical applications to improve team effectiveness and business outcomes.
Blaine Gaddis

12:30-1:20
Hemisphere E1
Mobile Assessment: Small Screens Become Mainstream (Demo & Panel Discussion)
This symposium brings together 3 assessment publishers demonstrating mobile assessment administration formats and comparing and contrasting their effectiveness. Audience members will participate by sampling assessment experiences “live” via mobile device. After demonstration, broader trends on mobile assessment will be highlighted from each publisher, and advantages/disadvantages of mobile assessment formats will be discussed.
Jennifer Lowe

3:00-4:20
Hemisphere A1
Looking Beyond Validity to Ensure Assessment Success
The successful implementation and sustainability of an assessment program is determined not only by the validity of an assessment but a number of other factors. Practitioners from 5 organizations will share their insights, perspectives, and experiences in how to successfully address these factors.
Jackie Sahm