Cognipsy - Blog #validity

Investigating close-mindedness with SPS

Mon, 28 Dec 2020 10:35:00 +0530

Success of human beings as a species ought to be due to higher cognitive functions, social bonding and the ability to be open to new solutions. From an evolutionary standpoint, close-mindedness should not be a matter of individual difference unless nature allowed it to be propagated for certain benefits. Close-mindedness offers certain evolutionary benefits which can be many and in the tradition of evolutionary psychology, we can only guess them. But the discourse leads to important insights into workplace behaviours.

In this article, we discuss close-mindedness and what the SPS research tells us about it. We define close-mindedness as a tendency to prefer established ways of thinking and doing things over looking for, acknowledging and utilising new information in intrapersonal and interpersonal decision-making. Close-mindedness is not just the opposite of open-mindedness and it is important to distinguish. It is easy to consider it a bipolar opposite of openness factor from the Big 5. It is a more complex construct and an important one given how much it impacts social interactions - something that is usually more linked to agreeableness and extraversion factors.

We asked 247 individuals attending selection interviews who took SPS Scan to rank order what they consider their common workplace challenges. The adjective inventory uses undesirable adjectives mapped to all of the big five factors (from the SPS Scan original item bank) and Likert statements from SPS Scan. All respondents were forced to choose at least 1 and upto 4 from a block of 8 negative phrases (standard SPS Scan format). We looked at individuals who self-reported close-mindedness as a descriptor for themselves and compared them with the Likert statement responses. We report below the top 5 items with highest differences of practical and statistical significance (p<0.01).

Item	Cohen's D (based on standardised theta scores)
I am always prepared	0.57 (Moderate)
I excel in what I do	0.45 (Low)
I prefer variety to routine	- 0.35 (Low)

We mapped these back to SPS Scales and found that self report differences between individuals who self-reported close-mindedness and those who didn't were the highest for scales Competence, Challenge and Creativity. Interestingly, these are all Drive scales indicating a motivation-related content of self-report close-mindedness. We expected large differences on the Humility and Progressivism scales but didn't fine one (d=-0.11, d=-0.19).

We looked at the other-report study done using SPS Scan self-report and other-report data from SPS Scan checklist. We studied the subjective comments for signs of close-mindedness in a binary manner. Profiles were sorted as those that indicated close-mindedness and those that didn't. Following effect sizes were found in 58 other-report data points:

Item	Cohen's d (based on standardised theta scores)
I like to keep busy.	0.93 (Large)
I am very productive.	0.80 (Large)
I see beauty in things that others might not notice.	0.72 (Moderate)
I am always on the go.	0.66 (Small)

Some outcomes were expected and some were surprising.

Self-report humility as well as progressivism were not very accurate indicators of close-mindedness. We hypothesise that social desirability may reduce the efficacy of these measures in predicting close-mindedness. Further, it is very much possible that close-minded individuals may not be open to feedback on their humility and progressivism from social interactions. This would lead them to believe that they are less close-minded than others. Interestingly, other-report Humility and Progressivism had higher effect size of 0.36 and 0.39 respectively. But neither of these were found to be statistically significant. Possibly due to the smaller sample size of the group.

We hypothesise that close-mindedness is an attitudinal construct given high association with Drive constructs. This could indicate that some individuals may find close-mindedness rewarding.

We analysed the items that best predicted self-report and other-report closemindedness. These correspond to SPS Scan scales which measure common workplace risks. Two themes from the four scales emerge:

- Avoids Complexity and Avoids Novelty which put together may be inversely interpreted as Abstraction-orientation

- Low Productivity and Lacks Drive which put together (along with competence and challenge drives) may be inversely interpreted as Closure-orientation

We suggest that abstraction-orientation (low) and closure-orientation (high) may be two dynamic affects that may constitute close-mindedness. This explains the rewarding attribute of close-minded behaviours. Sticking to biases and not being open to new inputs is likely to help bring quicker closure and give a sense of efficacy and confidence to individuals in dealing with day to day decision-making. On the other hand, it also reduces one's ability to work with abstract information which would impede one's ability to deal with complex problem, reconciling opposing viewpoints, paradoxes, new ideas and new situations.

Workplace implications of these findings are interesting as well. Our research suggests that individuals who consider themselves to be effective in carrying out tasks and productive are at higher risk of being close-minded. They may not be consciously leveraging the benefits of the dynamic and worse still, may not be aware that they are close-minded or carry that reputation, in the first place. Self-awareness is critical for high flying executives and managers as their close-mindedness might just be the cost they pay to perform their tasks quickly and effectively in organisational settings.

What we know about predicting leadership

Thu, 29 Oct 2020 18:33:10 +0530

Leadership, as you may already in know, is a rather complex phenomenon. But what you should also know is that great amount of money is spent by companies to hire leaders, train/develop them and deal with them should they not perform. Of all costs the latter is the greatest and is seldom accurately reported.

In this summary of research we briefly discuss the science of predicting leadership emergence and effectiveness. An emergent leadership is a very different from effectiveness model of leadership. However, this difference is not the scope of the article.

Intelligence

Better described as general cognitive ability, intelligence is a great predictor of leadership. In fact of all psychological factors g can predict leadership emergence the best (r=0.4). Ability to work with complexity, think about future, build scenarios, learning things quickly and solving problems on the fly are all much needed competence in a leader and g predicts just that. Needless to say g alone is not a mark of a good leader, but from statistical point of view, g predictss leadership the best.

Personality

Personality is a good predictor of job performance. Leadership requires different behaviours at different instances. The ability to exercise one's traits is a mark of good leadership. However, a great deal of research has proven that conscientiousness, neuroticism and openness to idea are all good predictors of leadership emergence. Extraversion and agreeableness have mixed results. Surprisingly, agreeableness has been found to have negative correlations with leadership emergence and positive correlations with effectiveness.

Handling Stress

Leaders are less impacted by stress. And the proof is irrefutable. Biological evidence from cortisol level (stress causing hormone) suggest that the basal levels of cortisol in leaders are higher. Which entails that leaders are used to high stress and their cortisol levels do not increase much with stress. Emotional stability factor from personality also has positive correlations with leadership emergence. Hence, ability to not be perturbed by stress is also a sound predictor of leadership capability.

Emotional Intelligence/Ability

While studies show that EI predicts leadership effectiveness, EI itself is highly correlated with g and big 5 factors. Hence , from psychometric standpoint it doesn't offer any additional insights into leadership capability. Alarmingly, some studies have shown that high EI leads to lower leadership effectiveness.

Creativity

Creativity both in terms of a trait as well as ability is a strong predictor of leadership effectiveness and not emergence. Surprisingly this is true across industries and sectors. This could be a reflection of the fact that strategizing and exploring new avenues of growth do require creativity.

In all, psychology has enough insights into what would make a great leader. Furthermore, definitions of what effective leadership looks like across cultures is similar. This doesn't mean that culture has no impact on leadership. Adequate evidence exists to suggest that socially acceptable leadership behaviours differ from culture to culture. E.g. in US a good leader is expected to be an extrovert while in Japan a good leader is culturally perceived to be introverted. That said, a great amount of leadership can be scientifically predicted with analytical models which are customised to an organization, industry and culture.

The Lens of Consequential Validity

Thu, 29 Oct 2020 17:38:15 +0530

For a very long time, publishers could easily move on after publishing an instrument and establishing its basic psychometric properties. Times have changed now. Just publishing, ensuring that certified professionals use the instruments and taking care of operations are not enough. Recent debates on consequential validity are taking the field by storm.

A critical question that some psychometricians are asking is what should matter the most to the user when it comes to validity. There are no easy answers. But it is obvious that the real test of validity of an instrument is not just that it measures what it purports to measure but also that it has the impact on end users that it purports to have.

Succinctly put, this concept for many is an issue of ethical concern and not an issue of validity. Many others have called it the soft validity of an instrument. The key component of the debate however is that whatever we may call it, can we really escape it? Can a publisher or practitioner take a hands off approach and claim that while an instrument was proved to be valid over a tested sample, its real impact need not be evaluated?

This allows us to ask other questions too. Where does a publisher's or a practitioner's responsibility end when it comes to assessments and its impact. Would it be fair to hold a publisher responsible if environmental factors dilute the utility of an instrument?

It is definitely not one of the easier debates where numbers could hold the verdict. A more comprehensive view and probably a more paradigm shifting view of the field and its applications is required.