Take Your Text Analysis up a Level


Organizational research is complicated and the levels of analysis we are interested in is no small contributor to this complexity. In one room at our annual Academy of Management meeting you could find one scholar looking at the within-individual cognitive processes of individuals at work and another comparing the influence of institutional-level processes on firm strategy. 

This diversity of interests enriches our field and the human/social origins of many of our constructs creates myriad opportunities for cross-pollination across levels. However, application of constructs across can introduce both theoretical and methodological challenges. For instance, management researchers famously debated the appropriate level of measurement and analysis of the organizational climate construct (i.e., Glick, 1985; James, Joyce, & Slocum, 1988; Glick, 1988).

There are well-established guidelines regarding how to use survey methods to elevate constructs’ level of measurement (i.e., Chen, Mathieu, & Bliese, 2005). However, survey response rates in organizational research can make rigorous applications of these methods difficult, causing researchers to turn to other techniques for organizational-level measurement. Computer-aided text analysis (CATA) is particularly attractive for measuring organizational-level constructs given the regular publication and availability of organizational texts. However, research in this area lacked guidelines for how to elevate constructs’ level of measurement.

Elevate Level of Analysis Using CATA

In our 2013 article, Using Computer-Aided Text Analysis to Elevate Constructs: An Illustration Using Psychological Capital, Jeremy Short, G. Tyge Payne, and I set out to adapt best practices from survey research to demonstrate how to elevate level of analysis using CATA while guarding against common theoretical and methodological concerns arising from doing so. We identify 19 steps in 5 phases that CATA researchers should follow.

Five phases:

  1. Definition of construct and development of deductive word lists
  2. Specification of the theoretical nature of the elevated construct
  3. Selection of appropriate texts and finalization of word lists
  4. Assessment of psychometric properties
  5. Examination of construct relationships

 

See the full table here

Key Considerations

In the first phase, you need to specify the definition and dimensionality of the construct at both the original and destination levels of analysis. These provide the basis for the development of initial deductive word lists for the CATA measures.

In the second phase, you need to provide the theoretical backing for the nature and existence of the higher-level construct. This involves not only accounting for the processes by which the higher-level construct emerges, but specifying the ways in which the elevated construct is similar to/differs from the original construct. This is key, because the conceptualization of the construct drives methodological decisions you will make in your content analysis.

In the third phase, you will use the theory developed in the first two phases to identify and collect a sample of appropriate texts with which to finalize and validate the measure. For instance, if measuring a construct at the organizational level, you will ideally collect a sample of texts reflecting organizational-level phenomena and contributed to by multiple individuals in the organization (e.g., Shareholder letters, 10-K filings, etc).

 

In the final two phases, you should put your measure through a battery of validation and psychometric assessments to ascertain the measurement qualities of the elevated construct. Many of these tests should be done to validate all new measures, regardless of their origin. However, a distinguishing characteristic here is that you want to compare your findings to the theory you specified regarding the elevation of the construct. Should there be close ties to findings of the lower-level construct, or is the elevated construct likely to be fairly different?

Organizational Psychological Capital

To demonstrate this process, our study elevated positive psychological capital to the organizational level. At the individual level, organizational behavior research defines psychological capital as being “an individual’s positive psychological state of development” (Luthans, Youssef, & Avolio, 2007: 3) and reflects the shared variance among an individuals optimism, hope, confidence, and resilience. Scholars in this literature have posited that this construct may also manifest at the organizational level, reflecting the positive psychological resources of the organization and may be a potential source of competitive advantage (e.g., Luthans, Luthans, & Luthans, 2004). 

Our study created CATA measures for Organizational Psychological Capital. Using a sample of shareholder letters from S&P 500 companies, we examined how CEOs use language associated with organizational psychological capital in communications with shareholders. In-line with Luthans, Luthans, and Luthans’ (2004) assertion, we found preliminary evidence that organizational psychological capital may be an organizational resource leading to improved firm performance.

The primary contribution of this study is methodological. However, given the well-documented benefits of positive psychology for individuals (e.g., Avey et al., 2011) and our finding that positive psychological firm resources may influence firm-level outcomes, there may be a valuable managerial implication as well. Firms that find ways to instill in their employees the optimism, hope, resilience, and confidence that engender psychological capital may benefit not only from a healthier, happier, and more productive employee base, but also (and likely by consequence) may find positive performance outcomes for the firm as well.

Starting Afresh…


A new look to keep the trains running smoothly.

404 – Page Not Found… it’s there, I swear!

Error Establishing Database Connection… *grumbles in PHP*

This is why I am in business and not computer science. (Seriously, I was a hair away from being a computer science major in undergrad, but debugging PHP during the dot-com boom turned me off of it.)

Anyway, welcome to the refreshed amckenny.com and catscanner.net. It’s now a WordPress/BoldGrid site instead of my own custom PHP code, so hopefully that means less upkeep and fewer bugs. It also enables me to do something I’ve been meaning to do for some time… this.

I’m doing this mostly to hold myself accountable for staying up-to-date on reading interesting things from both the academic and popular business press. As I get engrossed in my own research, I often slow down on reading other work not directly related to the paper in question. Some of that is natural and appropriate, but sometimes I end up taking it farther than I should… I want to see if this helps.

What to expect…

 

WILT  – What I Learned Today

I love learning new things, and I suspect I’m not the only one. So when I learn something I find novel/interesting enough to share, I intend to post it here. It’s not going to be every day, but a couple times a week seems reasonable.

Publications

As I publish new work, I will provide an extended abstract of the work, perhaps some thoughts about it, and provide a link for those interested in reading more. In academia, we typically take a bit of a passive view on promoting our work. I appreciate the modesty of it all, but it does lend to the feeling that we’re talking (only) to ourselves. Maybe this is a happy medium between isolation and aggressive self-promotion?

Useful Tools/Techniques/Tricks

Jeremy Short and I designed the computer-aided text analysis package CAT Scanner to provide researchers with a free text analysis tool. In my research, I’m often developing little tools to help me do things faster/ easier. If I think others might be interested in trying out these tools, I’ll share them here as well.

Musings

As I am working through perplexity—whether theory development, methodological choices, or pedagogical decisions—it is tremendously helpful for me to get my thoughts down in writing. By trying to articulate my thoughts in a clear manner, I often find that the solution becomes more apparent as well. I see this being useful to that end as well.
 

At any rate, welcome to the new, cleaner amckenny.com.