Creating short forms for construct measures: The role of exchangeable forms
A popular trend has invaded applied psychometrics in terms of creating short forms for construct measures. There seems to be a paucity of developing methodologies for creating short forms based on complete forms that meet psychometric standards related to reliability of the scores and valid inferences. The present article suggests a methodology that rests on the fundamental assumption that the concept of short form attains meaning when being derived from valid scores of a complete form. A pivotal construct for assessing the status of short form is the concept of exchangeable forms which incorporates two types of measurement invariance; invariance across groups frequently exercised in studies applying confirmatory factor analysis, and b) invariance across random facets as estimated in generalizability theory. The two types of measurement invariance involve two types of generalizations relevant for inferring constructs; generalizing from a sample of persons to a population of persons, and generalizing from a sample of construct indicators to a universe or domain of construct indicators. In addition structural invariance is required; exchangeable short forms should relate equivalently to external reference variables.
Keywords: Valid complete form; exchangeable forms; generalizing to population of persons and universe of construct indicators; two types of measurement invariance; structural invariance.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411â423.
Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley.
Brennan, R. L. (1992). Elements of generalizability theory. Revised edition. Iowa City: American College Testing.
Brennan, R. L. (2001a). Generalizability theory. New York: Springer.
Brennan, R. L. (2001b). Manual for urGENOVA. Occasional papers no. 49. Iowa Testing Programs: University of Iowa.
Brennan, R. L. (2004). Some perspectives on inconsistencies among measurement models. CASMA. Research Report No. 8. University of Iowa, Iowa City: CASMA.
Cardinet, J., Tourneur, Y., & Allal, L. (1981). Extension of generalizability theory and its application in educational measurement. Journal of Educational Measurement, 18, 183â204.
Carstensen, C. H. (2009). Linking PISA competencies over three cycles – results from Germany. In M. Prenzel, M. Kobarg, K. Schöps, & S. Rönnebeck (Eds.), Research on PISA (pp. 199â213). Dordrecht, Netherlands: Springer.
Cornfield, J., & Tukey, J. W. (1956). Average values of mean squares in factorials. Annals of Mathematical Statistics, 27, 907â949.
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurement: Theory of generalizability for scores and profiles. New York: Wiley.
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137â163.
Dorr, D. (1981). Factor Structure of the State-Trait Anxiety Inventory for Children. Personality and Individual Differences, 2, 113â117.
Guttman, L. (1953) A special review of Harold Gulliksen: Theory of mental tests. Psychometrika, 18, 123â130.
Hagtvet, K. A. (1989). The construct of test anxiety. Conceptual and methodological issues. Bergen/London: Sigma Forlag/Jessica Kingsley Publishers.
Hagtvet, K. A. (1998). Assessment of latent constructs: a joint application of generalizability theory and covariance modeling with an emphasis on inference and structure. Scandinavian Journal of Educational Research, 42, 41â63.
Hagtvet, K. A., & Sipos, K. (2004). Measuring anxiety by ordered categorical items in data with subgroup structure: The case of the Hungarian version of the trait anxiety scale of the State-Trait Anxiety Inventory (STAIC-H). Anxiety, Stress, and Coping, 17, 49â67.
Hanin, Y. L., & Spielberger, C. D. (1986). The development and validation of the Russian form of the State-Trait Anxiety Inventory. In C. D. Spielberger & R. Diaz-Guerrero (Eds.), Cross-cultural anxiety (Vol. 2, pp.15â23). Washington, DC: Hemisphere.
Hedl, J. J. & Papay, J. P. (1982). The factor structure of the State-Trait Anxiety Inventory for Children: kindergarten through the fourth grades. Personality and Individual Differences, 3, 439â446.
IBM Corp. Released 2013. IBM SPSS Statistics for Windows. Version 22.0. Armonk, NY: IBM Corp.
Ibsen, H. (1867). Peer Gynt. Et dramatisk digt. København: Gyldendalske Boghandel.
Jøreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443â477.
Jøreskog, K. G. (1993). Testing structural equation models. In K. A. Bollen & J. Scott Long (Eds.), Testing structural equation models (294â316). Newbury Park, CA: Sage.
Jøreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Revised version 10 February, 2005. http://www.ssicentral.com/lisrel/corner.htm
Jøreskog, K. G., & Sørbom, D. (2012). Some new features in LISREL9. Document available in Jøreskog and Sørbom (2013). Chicago, IL: Scientific Software International
Jøreskog, K. G., & Sørbom, D. (2013). LISREL (Version 9.10) [Computer software]. Chicago, IL: Scientific Software International.
Kaiser, H. F., & Michael, W. B. (1975). Domain validity and generalizability. Educational and Psychological Measurement, 35, 31â35.
Kane, M. (2002). Inferences about variance components and reliability â generalizability coefficients in the absence of random sampling. Journal of Educational Measurement, 39, 165â181.
Lord, F. M., & Novick, M. E. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
Marcoulides, G. (1996). Estimating variance components in generalizability theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290â299.
Marsh, H. W., Ellis, L. A., Parada, R. H., Richards, G., & Heubeck, B. G. (2005). A short version of the Self Description Questionnaire II: Operationalizing criteria for short-form evaluation with new applications of confirmatory factor analyses. Psychological Assessment, 17, 81â102.
Marteau, T. M., & Bekker, H. (1992) The development of a six-item short form of the state scale of the Spielberger State-Trait Scale Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301â305.
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Mulaik, S. A. (1972). The foundations of factor analysis. New York: McGraw-Hill.
Muthén, B. O. (1989). Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557â585.
Northam, J. (1995). Henrik Ibsen. Peer Gynt. A dramatic Poem. Oslo, Norway: Scandinavian University Press.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Sanderson, F. H. (1988). Analysis of anxiety levels in sport. In D. Hackfort & C. D. Spielberger (Eds.), Anxiety in sport: An international perspective (Chap. 4). Washington, DC: Hemisphere.
Satorra, A., & Bentler, P. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the Business and Economic Statistics Section of the American Statistical Association, 308–313.
Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973â1980. British Journal of Mathematical and Statistical Psychology, 34, 133â166.
Shavelson, R. J., & Webb, N. M. (1991). Generalizabilty theory. A primer. Newbury Park: Sage.
Sipos, K. & Sipos, M. (1979). The development and validation of the Hungarian form of the State-Trait Anxiety Inventory for Children (STAIC-H). Magyar Pediater, 13(6), 47.
Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12, 102â111.
Spielberger, Ch. D. (1972). Anxiety: Current trends in theory and research (481â493). New York: Academic Press.
Spielberger, Ch. D. (1973). STAIC Preliminary Manual for the State-Trait Anxiety Inventory for Children. Palo Alto, CA: Consulting Psychologist Press.
Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229â249.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4â70.
Widaman, K. F., Little, T. D., Preacher, K. J., & Sawalani, G. M. (2011), On creating and using short forms of scales in secondary research. In K. H. Trzesniewski, M. B. Donnellan, & Lucas, R. (Eds.), Secondary data analysis (pp. 39â61). Washington, DC: American Psychological Association.
Zeidner, M. (1998). Test anxiety; The state of the art. New York: Plenum.