Creating short forms for construct measures: The role of exchangeable forms
Abstrakt
A popular trend has invaded applied psychometrics in terms of creating short forms for construct measures. There seems to be a paucity of developing methodologies for creating short forms based on complete forms that meet psychometric standards related to reliability of the scores and valid inferences. The present article suggests a methodology that rests on the fundamental assumption that the concept of short form attains meaning when being derived from valid scores of a complete form. A pivotal construct for assessing the status of short form is the concept of exchangeable forms which incorporates two types of measurement invariance; invariance across groups frequently exercised in studies applying confirmatory factor analysis, and b) invariance across random facets as estimated in generalizability theory. The two types of measurement invariance involve two types of generalizations relevant for inferring constructs; generalizing from a sample of persons to a population of persons, and generalizing from a sample of construct indicators to a universe or domain of construct indicators. In addition structural invariance is required; exchangeable short forms should relate equivalently to external reference variables.
Keywords: Valid complete form; exchangeable forms; generalizing to population of persons and universe of construct indicators; two types of measurement invariance; structural invariance.
Reference
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411â423.
https://doi.org/10.1037/0033-2909.103.3.411
Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley.
https://doi.org/10.1002/9781118619179
Brennan, R. L. (1992). Elements of generalizability theory. Revised edition. Iowa City: American College Testing.
Brennan, R. L. (2001a). Generalizability theory. New York: Springer.
https://doi.org/10.1007/978-1-4757-3456-0
Brennan, R. L. (2001b). Manual for urGENOVA. Occasional papers no. 49. Iowa Testing Programs: University of Iowa.
Brennan, R. L. (2004). Some perspectives on inconsistencies among measurement models. CASMA. Research Report No. 8. University of Iowa, Iowa City: CASMA.
Cardinet, J., Tourneur, Y., & Allal, L. (1981). Extension of generalizability theory and its application in educational measurement. Journal of Educational Measurement, 18, 183â204.
https://doi.org/10.1111/j.1745-3984.1981.tb00852.x
Carstensen, C. H. (2009). Linking PISA competencies over three cycles – results from Germany. In M. Prenzel, M. Kobarg, K. Schöps, & S. Rönnebeck (Eds.), Research on PISA (pp. 199â213). Dordrecht, Netherlands: Springer.
Cornfield, J., & Tukey, J. W. (1956). Average values of mean squares in factorials. Annals of Mathematical Statistics, 27, 907â949.
https://doi.org/10.1214/aoms/1177728067
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurement: Theory of generalizability for scores and profiles. New York: Wiley.
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137â163.
https://doi.org/10.1111/j.2044-8317.1963.tb00206.x
Dorr, D. (1981). Factor Structure of the State-Trait Anxiety Inventory for Children. Personality and Individual Differences, 2, 113â117.
https://doi.org/10.1016/0191-8869(81)90005-2
Guttman, L. (1953) A special review of Harold Gulliksen: Theory of mental tests. Psychometrika, 18, 123â130.
https://doi.org/10.1007/bf02289002
Hagtvet, K. A. (1989). The construct of test anxiety. Conceptual and methodological issues. Bergen/London: Sigma Forlag/Jessica Kingsley Publishers.
Hagtvet, K. A. (1998). Assessment of latent constructs: a joint application of generalizability theory and covariance modeling with an emphasis on inference and structure. Scandinavian Journal of Educational Research, 42, 41â63.
https://doi.org/10.1080/0031383980420103
Hagtvet, K. A., & Sipos, K. (2004). Measuring anxiety by ordered categorical items in data with subgroup structure: The case of the Hungarian version of the trait anxiety scale of the State-Trait Anxiety Inventory (STAIC-H). Anxiety, Stress, and Coping, 17, 49â67.
https://doi.org/10.1080/1061580031000151611
Hanin, Y. L., & Spielberger, C. D. (1986). The development and validation of the Russian form of the State-Trait Anxiety Inventory. In C. D. Spielberger & R. Diaz-Guerrero (Eds.), Cross-cultural anxiety (Vol. 2, pp.15â23). Washington, DC: Hemisphere.
Hedl, J. J. & Papay, J. P. (1982). The factor structure of the State-Trait Anxiety Inventory for Children: kindergarten through the fourth grades. Personality and Individual Differences, 3, 439â446.
https://doi.org/10.1016/0191-8869(82)90008-3
IBM Corp. Released 2013. IBM SPSS Statistics for Windows. Version 22.0. Armonk, NY: IBM Corp.
Ibsen, H. (1867). Peer Gynt. Et dramatisk digt. København: Gyldendalske Boghandel.
Jøreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443â477.
https://doi.org/10.1007/BF02293808
Jøreskog, K. G. (1993). Testing structural equation models. In K. A. Bollen & J. Scott Long (Eds.), Testing structural equation models (294â316). Newbury Park, CA: Sage.
Jøreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Revised version 10 February, 2005. http://www.ssicentral.com/lisrel/corner.htm
Jøreskog, K. G., & Sørbom, D. (2012). Some new features in LISREL9. Document available in Jøreskog and Sørbom (2013). Chicago, IL: Scientific Software International
Jøreskog, K. G., & Sørbom, D. (2013). LISREL (Version 9.10) [Computer software]. Chicago, IL: Scientific Software International.
Kaiser, H. F., & Michael, W. B. (1975). Domain validity and generalizability. Educational and Psychological Measurement, 35, 31â35.
https://doi.org/10.1177/001316447503500103
Kane, M. (2002). Inferences about variance components and reliability â generalizability coefficients in the absence of random sampling. Journal of Educational Measurement, 39, 165â181.
https://doi.org/10.1111/j.1745-3984.2002.tb01141.x
Lord, F. M., & Novick, M. E. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
Marcoulides, G. (1996). Estimating variance components in generalizability theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290â299.
https://doi.org/10.1080/10705519609540045
Marsh, H. W., Ellis, L. A., Parada, R. H., Richards, G., & Heubeck, B. G. (2005). A short version of the Self Description Questionnaire II: Operationalizing criteria for short-form evaluation with new applications of confirmatory factor analyses. Psychological Assessment, 17, 81â102.
https://doi.org/10.1037/1040-3590.17.1.81
Marteau, T. M., & Bekker, H. (1992) The development of a six-item short form of the state scale of the Spielberger State-Trait Scale Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301â305.
https://doi.org/10.1111/j.2044-8260.1992.tb00997.x
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Mulaik, S. A. (1972). The foundations of factor analysis. New York: McGraw-Hill.
Muthén, B. O. (1989). Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557â585.
https://doi.org/10.1007/BF02296397
Northam, J. (1995). Henrik Ibsen. Peer Gynt. A dramatic Poem. Oslo, Norway: Scandinavian University Press.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Sanderson, F. H. (1988). Analysis of anxiety levels in sport. In D. Hackfort & C. D. Spielberger (Eds.), Anxiety in sport: An international perspective (Chap. 4). Washington, DC: Hemisphere.
Satorra, A., & Bentler, P. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the Business and Economic Statistics Section of the American Statistical Association, 308–313.
Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973â1980. British Journal of Mathematical and Statistical Psychology, 34, 133â166.
https://doi.org/10.1111/j.2044-8317.1981.tb00625.x
Shavelson, R. J., & Webb, N. M. (1991). Generalizabilty theory. A primer. Newbury Park: Sage.
Sipos, K. & Sipos, M. (1979). The development and validation of the Hungarian form of the State-Trait Anxiety Inventory for Children (STAIC-H). Magyar Pediater, 13(6), 47.
Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12, 102â111.
https://doi.org/10.1037/1040-3590.12.1.102
Spielberger, Ch. D. (1972). Anxiety: Current trends in theory and research (481â493). New York: Academic Press.
https://doi.org/10.1016/B978-0-12-657401-2.50008-3
https://doi.org/10.1016/B978-0-12-657402-9.50013-2
https://doi.org/10.1016/B978-0-12-657401-2.50009-5
Spielberger, Ch. D. (1973). STAIC Preliminary Manual for the State-Trait Anxiety Inventory for Children. Palo Alto, CA: Consulting Psychologist Press.
Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229â249.
https://doi.org/10.1037/h0047980
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4â70.
https://doi.org/10.1177/109442810031002
Widaman, K. F., Little, T. D., Preacher, K. J., & Sawalani, G. M. (2011), On creating and using short forms of scales in secondary research. In K. H. Trzesniewski, M. B. Donnellan, & Lucas, R. (Eds.), Secondary data analysis (pp. 39â61). Washington, DC: American Psychological Association.
https://doi.org/10.1037/12350-003
Zeidner, M. (1998). Test anxiety; The state of the art. New York: Plenum.
