(CZ) Creating Short Forms for Construct Measures: The role of exchangeable forms
(EN) Creating Short Forms for Construct Measures: The role of exchangeable forms

A popular trend has invaded applied psychometrics in a broad range of social science, in particular in research fields of educational psychology, in terms of creating short forms for construct measures. There seems to be a paucity of developing methodologies for creating short forms based on complete forms that meet psychometric standards related to the reliability of scores and valid inferences. The present article suggests a methodology that rests on the fundamental assumption that the concept of a short form attains meaning when derived from valid scores of a complete form. A pivotal construct for assessing the status of a short form is the concept of exchangeable forms, which incorporates two types of measurement invariance; a) invariance across groups, frequently exercised in studies applying confirmatory factor analysis, and b) invariance across random facets, as estimated in generalizability theory. The two types of measurement invariance involve two types of generalizations relevant for inferring constructs; generalizing from a sample of persons to a population of persons, and generalizing from a sample of construct indicators to a universe or domain of construct indicators. In addition, structural invariance is required; exchangeable short forms should relate equivalently to external reference variables. The Hungarian version of the State-Trait Anxiety Inventory for Children (STAIC-H) was used to illustrate the suggested short form methodology.

A popular trend has invaded applied psychometrics in a broad range of social science, in particular in research fields of educational psychology, in terms of creating short forms for construct measures. There seems to be a paucity of developing methodologies for creating short forms based on complete forms that meet psychometric standards related to the reliability of scores and valid inferences. The present article suggests a methodology that rests on the fundamental assumption that the concept of a short form attains meaning when derived from valid scores of a complete form. A pivotal construct for assessing the status of a short form is the concept of exchangeable forms, which incorporates two types of measurement invariance; a) invariance across groups, frequently exercised in studies applying confirmatory factor analysis, and b) invariance across random facets, as estimated in generalizability theory. The two types of measurement invariance involve two types of generalizations relevant for inferring constructs; generalizing from a sample of persons to a population of persons, and generalizing from a sample of construct indicators to a universe or domain of construct indicators. In addition, structural invariance is required; exchangeable short forms should relate equivalently to external reference variables. The Hungarian version of the State-Trait Anxiety Inventory for Children (STAIC-H) was used to illustrate the suggested short form methodology.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411−423.
https://doi.org/10.1037/0033-2909.103.3.411

Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley.
https://doi.org/10.1002/9781118619179

Brennan, R. L. (1992). Elements of generalizability theory. Revised edition. Iowa City: American College Testing.

Brennan, R. L. (2001a). Generalizability theory. New York: Springer.
https://doi.org/10.1007/978-1-4757-3456-0

Brennan, R. L. (2001b). Manual for urGENOVA. Occasional papers no. 49. Iowa Testing Programs: University of Iowa.

Brennan, R. L. (2004). Some perspectives on inconsistencies among measurement models. CASMA. Research Report No. 8. University of Iowa, Iowa City: CASMA.

Cardinet, J., Tourneur, Y., & Allal, L. (1981). Extension of generalizability theory and its application in educational measurement. Journal of Educational Measurement, 18, 183−204.
https://doi.org/10.1111/j.1745-3984.1981.tb00852.x

Carstensen, C. H. (2009). Linking PISA competencies over three cycles – results from Germany. In M. Prenzel, M. Kobarg, K. Schöps, & S. Rönnebeck (Eds.), Research on PISA (pp. 199−213). Dordrecht, Netherlands: Springer.

Cornfield, J., & Tukey, J. W. (1956). Average values of mean squares in factorials. Annals of Mathematical Statistics, 27, 907−949.
https://doi.org/10.1214/aoms/1177728067

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurement: Theory of generalizability for scores and profiles. New York: Wiley.

Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137−163.
https://doi.org/10.1111/j.2044-8317.1963.tb00206.x

Dorr, D. (1981). Factor Structure of the State-Trait Anxiety Inventory for Children. Personality and Individual Differences, 2, 113−117.
https://doi.org/10.1016/0191-8869(81)90005-2

Guttman, L. (1953) A special review of Harold Gulliksen: Theory of mental tests. Psychometrika, 18, 123−130.
https://doi.org/10.1007/bf02289002

Hagtvet, K. A. (1989). The construct of test anxiety. Conceptual and methodological issues. Bergen/London: Sigma Forlag/Jessica Kingsley Publishers.

Hagtvet, K. A. (1998). Assessment of latent constructs: a joint application of generalizability theory and covariance modeling with an emphasis on inference and structure. Scandinavian Journal of Educational Research, 42, 41−63.
https://doi.org/10.1080/0031383980420103

Hagtvet, K. A., & Sipos, K. (2004). Measuring anxiety by ordered categorical items in data with subgroup structure: The case of the Hungarian version of the trait anxiety scale of the State-Trait Anxiety Inventory (STAIC-H). Anxiety, Stress, and Coping, 17, 49−67.
https://doi.org/10.1080/1061580031000151611

Hanin, Y. L., & Spielberger, C. D. (1986). The development and validation of the Russian form of the State-Trait Anxiety Inventory. In C. D. Spielberger & R. Diaz-Guerrero (Eds.), Cross-cultural anxiety (Vol. 2, pp.15−23). Washington, DC: Hemisphere.

Hedl, J. J. & Papay, J. P. (1982). The factor structure of the State-Trait Anxiety Inventory for Children: kindergarten through the fourth grades. Personality and Individual Differences, 3, 439−446.
https://doi.org/10.1016/0191-8869(82)90008-3

IBM Corp. Released 2013. IBM SPSS Statistics for Windows. Version 22.0. Armonk, NY: IBM Corp.

Ibsen, H. (1867). Peer Gynt. Et dramatisk digt. København: Gyldendalske Boghandel.

Jøreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443−477.
https://doi.org/10.1007/BF02293808

Jøreskog, K. G. (1993). Testing structural equation models. In K. A. Bollen & J. Scott Long (Eds.), Testing structural equation models (294−316). Newbury Park, CA: Sage.

Jøreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Revised version 10 February, 2005. http://www.ssicentral.com/lisrel/corner.htm

Jøreskog, K. G., & Sørbom, D. (2012). Some new features in LISREL9. Document available in Jøreskog and Sørbom (2013). Chicago, IL: Scientific Software International

Jøreskog, K. G., & Sørbom, D. (2013). LISREL (Version 9.10) [Computer software]. Chicago, IL: Scientific Software International.

Kaiser, H. F., & Michael, W. B. (1975). Domain validity and generalizability. Educational and Psychological Measurement, 35, 31−35.
https://doi.org/10.1177/001316447503500103

Kane, M. (2002). Inferences about variance components and reliability − generalizability coefficients in the absence of random sampling. Journal of Educational Measurement, 39, 165−181.
https://doi.org/10.1111/j.1745-3984.2002.tb01141.x

Lord, F. M., & Novick, M. E. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.

Marcoulides, G. (1996). Estimating variance components in generalizability theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290−299.
https://doi.org/10.1080/10705519609540045

Marsh, H. W., Ellis, L. A., Parada, R. H., Richards, G., & Heubeck, B. G. (2005). A short version of the Self Description Questionnaire II: Operationalizing criteria for short-form evaluation with new applications of confirmatory factor analyses. Psychological Assessment, 17, 81−102.
https://doi.org/10.1037/1040-3590.17.1.81

Marteau, T. M., & Bekker, H. (1992) The development of a six-item short form of the state scale of the Spielberger State-Trait Scale Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301−305.
https://doi.org/10.1111/j.2044-8260.1992.tb00997.x

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.

Mulaik, S. A. (1972). The foundations of factor analysis. New York: McGraw-Hill.

Muthén, B. O. (1989). Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557−585.
https://doi.org/10.1007/BF02296397

Northam, J. (1995). Henrik Ibsen. Peer Gynt. A dramatic Poem. Oslo, Norway: Scandinavian University Press.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Sanderson, F. H. (1988). Analysis of anxiety levels in sport. In D. Hackfort & C. D. Spielberger (Eds.), Anxiety in sport: An international perspective (Chap. 4). Washington, DC: Hemisphere.

Satorra, A., & Bentler, P. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the Business and Economic Statistics Section of the American Statistical Association, 308–313.

Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973−1980. British Journal of Mathematical and Statistical Psychology, 34, 133−166.
https://doi.org/10.1111/j.2044-8317.1981.tb00625.x

Shavelson, R. J., & Webb, N. M. (1991). Generalizabilty theory. A primer. Newbury Park: Sage.

Sipos, K. & Sipos, M. (1979). The development and validation of the Hungarian form of the State-Trait Anxiety Inventory for Children (STAIC-H). Magyar Pediater, 13(6), 47.

Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12, 102−111.
https://doi.org/10.1037/1040-3590.12.1.102

Spielberger, Ch. D. (1972). Anxiety: Current trends in theory and research (481−493). New York: Academic Press.
https://doi.org/10.1016/B978-0-12-657401-2.50008-3
https://doi.org/10.1016/B978-0-12-657402-9.50013-2
https://doi.org/10.1016/B978-0-12-657401-2.50009-5

Spielberger, Ch. D. (1973). STAIC Preliminary Manual for the State-Trait Anxiety Inventory for Children. Palo Alto, CA: Consulting Psychologist Press.

Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229−249.
https://doi.org/10.1037/h0047980

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4−70.
https://doi.org/10.1177/109442810031002

Widaman, K. F., Little, T. D., Preacher, K. J., & Sawalani, G. M. (2011), On creating and using short forms of scales in secondary research. In K. H. Trzesniewski, M. B. Donnellan, & Lucas, R. (Eds.), Secondary data analysis (pp. 39−61). Washington, DC: American Psychological Association.
https://doi.org/10.1037/12350-003

Zeidner, M. (1998). Test anxiety; The state of the art. New York: Plenum.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411−423.
https://doi.org/10.1037/0033-2909.103.3.411

Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley.
https://doi.org/10.1002/9781118619179

Brennan, R. L. (1992). Elements of generalizability theory. Revised edition. Iowa City: American College Testing.

Brennan, R. L. (2001a). Generalizability theory. New York: Springer.
https://doi.org/10.1007/978-1-4757-3456-0

Brennan, R. L. (2001b). Manual for urGENOVA. Occasional papers no. 49. Iowa Testing Programs: University of Iowa.

Brennan, R. L. (2004). Some perspectives on inconsistencies among measurement models. CASMA. Research Report No. 8. University of Iowa, Iowa City: CASMA.

Cardinet, J., Tourneur, Y., & Allal, L. (1981). Extension of generalizability theory and its application in educational measurement. Journal of Educational Measurement, 18, 183−204.
https://doi.org/10.1111/j.1745-3984.1981.tb00852.x

Carstensen, C. H. (2009). Linking PISA competencies over three cycles – results from Germany. In M. Prenzel, M. Kobarg, K. Schöps, & S. Rönnebeck (Eds.), Research on PISA (pp. 199−213). Dordrecht, Netherlands: Springer.

Cornfield, J., & Tukey, J. W. (1956). Average values of mean squares in factorials. Annals of Mathematical Statistics, 27, 907−949.
https://doi.org/10.1214/aoms/1177728067

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurement: Theory of generalizability for scores and profiles. New York: Wiley.

Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137−163.
https://doi.org/10.1111/j.2044-8317.1963.tb00206.x

Dorr, D. (1981). Factor Structure of the State-Trait Anxiety Inventory for Children. Personality and Individual Differences, 2, 113−117.
https://doi.org/10.1016/0191-8869(81)90005-2

Guttman, L. (1953) A special review of Harold Gulliksen: Theory of mental tests. Psychometrika, 18, 123−130.
https://doi.org/10.1007/bf02289002

Hagtvet, K. A. (1989). The construct of test anxiety. Conceptual and methodological issues. Bergen/London: Sigma Forlag/Jessica Kingsley Publishers.

Hagtvet, K. A. (1998). Assessment of latent constructs: a joint application of generalizability theory and covariance modeling with an emphasis on inference and structure. Scandinavian Journal of Educational Research, 42, 41−63.
https://doi.org/10.1080/0031383980420103

Hagtvet, K. A., & Sipos, K. (2004). Measuring anxiety by ordered categorical items in data with subgroup structure: The case of the Hungarian version of the trait anxiety scale of the State-Trait Anxiety Inventory (STAIC-H). Anxiety, Stress, and Coping, 17, 49−67.
https://doi.org/10.1080/1061580031000151611

Hanin, Y. L., & Spielberger, C. D. (1986). The development and validation of the Russian form of the State-Trait Anxiety Inventory. In C. D. Spielberger & R. Diaz-Guerrero (Eds.), Cross-cultural anxiety (Vol. 2, pp.15−23). Washington, DC: Hemisphere.

Hedl, J. J. & Papay, J. P. (1982). The factor structure of the State-Trait Anxiety Inventory for Children: kindergarten through the fourth grades. Personality and Individual Differences, 3, 439−446.
https://doi.org/10.1016/0191-8869(82)90008-3

IBM Corp. Released 2013. IBM SPSS Statistics for Windows. Version 22.0. Armonk, NY: IBM Corp.

Ibsen, H. (1867). Peer Gynt. Et dramatisk digt. København: Gyldendalske Boghandel.

Jøreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443−477.
https://doi.org/10.1007/BF02293808

Jøreskog, K. G. (1993). Testing structural equation models. In K. A. Bollen & J. Scott Long (Eds.), Testing structural equation models (294−316). Newbury Park, CA: Sage.

Jøreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Revised version 10 February, 2005. http://www.ssicentral.com/lisrel/corner.htm

Jøreskog, K. G., & Sørbom, D. (2012). Some new features in LISREL9. Document available in Jøreskog and Sørbom (2013). Chicago, IL: Scientific Software International

Jøreskog, K. G., & Sørbom, D. (2013). LISREL (Version 9.10) [Computer software]. Chicago, IL: Scientific Software International.

Kaiser, H. F., & Michael, W. B. (1975). Domain validity and generalizability. Educational and Psychological Measurement, 35, 31−35.
https://doi.org/10.1177/001316447503500103

Kane, M. (2002). Inferences about variance components and reliability − generalizability coefficients in the absence of random sampling. Journal of Educational Measurement, 39, 165−181.
https://doi.org/10.1111/j.1745-3984.2002.tb01141.x

Lord, F. M., & Novick, M. E. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.

Marcoulides, G. (1996). Estimating variance components in generalizability theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290−299.
https://doi.org/10.1080/10705519609540045

Marsh, H. W., Ellis, L. A., Parada, R. H., Richards, G., & Heubeck, B. G. (2005). A short version of the Self Description Questionnaire II: Operationalizing criteria for short-form evaluation with new applications of confirmatory factor analyses. Psychological Assessment, 17, 81−102.
https://doi.org/10.1037/1040-3590.17.1.81

Marteau, T. M., & Bekker, H. (1992) The development of a six-item short form of the state scale of the Spielberger State-Trait Scale Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301−305.
https://doi.org/10.1111/j.2044-8260.1992.tb00997.x

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.

Mulaik, S. A. (1972). The foundations of factor analysis. New York: McGraw-Hill.

Muthén, B. O. (1989). Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557−585.
https://doi.org/10.1007/BF02296397

Northam, J. (1995). Henrik Ibsen. Peer Gynt. A dramatic Poem. Oslo, Norway: Scandinavian University Press.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Sanderson, F. H. (1988). Analysis of anxiety levels in sport. In D. Hackfort & C. D. Spielberger (Eds.), Anxiety in sport: An international perspective (Chap. 4). Washington, DC: Hemisphere.

Satorra, A., & Bentler, P. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the Business and Economic Statistics Section of the American Statistical Association, 308–313.

Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973−1980. British Journal of Mathematical and Statistical Psychology, 34, 133−166.
https://doi.org/10.1111/j.2044-8317.1981.tb00625.x

Shavelson, R. J., & Webb, N. M. (1991). Generalizabilty theory. A primer. Newbury Park: Sage.

Sipos, K. & Sipos, M. (1979). The development and validation of the Hungarian form of the State-Trait Anxiety Inventory for Children (STAIC-H). Magyar Pediater, 13(6), 47.

Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12, 102−111.
https://doi.org/10.1037/1040-3590.12.1.102

Spielberger, Ch. D. (1972). Anxiety: Current trends in theory and research (481−493). New York: Academic Press.
https://doi.org/10.1016/B978-0-12-657401-2.50008-3
https://doi.org/10.1016/B978-0-12-657402-9.50013-2
https://doi.org/10.1016/B978-0-12-657401-2.50009-5

Spielberger, Ch. D. (1973). STAIC Preliminary Manual for the State-Trait Anxiety Inventory for Children. Palo Alto, CA: Consulting Psychologist Press.

Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229−249.
https://doi.org/10.1037/h0047980

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4−70.
https://doi.org/10.1177/109442810031002

Widaman, K. F., Little, T. D., Preacher, K. J., & Sawalani, G. M. (2011), On creating and using short forms of scales in secondary research. In K. H. Trzesniewski, M. B. Donnellan, & Lucas, R. (Eds.), Secondary data analysis (pp. 39−61). Washington, DC: American Psychological Association.
https://doi.org/10.1037/12350-003

Zeidner, M. (1998). Test anxiety; The state of the art. New York: Plenum.