Validation of Honesty Test using Situational Judgment Test Format

Main Article Content

EunA Kwon
JongGoo Lee

Abstract

The purposes of this research were to develop the Honesty test using situation judgment test format to reduce the possibility of faking good in personality scales and to identify construct validity and concurrent validity of the developed scale. Total 359 employees(male, 46.24%; female, 53.76%) were used in this study. The Honesty test developed in this study consists of 5 to 6 items 6 items for each 4 sub-constructs (i.e., sincerity, fairness, greed avoidance and modesty) of honesty-humility factor in HEXACO model. Confirmatory factor analysis to assess the construct validity of the Honesty test resulted in favorable goodness of fit indexes for the 4-factor model(CFI = .946, TLI = .925, RMSEA = .046). Also, favorable goodness of fit indexes(CFI = .934, TLI = .911, RMSEA = .049) in the higher-order factor analysis showed that the 4 sub-constructs were converged into the single honesty-humility factor. To investigate the possibility of faking good could be reduced in the situational judgment test format compared to the Likert test format in measuring honesty, correlations between the honesty scores in each test format and the scores of faking good were compared. Although the correlations between the humility sub-constructs(i.e., greed avoidance and modesty) and faking good scores were not significantly different depending on the measurement format, the correlations between the honesty sub-constructs(i.e., sincerity and fairness) and faking good scores were significantly low in the situational judgment test format than in the Likert test format. The concurrent validity analyses revealed that the Honesty test using situational judgment test format had significant correlations with Machiavellism personality(r = -.21) and self-control(r = .27). Limitations of the study and directions for future study were discussed.

Metrics

Metrics Loading ...

Article Details

How to Cite
Kwon, E., & Lee, J. (2020). Validation of Honesty Test using Situational Judgment Test Format. Korean Journal of Industrial and Organizational Psychology, 33(4), 545–569. https://doi.org/10.24230/kjiop.v33i4.545-569
Section
Empirical Articles
Author Biographies

EunA Kwon, Daegu University

Department of Psychology

JongGoo Lee, Daegu University

Department of Psychology

Funding data

References

Ahmed, S. M., & Stewart, R. A. (1981). Factor analysis of the Machiavellian scale. Social Behavior and Personality, 9(1), 113-115.https://doi.org/10.2224/sbp.1981.9.1.113 DOI: https://doi.org/10.2224/sbp.1981.9.1.113

Arthur Jr. W., & Villado, A. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection and research in practice. Journal of Applied Psychology, 93(2), 435-442.https://doi.org/10.1037/0021-9010.93.2.435 DOI: https://doi.org/10.1037/0021-9010.93.2.435

Bartram, D. (1996). The relationship between ipsatized and normative measures of personality. Journal of Occupational and Organizational Psychology, 69(1), 25-39.https://doi.org/10.1111/j.2044-8325.1996.tb00597.x DOI: https://doi.org/10.1111/j.2044-8325.1996.tb00597.x

Bartram, D. (2005). The great eight competencies: Acriterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185-1203. https://doi.org/10.1037/0021-9010.90.6.1185 DOI: https://doi.org/10.1037/0021-9010.90.6.1185

Bowen, C. C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. The International Journal of Organizational Analysis, 10(3), 240-259.https://doi.org/10.1108/eb028952 DOI: https://doi.org/10.1108/eb028952

Campion, M. C., & Ployhart, R. E. (2013). Assessing personality with situational judgment measures. In N. D. Christiansen & R. P. Tett (Eds.), Handbook of personality at work (pp. 439-456). NY: Routledge.

ISBN-10: 1848729421

Chung, H. S., & Park, D. G. (2009). The study of criterion-related validities of different combinations of keying and scoring methods in situational judgment test with knowledge instructions. Korean Journal of Industrial and Organizational Psychology, 22(1), 53-70.httpps://doi.org/10.24230/ksiop.22.1.200902.53 DOI: https://doi.org/10.24230/ksiop.22.1.200902.53

Conway, J. S. (2014). The invention of lying (at work): The development and validation of a situational judgment. Doctoral Dissertation. University of South Florida, Florida. USA. Retrieved from https://scholarcommons.usf.edu/etd/5204

Dahling, J. J., Whitaker, B. G., & Levy, P. E. (2009). The development and validation of a new Machiavellianism scale. Journal of Management, 35(2), 219-257.https://doi.org/10.1177/0149206308318618 DOI: https://doi.org/10.1177/0149206308318618

Diamond, A. (2013) Executive functions. Annual Review of Psychology, 64, 135-168.https://doi.org/10.1146/annurev-psych-113011-143750 DOI: https://doi.org/10.1146/annurev-psych-113011-143750

Finkenauer, C., Engels, R., & Baumeister, R. (2005). Parenting behaviour and adolescent behavioural and emotional problems: The role of self-control. International Journal of Behavioral Development, 29(1), 58-69.https://doi.org/10.1080/01650250444000333 DOI: https://doi.org/10.1080/01650250444000333

Gozna, L. F., Vrij, A., & Bull, R. (2001). The impact of individual differences on perceptions of lying in everyday life and in a high stake situation. Personality and Individual Differences, 31(7), 1203-1216.https://doi.org/10.1016/S0191-8869(00)00219-1 DOI: https://doi.org/10.1016/S0191-8869(00)00219-1

Han, H. S. (2007). Self-control and ego-resiliency as chief self-regulatory variables. Master’s Dissertation. Korea University. Retrieved fromhttp://www.riss.kr/link?id=T11104643

Hargrave, G. E., & Hiatt, D. (1989). Use of the California Psychological Inventory in law enforcement officer selection. Journal of Personality Assessment, 53(2), 267-277.https://doi.org/10.1207/s15327752jpa5302_5 DOI: https://doi.org/10.1207/s15327752jpa5302_5

Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88(1), 100-112.https://doi.org/10.1037/0021-9010.88.1.100 DOI: https://doi.org/10.1037/0021-9010.88.1.100

Hooper, A. C., Cullen, M. J., & Sackett, P. R. (2006). Operational threats to the use of SJTs: Faking, coaching, and retesting issues. In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 205-232). NJ: Erlbaum. ISBN-10: 0805852514

Kang, M. U. (2005). Comparison of validities in accordance with response instructions and scoring algorithms in situational judgement tests. Master Dissertation. Sungkyunkwan University, Seoul, Korea.

Retrieved from http://dll.kr/azP

Kanning, U. P., & Kuhne, S. (2006). Social desirability in a multimodal personnel selection test battery. European Journal of Work and Organizational Psychology, 15(3), 241-261.https://doi.org/10.1080/13594320600625872 DOI: https://doi.org/10.1080/13594320600625872

Kashy, D. A., & DePaulo, B. M. (1996). Who lies? Journal of Personality and Social Psychology, 70(5), 1037-1051.https://doi.org/10.1037/0022-3514.70.5.1037 DOI: https://doi.org/10.1037/0022-3514.70.5.1037

Kasten, N., Freund, P. A., & Staufenbiel, T. (2020). “Sweet little lies”: An in-depth analysis of faking behavior on situational judgment tests compared to personality questionnaires. European Journal of Psychological Assessment, 36(1), 136-148.https://doi.org/10.1027/1015-5759/a000479 DOI: https://doi.org/10.1027/1015-5759/a000479

Kim, E. S. (2010). Comparison of validities for scoring keys and scoring algorithms in situational judgment test: the influence of faking. Master Dissertation. Hoseo University, Cheonan, Korea.

Retrieved from http://dll.kr/azN

Kim, E. S., Han, Y. S., & Kim, M. S. (2011). Comparison of validities for scoring keys and scoring algorithms in situational judgment test: The influence of faking. Korean Journal of Industrial and Organizational Psychology, 24(1), 231-255.httpps://doi.org/10.24230/ksiop.24.1.201102.231 DOI: https://doi.org/10.24230/kjiop.v24i1.231-255

Kim, H. S., Hong, H. G., & Hyun, M. H. (2011). Validity and Reliability of the Korean Version of the Machiavellianism Personality Scale(K-MPS). The Korean Journal of Stress Research, 19(1), 21-30.

Retrieved from https://url.kr/8NOyGo

Kim, M. J. (1998). Measurement approach for the individual conflict handing in the organization. Doctoral Dissertation. Yonsei University. Retrieved fromhttp://www.riss.kr/link?id=T7125112

Kim, S. Y. (2017). Fundamentals and extensions of structural equation modeling. Seoul: Hakjisa. ISBN-10: 8999709671

Kopp, C. B. (1982). Antecedent of self-regulation: A developmental perspective. Developmental Psychology, 18(2), 199-214.https://doi.org/10.1037/0012-1649.18.2.199 DOI: https://doi.org/10.1037/0012-1649.18.2.199

Lee, J. G., & Han, Y. S. (2008). The reliability and validity of tele-marketer selecting competency tests in different formats: A comparison between ipsative and normative measures. Korean Journal of Industrial and Organizational Psychology, 21(3), 411-428. httpps://doi.org/10.24230/ksiop.21.3.200808.411 DOI: https://doi.org/10.24230/ksiop.21.3.200808.411

Lee, J. H. (2007). Validation of behavior tendency situational judgment tests: Validity based on structural equation modeling. Master’s Dissertation. Sungkyunkwan University. Retrieved fromhttp://www.riss.kr/link?id=T11043274

Lee, K. B., & Ashton, M. C. (2013). The h factor of personality. Seoul: Moonye publishing. ISBN-10: 8931007442

Lee, K. B., Ashton, M. C., & de Vries, R. E. (2005). Predicting workplace delinquency and integrity with the HEXACO and five-factor models of personality structure. Human Performance, 18(2), 179-197.

https://doi.org/10.1207/s15327043hup1802_4 DOI: https://doi.org/10.1207/s15327043hup1802_4

Lee, K., & Ashton, M. C. (2004). Psychometric proporties of the HEXACO personality inventory. Multivariate Behavioral Research, 39(2), 329-358.https://doi.org/10.1207/s15327906mbr3902_8 DOI: https://doi.org/10.1207/s15327906mbr3902_8

Lee, K., Ashton, M. C., & Shin, K. H. (2005). Personality correlates of workplace anti-social behavior. Applied Psychology: An International Review, 54(1), 81-98.https://doi.org/10.1111/j.1464-0597.2005.00197.x DOI: https://doi.org/10.1111/j.1464-0597.2005.00197.x

Lee, S. C., Lee, S. M., & Jo. Y. I. (2003). Critical review on paper and pencil situational judgement test. Korean Journal of Social and Personality Psychology, 16(3), 129-154. Retrieved from http://bitly.kr/7MEKrkaC38

Lee, S. J. (2010). Development of a situational judgement test type integrity test and investigation of based on internal structure. Master Dissertation. Sungkyunkwan University, Seoul, Korea.

Retrieved from http://dll.kr/azM

McDaniel, M. A. & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1-2), 103-113.https://doi.org/10.1111/1468-2389.00167 DOI: https://doi.org/10.1111/1468-2389.00167

McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730-740.https://doi.org/10.1037/0021-9010.86.4.730 DOI: https://doi.org/10.1037/0021-9010.86.4.730

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75(6), 640-647.https://doi.org/10.1037/0021-9010.75.6.640 DOI: https://doi.org/10.1037/0021-9010.75.6.640

Mumford, M. D., Connelly, M. S., Helton, W. B., Strange, J. M., & Osburn, H. K. (2001). On the construct validity of integrity tests: Individual and situational factors as predictors of test performance. International Journal of Selection and Assessment, 9(3), 240-257.https://doi.org/10.1111/1468-2389.00177 DOI: https://doi.org/10.1111/1468-2389.00177

Mussel, P., Gatzka, T., & Hewig, J. (2018). Situational judgment tests as an alternative measure for personality assessment. European Journal of Psychological Assessment, 34(5), 328- 335.https://doi.org/10.1027/1015-5759/a000346 DOI: https://doi.org/10.1027/1015-5759/a000346

Namgung, J. J. (2010). Development and validation of a situational judgment test for high school students in the academic domain. Master Dissertation. Sungkyunkwan University, Seoul, Korea.

Retrieved from http://dll.kr/azQ

Nguyen, N. T. (2002). Faking in situational judgment tests: An empirical investigation of the work judgment survey. Doctoral Dissertation. Virginia Commonwealth University, Virginia. USA. Retrieved fromhttps://search.proquest.com/docview/304788837?accountid=48028

Nguyen, N. T., Biderman, M. D., & McDaniel, M. A. (2005). Effects of response instructions on faking a situational judgment test. International Journal of Selection and Assessment, 13(4), 250-260.https://doi.org/10.1111/j.1468-2389.2005.00322.x DOI: https://doi.org/10.1111/j.1468-2389.2005.00322.x

O’Boyle Jr, E. H., Forsyth, D. R., Banks, G. C., & McDaniel, M. A. (2012). A meta-analysis of the dark triad and work behavior: A social exchange perspective. Journal of Applied Psychology, 97(3), 557-579.https://doi.org/10.1037/a0025679 DOI: https://doi.org/10.1037/a0025679

Park, J. H. (2007). A validation study of competency factors using partially ipsative measures. Master’s Dissertation. Deagu University. Retrieved from http://www.riss.kr/link?id=T11293193

Paulhus, D. L., & Williams, K. M. (2002). The dark triad of personality: Narcissism, Machiavellianism, and psychopathy. Journal of Research in Personality, 36(6), 556-563.https://doi.org/10.1016/S0092-6566(02)00505-6 DOI: https://doi.org/10.1016/S0092-6566(02)00505-6

Roberts, B. W., Chernyshenko, O. S., Stark, S., & Goldberg, L. R. (2005). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires. Personnel Psychology, 58(1), 103-139.https://doi.org/10.1111/j.1744-6570.2005.00301.x DOI: https://doi.org/10.1111/j.1744-6570.2005.00301.x

Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of black-white differences in overall and exercise scores. Personnel Psychology, 61(3), 637-661.https://doi.org/10.1111/j.1744-6570.2008.00125.x DOI: https://doi.org/10.1111/j.1744-6570.2008.00125.x

Sarchione, C. D., Cuttler, M. J., Muchinsky, P. M., & Nelson-Gray, R. O. (1998). Prediction of dysfunctional job behaviors among law enforcement officers. Journal of Applied Psychology, 83(6), 904-912.https://doi.org/10.1037/0021-9010.83.6.904 DOI: https://doi.org/10.1037/0021-9010.83.6.904

Schmidt, F. L., & Hunter, J. E. (1993). Tacit knowledge, practical intelligence, general mental ability, and job knowledge. Current Directions in Psychological Science, 2(1), 8-9. https://doi.org/10.1111/1467-8721.ep10770456 DOI: https://doi.org/10.1111/1467-8721.ep10770456

Schmitt, N., & Chan, D. (2006). Situational judgment tests: Method or construct? In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 135-156). NJ: Erlbaum. ISBN-10: 0805852514

Snell, A. F., Sydell, E. J., & Lueke, S. B. (1999). Towards a theory of applicant faking: Integrating studies of deception. Human Resource Management Review, 9(2), 219-242.https://doi.org/10.1016/S1053-4822(99)00019-4 DOI: https://doi.org/10.1016/S1053-4822(99)00019-4

Song, M. J. (2008). Developmental Psychology. Seoul: Hakjisa. ISBN: 978-89-5891-855-4 93180

Sternberg, R. J., & Wagner, R. K. (1993). The g-ocentric view of intelligence and job performance is wrong. Current Directions in Psychological Science, 2(1), 1-4.https://doi.org/10.1111/1467-8721.ep10770441 DOI: https://doi.org/10.1111/1467-8721.ep10770441

Tangney, J. P., Baumeister, R. F., & Boone, A. L. (2004). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. Journal of Personality, 72(2), 271-322.https://doi.org/10.1111/j.0022-3506.2004.00263.x. DOI: https://doi.org/10.1111/j.0022-3506.2004.00263.x

Timpano, K. R., & Schmidt, N. B. (2013). The relationship between self control deficits and hoarding: A multimethod investigation across three samples. Journal of Abnormal Psychology, 122(1), 13-25.https://doi.org/10.1037/a0029760 DOI: https://doi.org/10.1037/a0029760

Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50(1), 25-49.https://doi.org/10.1111/j.1744-6570.1997.tb00899.x DOI: https://doi.org/10.1111/j.1744-6570.1997.tb00899.x

Wendler, K., Liu, J., & Zettler, I. (2018). Honesty-humility interacts with context perception in predicting task performance and organizational citizenship behavior. Journal of Personnel Psychology, 17(4), 161-171.https://doi.org/10.1027/1866-5888/a000203. DOI: https://doi.org/10.1027/1866-5888/a000203

White, G. L. (1984) Comparison of four jealousy scales. Journal of Research in Personality, 18(2), 115-130.https://doi.org/10.1016/0092-6566(84)90024-2 DOI: https://doi.org/10.1016/0092-6566(84)90024-2

Yoo, T. Y., Lee, K. B., & Ashton, M. C. (2004). Psychometric properties of the Korean version of the HEXACO personality inventory. Korean Journal of Social and Personality Psychology, 18(3), 61-75. Retrieved fromhttp://www.riss.kr/link?id=A100629491