Validation of Honesty Test using Situational Judgment Test Format
Article Sidebar
Main Article Content
Abstract
The purposes of this research were to develop the Honesty test using situation judgment test format to reduce the possibility of faking good in personality scales and to identify construct validity and concurrent validity of the developed scale. Total 359 employees(male, 46.24%; female, 53.76%) were used in this study. The Honesty test developed in this study consists of 5 to 6 items 6 items for each 4 sub-constructs (i.e., sincerity, fairness, greed avoidance and modesty) of honesty-humility factor in HEXACO model. Confirmatory factor analysis to assess the construct validity of the Honesty test resulted in favorable goodness of fit indexes for the 4-factor model(CFI = .946, TLI = .925, RMSEA = .046). Also, favorable goodness of fit indexes(CFI = .934, TLI = .911, RMSEA = .049) in the higher-order factor analysis showed that the 4 sub-constructs were converged into the single honesty-humility factor. To investigate the possibility of faking good could be reduced in the situational judgment test format compared to the Likert test format in measuring honesty, correlations between the honesty scores in each test format and the scores of faking good were compared. Although the correlations between the humility sub-constructs(i.e., greed avoidance and modesty) and faking good scores were not significantly different depending on the measurement format, the correlations between the honesty sub-constructs(i.e., sincerity and fairness) and faking good scores were significantly low in the situational judgment test format than in the Likert test format. The concurrent validity analyses revealed that the Honesty test using situational judgment test format had significant correlations with Machiavellism personality(r = -.21) and self-control(r = .27). Limitations of the study and directions for future study were discussed.
Metrics
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Funding data
-
Daegu University
Grant numbers 20170316
References
Ahmed, S. M., & Stewart, R. A. (1981). Factor analysis of the Machiavellian scale. Social Behavior and Personality, 9(1), 113-115.https://doi.org/10.2224/sbp.1981.9.1.113
Arthur Jr. W., & Villado, A. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection and research in practice. Journal of Applied Psychology, 93(2), 435-442.https://doi.org/10.1037/0021-9010.93.2.435
Bartram, D. (1996). The relationship between ipsatized and normative measures of personality. Journal of Occupational and Organizational Psychology, 69(1), 25-39.https://doi.org/10.1111/j.2044-8325.1996.tb00597.x
Bartram, D. (2005). The great eight competencies: Acriterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185-1203. https://doi.org/10.1037/0021-9010.90.6.1185
Bowen, C. C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. The International Journal of Organizational Analysis, 10(3), 240-259.https://doi.org/10.1108/eb028952
Campion, M. C., & Ployhart, R. E. (2013). Assessing personality with situational judgment measures. In N. D. Christiansen & R. P. Tett (Eds.), Handbook of personality at work (pp. 439-456). NY: Routledge.
ISBN-10: 1848729421
Chung, H. S., & Park, D. G. (2009). The study of criterion-related validities of different combinations of keying and scoring methods in situational judgment test with knowledge instructions. Korean Journal of Industrial and Organizational Psychology, 22(1), 53-70.httpps://doi.org/10.24230/ksiop.22.1.200902.53
Conway, J. S. (2014). The invention of lying (at work): The development and validation of a situational judgment. Doctoral Dissertation. University of South Florida, Florida. USA. Retrieved from https://scholarcommons.usf.edu/etd/5204
Dahling, J. J., Whitaker, B. G., & Levy, P. E. (2009). The development and validation of a new Machiavellianism scale. Journal of Management, 35(2), 219-257.https://doi.org/10.1177/0149206308318618
Diamond, A. (2013) Executive functions. Annual Review of Psychology, 64, 135-168.https://doi.org/10.1146/annurev-psych-113011-143750
Finkenauer, C., Engels, R., & Baumeister, R. (2005). Parenting behaviour and adolescent behavioural and emotional problems: The role of self-control. International Journal of Behavioral Development, 29(1), 58-69.https://doi.org/10.1080/01650250444000333
Gozna, L. F., Vrij, A., & Bull, R. (2001). The impact of individual differences on perceptions of lying in everyday life and in a high stake situation. Personality and Individual Differences, 31(7), 1203-1216.https://doi.org/10.1016/S0191-8869(00)00219-1
Han, H. S. (2007). Self-control and ego-resiliency as chief self-regulatory variables. Master’s Dissertation. Korea University. Retrieved fromhttp://www.riss.kr/link?id=T11104643
Hargrave, G. E., & Hiatt, D. (1989). Use of the California Psychological Inventory in law enforcement officer selection. Journal of Personality Assessment, 53(2), 267-277.https://doi.org/10.1207/s15327752jpa5302_5
Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88(1), 100-112.https://doi.org/10.1037/0021-9010.88.1.100
Hooper, A. C., Cullen, M. J., & Sackett, P. R. (2006). Operational threats to the use of SJTs: Faking, coaching, and retesting issues. In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 205-232). NJ: Erlbaum. ISBN-10: 0805852514
Kang, M. U. (2005). Comparison of validities in accordance with response instructions and scoring algorithms in situational judgement tests. Master Dissertation. Sungkyunkwan University, Seoul, Korea.
Retrieved from http://dll.kr/azP
Kanning, U. P., & Kuhne, S. (2006). Social desirability in a multimodal personnel selection test battery. European Journal of Work and Organizational Psychology, 15(3), 241-261.https://doi.org/10.1080/13594320600625872
Kashy, D. A., & DePaulo, B. M. (1996). Who lies? Journal of Personality and Social Psychology, 70(5), 1037-1051.https://doi.org/10.1037/0022-3514.70.5.1037
Kasten, N., Freund, P. A., & Staufenbiel, T. (2020). “Sweet little lies”: An in-depth analysis of faking behavior on situational judgment tests compared to personality questionnaires. European Journal of Psychological Assessment, 36(1), 136-148.https://doi.org/10.1027/1015-5759/a000479
Kim, E. S. (2010). Comparison of validities for scoring keys and scoring algorithms in situational judgment test: the influence of faking. Master Dissertation. Hoseo University, Cheonan, Korea.
Retrieved from http://dll.kr/azN
Kim, E. S., Han, Y. S., & Kim, M. S. (2011). Comparison of validities for scoring keys and scoring algorithms in situational judgment test: The influence of faking. Korean Journal of Industrial and Organizational Psychology, 24(1), 231-255.httpps://doi.org/10.24230/ksiop.24.1.201102.231
Kim, H. S., Hong, H. G., & Hyun, M. H. (2011). Validity and Reliability of the Korean Version of the Machiavellianism Personality Scale(K-MPS). The Korean Journal of Stress Research, 19(1), 21-30.
Retrieved from https://url.kr/8NOyGo
Kim, M. J. (1998). Measurement approach for the individual conflict handing in the organization. Doctoral Dissertation. Yonsei University. Retrieved fromhttp://www.riss.kr/link?id=T7125112
Kim, S. Y. (2017). Fundamentals and extensions of structural equation modeling. Seoul: Hakjisa. ISBN-10: 8999709671
Kopp, C. B. (1982). Antecedent of self-regulation: A developmental perspective. Developmental Psychology, 18(2), 199-214.https://doi.org/10.1037/0012-1649.18.2.199
Lee, J. G., & Han, Y. S. (2008). The reliability and validity of tele-marketer selecting competency tests in different formats: A comparison between ipsative and normative measures. Korean Journal of Industrial and Organizational Psychology, 21(3), 411-428. httpps://doi.org/10.24230/ksiop.21.3.200808.411
Lee, J. H. (2007). Validation of behavior tendency situational judgment tests: Validity based on structural equation modeling. Master’s Dissertation. Sungkyunkwan University. Retrieved fromhttp://www.riss.kr/link?id=T11043274
Lee, K. B., & Ashton, M. C. (2013). The h factor of personality. Seoul: Moonye publishing. ISBN-10: 8931007442
Lee, K. B., Ashton, M. C., & de Vries, R. E. (2005). Predicting workplace delinquency and integrity with the HEXACO and five-factor models of personality structure. Human Performance, 18(2), 179-197.
https://doi.org/10.1207/s15327043hup1802_4
Lee, K., & Ashton, M. C. (2004). Psychometric proporties of the HEXACO personality inventory. Multivariate Behavioral Research, 39(2), 329-358.https://doi.org/10.1207/s15327906mbr3902_8
Lee, K., Ashton, M. C., & Shin, K. H. (2005). Personality correlates of workplace anti-social behavior. Applied Psychology: An International Review, 54(1), 81-98.https://doi.org/10.1111/j.1464-0597.2005.00197.x
Lee, S. C., Lee, S. M., & Jo. Y. I. (2003). Critical review on paper and pencil situational judgement test. Korean Journal of Social and Personality Psychology, 16(3), 129-154. Retrieved from http://bitly.kr/7MEKrkaC38
Lee, S. J. (2010). Development of a situational judgement test type integrity test and investigation of based on internal structure. Master Dissertation. Sungkyunkwan University, Seoul, Korea.
Retrieved from http://dll.kr/azM
McDaniel, M. A. & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1-2), 103-113.https://doi.org/10.1111/1468-2389.00167
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730-740.https://doi.org/10.1037/0021-9010.86.4.730
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75(6), 640-647.https://doi.org/10.1037/0021-9010.75.6.640
Mumford, M. D., Connelly, M. S., Helton, W. B., Strange, J. M., & Osburn, H. K. (2001). On the construct validity of integrity tests: Individual and situational factors as predictors of test performance. International Journal of Selection and Assessment, 9(3), 240-257.https://doi.org/10.1111/1468-2389.00177
Mussel, P., Gatzka, T., & Hewig, J. (2018). Situational judgment tests as an alternative measure for personality assessment. European Journal of Psychological Assessment, 34(5), 328- 335.https://doi.org/10.1027/1015-5759/a000346
Namgung, J. J. (2010). Development and validation of a situational judgment test for high school students in the academic domain. Master Dissertation. Sungkyunkwan University, Seoul, Korea.
Retrieved from http://dll.kr/azQ
Nguyen, N. T. (2002). Faking in situational judgment tests: An empirical investigation of the work judgment survey. Doctoral Dissertation. Virginia Commonwealth University, Virginia. USA. Retrieved fromhttps://search.proquest.com/docview/304788837?accountid=48028
Nguyen, N. T., Biderman, M. D., & McDaniel, M. A. (2005). Effects of response instructions on faking a situational judgment test. International Journal of Selection and Assessment, 13(4), 250-260.https://doi.org/10.1111/j.1468-2389.2005.00322.x
O’Boyle Jr, E. H., Forsyth, D. R., Banks, G. C., & McDaniel, M. A. (2012). A meta-analysis of the dark triad and work behavior: A social exchange perspective. Journal of Applied Psychology, 97(3), 557-579.https://doi.org/10.1037/a0025679
Park, J. H. (2007). A validation study of competency factors using partially ipsative measures. Master’s Dissertation. Deagu University. Retrieved from http://www.riss.kr/link?id=T11293193
Paulhus, D. L., & Williams, K. M. (2002). The dark triad of personality: Narcissism, Machiavellianism, and psychopathy. Journal of Research in Personality, 36(6), 556-563.https://doi.org/10.1016/S0092-6566(02)00505-6
Roberts, B. W., Chernyshenko, O. S., Stark, S., & Goldberg, L. R. (2005). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires. Personnel Psychology, 58(1), 103-139.https://doi.org/10.1111/j.1744-6570.2005.00301.x
Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of black-white differences in overall and exercise scores. Personnel Psychology, 61(3), 637-661.https://doi.org/10.1111/j.1744-6570.2008.00125.x
Sarchione, C. D., Cuttler, M. J., Muchinsky, P. M., & Nelson-Gray, R. O. (1998). Prediction of dysfunctional job behaviors among law enforcement officers. Journal of Applied Psychology, 83(6), 904-912.https://doi.org/10.1037/0021-9010.83.6.904
Schmidt, F. L., & Hunter, J. E. (1993). Tacit knowledge, practical intelligence, general mental ability, and job knowledge. Current Directions in Psychological Science, 2(1), 8-9. https://doi.org/10.1111/1467-8721.ep10770456
Schmitt, N., & Chan, D. (2006). Situational judgment tests: Method or construct? In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 135-156). NJ: Erlbaum. ISBN-10: 0805852514
Snell, A. F., Sydell, E. J., & Lueke, S. B. (1999). Towards a theory of applicant faking: Integrating studies of deception. Human Resource Management Review, 9(2), 219-242.https://doi.org/10.1016/S1053-4822(99)00019-4
Song, M. J. (2008). Developmental Psychology. Seoul: Hakjisa. ISBN: 978-89-5891-855-4 93180
Sternberg, R. J., & Wagner, R. K. (1993). The g-ocentric view of intelligence and job performance is wrong. Current Directions in Psychological Science, 2(1), 1-4.https://doi.org/10.1111/1467-8721.ep10770441
Tangney, J. P., Baumeister, R. F., & Boone, A. L. (2004). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. Journal of Personality, 72(2), 271-322.https://doi.org/10.1111/j.0022-3506.2004.00263.x.
Timpano, K. R., & Schmidt, N. B. (2013). The relationship between self control deficits and hoarding: A multimethod investigation across three samples. Journal of Abnormal Psychology, 122(1), 13-25.https://doi.org/10.1037/a0029760
Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50(1), 25-49.https://doi.org/10.1111/j.1744-6570.1997.tb00899.x
Wendler, K., Liu, J., & Zettler, I. (2018). Honesty-humility interacts with context perception in predicting task performance and organizational citizenship behavior. Journal of Personnel Psychology, 17(4), 161-171.https://doi.org/10.1027/1866-5888/a000203.
White, G. L. (1984) Comparison of four jealousy scales. Journal of Research in Personality, 18(2), 115-130.https://doi.org/10.1016/0092-6566(84)90024-2
Yoo, T. Y., Lee, K. B., & Ashton, M. C. (2004). Psychometric properties of the Korean version of the HEXACO personality inventory. Korean Journal of Social and Personality Psychology, 18(3), 61-75. Retrieved fromhttp://www.riss.kr/link?id=A100629491