Machine Learning Approach to Personality Assessment and Its Application to Personnel Selection A Brief Review of the Current Research and Suggestions for the Future
Article Sidebar
Main Article Content
Abstract
As we enter the digital age, new methods of personality testing-namely, machine learning-based personality assessment scales-are quickly gaining attraction. Because machine learning-based personality assessments are made based on algorithms that analyze digital footprints of people’s online behaviors, they are supposedly less prone to human biases or cognitive fallacies that are often cited as limitations of traditional personality tests. As a result, machine learning-based assessment tools are becoming increasingly popular in operational settings across the globe with the anticipation that they can effectively overcome the limitations of traditional personality testing. However, the provision of scientific evidence regarding the psychometric soundness and the fairness of machine learning-based assessment tools have lagged behind their use in practice. The current paper provides a brief review of empirical studies that have examined the validity of machine learning-based personality assessment, focusing primarily on social media text mining method. Based on this review, we offer some suggestions about future research directions, particularly regarding the important and immediate need to examine the machine learning-based personality assessment tools’ compliance with the practical and legal standards for use in practice (such as inter-algorithm reliability, test-retest reliability, and differential prediction across demographic groups). Additionally, we emphasize that the goal of machine learning-based personality assessment tools should not be to simply maximize the prediction of personality ratings. Rather, we should explore ways to use this new technology to further develop our fundamental understanding of human personality and to contribute to the development of personality theory.
Metrics
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Alexander, L., III, Mulfinger, E., & Oswald, F. L. (2020). Using big data and machine learning in personality measurement: Opportunities and challenges. European Journal of Personality, 34, 632-648. https://doi.org/10.1002/per.2305 DOI: https://doi.org/10.1002/per.2305
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education.
Back, M. D., Stopfer, J. M., Vazire, S., Gaddis, S., Schmukle, S. C., Egloff, B., & Gosling, S. D. (2010). Facebook profiles reflect actual personality, not self-idealization. Psychological Science, 21, 372-374.https://doi.org/10.1177/0956797609360756 DOI: https://doi.org/10.1177/0956797609360756
Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018). A review of best practice recommendations for text analysis in R (and a user-friendly app). Journal of Business and Psychology, 33, 445-459.https://doi.org/10.1007/s10869-017-9528-3 DOI: https://doi.org/10.1007/s10869-017-9528-3
Barrick, M. R. (2005). Yes, personality matters: Moving on to more important matters. Human Performance, 18, 359-372.https://doi.org/10.1207/s15327043hup1804_3 DOI: https://doi.org/10.1207/s15327043hup1804_3
Bleidorn, W., & Hopwood, C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23, 190-203.https://dx.doi.org/10.1177/1088868318772990 DOI: https://doi.org/10.1177/1088868318772990
Boyd, R. L., Pennebaker, J. W. (2017). Language-based personality: A new approach to personality in a digital world. Current Opinion in Behavioral Sciences, 18, 63-68. https://doi.org/10.1016/j.cobeha.2017.07.017 DOI: https://doi.org/10.1016/j.cobeha.2017.07.017
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.https://doi.org/10.1037/h0046016 DOI: https://doi.org/10.1037/h0046016
Chittaranjan, G., Blom, J., & Gatica-Perez, D. (2013). Mining large-scale smartphone data for personality studies. Personal and Ubiquitous Computing, 17, 433-450.https://doi.org/10.1007/s00779-011-0490-1 DOI: https://doi.org/10.1007/s00779-011-0490-1
Dastin, J. (2018, October 11). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters.https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G
Denny, M. J., & Spirling, A. (2018). Text preprocessing for unsupervised learning: Why it matters, when it misleads, and what to do about it. Political Analysis, 26, 168-189. https://doi.org/10.1017/pan.2017.44 DOI: https://doi.org/10.1017/pan.2017.44
Eichstaedt, J. C., Kern, M. L., Yaden, D. B., Schwartz, H. A., Giorgi, S., Park, G., Hagan, C. A., Tobolsky, V., Smith, L. K., Buffone, A., Iwry, J., Seligman, M. E. P., & Ungar, L. H. (2020). Closed and open vocabulary approaches to text analysis: A review, quantitative comparison, and recommendations. https://doi.org/10.31234/osf.io/t52c6 DOI: https://doi.org/10.31234/osf.io/t52c6
Gladstone, J. J., Matz, S., & Lemaire, A. (2019). Can psychological traits be inferred from spending? Evidence from transaction data. Psychological Science, 30,1087-1096.https://dx.doi.org/10.1177/0956797619849435 DOI: https://doi.org/10.1177/0956797619849435
Golbeck, J. A. (2016). Predicting personality from social media text. AIS Transactions on Replication Research, 2, 1-10.https://doi.org/10.17705/1atrr.00009 DOI: https://doi.org/10.17705/1atrr.00009
Goldberg, L. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216-1229.https://doi.org/10.1037/0022-3514.59.6.1216 DOI: https://doi.org/10.1037/0022-3514.59.6.1216
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96.https://doi.org/10.1016/j.jrp.2005.08.007 DOI: https://doi.org/10.1016/j.jrp.2005.08.007
Gonzalez, M. F., Capman, J. F., Oswald, F. L., Theys, E. R., & Tomczak, D. L. (2019). Where’s the I-O? Artificial intelligence and machine learning in talent management systems. Personnel Assessment and Decisions, 5, 33-44.https://doi.org/10.25035/pad.2019.03.005 DOI: https://doi.org/10.25035/pad.2019.03.005
Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (in press). Text preprocessing for text mining in organizational research: Review and recommendations. Organizational Research Methods.https://doi.org/10.1177/1094428120971683 DOI: https://doi.org/10.1177/1094428120971683
Hough, L. M. (1998). The millenium for personality psychology: New horizons or good old daze. Applied Psychology, 47, 233-261. https://doi.org/10.1111/j.1464-0597.1998.tb00023.x DOI: https://doi.org/10.1111/j.1464-0597.1998.tb00023.x
Iacobelli, F., Gill, A. J., Nowson, S., & Oberlander, J. (2011). Large scale personality classification of bloggers. In S. D’Mello, A. Graesser, B. Schuller, & J. Martin (Eds.), Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction (pp. 568-577). New York, NY: Springer-Verlag. https://doi.org/10.1007/978-3-642-24571-8_71 DOI: https://doi.org/10.1007/978-3-642-24571-8_71
Jockers, M. (2020). syuzhet: Extracts sentiment and sentiment-derived plot arcs from text [Computer software manual].https://cran.r-project.org/web/packages/syuzhet.
Kern, M. L., Park, G., Eichstaedt, J. C., Schwartz, H. A., Sap, M., Smith, L. K., & Ungar, L. H. (2016). Gaining insights from social media language: Methodologies and challenges. Psychological Methods, 21, 507-525.https://dx.doi.org/10.1037/met0000091 DOI: https://doi.org/10.1037/met0000091
Kobayashi, V. B., Mol, S. T., Berkers, H. A., Kismihók, G., & Den Hartog, D. N. (2018). Text mining in organizational research. Organizational Research Methods, 21, 733-765. https://doi.org/10.1177/1094428117722619 DOI: https://doi.org/10.1177/1094428117722619
Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D., & Graepel, T. (2014). Manifestations of user personality in website choice and behaviour on online social networks. Machine Learning, 95, 357-380.https://dx.doi.org/10.1007/s10994-013-5415-y DOI: https://doi.org/10.1007/s10994-013-5415-y
Lee, K., & Ashton, M. C. (2004). Psychometric properties of the HEXACO Personality Inventory. Multivariate Behavioral Research, 39, 329-358.https://doi.org/10.1207/s15327906mbr3902_8 DOI: https://doi.org/10.1207/s15327906mbr3902_8
Lenhart, A., Duggan, M., Perrin, A., Steepler, R., Rainie, L., & Parker, K. (2015). Teens, social media, & technology overview 2015: Smartphones facilitate shifts in communication landscape for teens (p. 48). Retrieved fromhttps://www.pewresearch.org/wp-content/uploads/sites/9/2015/04/PI_TeensandTech_Update2015_0409151.pdf
McAbee, S. T., & Connelly, B. S. (2016). A multi-rater framework for studying personality: The trait-reputation-identity model. Psychological Review, 123, 569-591.https://doi.org/10.1037/rev0000035 DOI: https://doi.org/10.1037/rev0000035
Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K.., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Selection, 60, 683-729. https://doi.org/10.1111/j.1744-6570.2007.00089.x DOI: https://doi.org/10.1111/j.1744-6570.2007.00089.x
Murphy, K. R. (2020). Performance evaluation will not die, but it should. Human Resource Management, 30, 13-31.https://doi.org/10.1111/1748-8583.12259 DOI: https://doi.org/10.1111/1748-8583.12259
Oswald, F. L., Behrend, T. S., Putka, D. J., & Sinar, E. (2020). Big data in industrial-organizational psychology and human resource management: Forward progress for organizational research and practice. Annual Review of Organizational Psychology and Organizational Behavior, 7, 505-533.https://doi.org/10.1146/annurev-orgpsych-032117-104553 DOI: https://doi.org/10.1146/annurev-orgpsych-032117-104553
Park, G., Schwartz, A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934-952.https://doi.org/10.1037/pspp0000020 DOI: https://doi.org/10.1037/pspp0000020
Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015). The development and psychometric properties of LIWC2015. Austin, TX: University of Texas at Austin. https://dx.doi.org/10.15781/T29G6Z
Perrin, A., & Anderson, M. (2019, April 10). Share of U.S. adults using social media, including Facebook, is mostly unchanged since 2018. Retrieved May 27, 2019, from Pew Research Center website:https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/
Sajjadiani, S., Sojourner, A. J., Kammeyer-Mueller, J. D., & Mykerezi, E. (2019). Using machine learning to translate applicant work history into predictors of performance and turnover. Journal of Applied Psychology, 104, 1207-1225. https://doi.org/10.1037/apl0000405 DOI: https://doi.org/10.1037/apl0000405
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., … Ungar, L. H. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE, 8, e73791.https://dx.doi.org/10.1371/journal.pone.0073791 DOI: https://doi.org/10.1371/journal.pone.0073791
Seih, Y.-T., Lepicovsky, M., & Chang, Y.-Y. (2020). Your words reveal your thoughts: A two-wave study of assessing language dimensions in predicting employee turnover intention. International Journal of Selection and Assessment, 28, 484-497.https://doi.org/10.1111/ijsa.12302 DOI: https://doi.org/10.1111/ijsa.12302
Smith, E., Greco, N., Bosnjak, M., & Vlachos, A. (2015, September). A strong lexical matching method for the machine comprehension test. In Proceedings of the 2015 Conference on the Empirical Methods in Natural Language Processing (pp. 1693-1698). DOI: https://doi.org/10.18653/v1/D15-1197
Society for Industrial and Organizational Psychology (2018). Principles for the validation and use of personnel selection procedures (5th ed.). Bowling Green, OH: The Society for Industrial and Organizational Psychology.
Stachl, C., Pargent, F., Hilbert, S., Harari, G. M., Schoedel, R., Vaid, S., Gosling, S. D., & Bühner, M. (2020). Personality research and assessment in the era of machine learning. European Journal of Personality, 34, 613-631. https://doi.org/10.1002/per.2257 DOI: https://doi.org/10.1002/per.2257
Tay, L., Woo, S. E., Hickman, L., & Saef, R. M. (2020). Psychometric and validity issues in machine learning approaches to personality assessment: A focus on social media text mining. European Journal of Personality, 34, 826-844. https://doi.org/10.1002/per.2290 DOI: https://doi.org/10.1002/per.2290
Tonidandel, S., King, E. B., & Cortina, J. M. (2018). Big data methods: Leveraging modern data analytic techniques to build organizational science. Organizational Research Methods, 21, 525-547.https://doi.org/10.1177/1094428116677299 DOI: https://doi.org/10.1177/1094428116677299
Uysal, A. K., & Gunal, S. (2014). The impact of preprocessing on text classification. Information Processing & Management, 50, 104-112.https://doi.org/10.1016/j.ipm.2013.08.006 DOI: https://doi.org/10.1016/j.ipm.2013.08.006
Welbers, K., Van Atteveldt, W., & Benoit, K. (2017). Text analysis in R. Communication Methods and Measures, 11, 245-265.https://doi.org/10.1080/19312458.2017.1387238 DOI: https://doi.org/10.1080/19312458.2017.1387238
Woo, S. E., Tay, L., Proctor, R. W. (2020). Big data in psychological research. Washington: American Psychological Association. DOI: https://doi.org/10.1037/0000193-000
Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112, 1036-1040.https://doi.org/10.1073/pnas.1418680112 DOI: https://doi.org/10.1073/pnas.1418680112