Machine Learning Approach to Personality Assessment and Its Application to Personnel Selection A Brief Review of the Current Research and Suggestions for the Future

Main Article Content

JiSoo Ock
HyeRyeon An


As we enter the digital age, new methods of personality testing-namely, machine learning-based personality assessment scales-are quickly gaining attraction. Because machine learning-based personality assessments are made based on algorithms that analyze digital footprints of people’s online behaviors, they are supposedly less prone to human biases or cognitive fallacies that are often cited as limitations of traditional personality tests. As a result, machine learning-based assessment tools are becoming increasingly popular in operational settings across the globe with the anticipation that they can effectively overcome the limitations of traditional personality testing. However, the provision of scientific evidence regarding the psychometric soundness and the fairness of machine learning-based assessment tools have lagged behind their use in practice. The current paper provides a brief review of empirical studies that have examined the validity of machine learning-based personality assessment, focusing primarily on social media text mining method. Based on this review, we offer some suggestions about future research directions, particularly regarding the important and immediate need to examine the machine learning-based personality assessment tools’ compliance with the practical and legal standards for use in practice (such as inter-algorithm reliability, test-retest reliability, and differential prediction across demographic groups). Additionally, we emphasize that the goal of machine learning-based personality assessment tools should not be to simply maximize the prediction of personality ratings. Rather, we should explore ways to use this new technology to further develop our fundamental understanding of human personality and to contribute to the development of personality theory.


Metrics Loading ...

Article Details

How to Cite
Ock, J., & An, H. (2021). Machine Learning Approach to Personality Assessment and Its Application to Personnel Selection: A Brief Review of the Current Research and Suggestions for the Future. Korean Journal of Industrial and Organizational Psychology, 34(2), 213–236.
Systematic review and meta-analysis articles


Alexander, L., III, Mulfinger, E., & Oswald, F. L. (2020). Using big data and machine learning in personality measurement: Opportunities and challenges. European Journal of Personality, 34, 632-648.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education.

Back, M. D., Stopfer, J. M., Vazire, S., Gaddis, S., Schmukle, S. C., Egloff, B., & Gosling, S. D. (2010). Facebook profiles reflect actual personality, not self-idealization. Psychological Science, 21, 372-374.

Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018). A review of best practice recommendations for text analysis in R (and a user-friendly app). Journal of Business and Psychology, 33, 445-459.

Barrick, M. R. (2005). Yes, personality matters: Moving on to more important matters. Human Performance, 18, 359-372.

Bleidorn, W., & Hopwood, C. J. (2019). Using machine learning to advance personality assessment and theory. Personality and Social Psychology Review, 23, 190-203.

Boyd, R. L., Pennebaker, J. W. (2017). Language-based personality: A new approach to personality in a digital world. Current Opinion in Behavioral Sciences, 18, 63-68.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.

Chittaranjan, G., Blom, J., & Gatica-Perez, D. (2013). Mining large-scale smartphone data for personality studies. Personal and Ubiquitous Computing, 17, 433-450.

Dastin, J. (2018, October 11). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters.

Denny, M. J., & Spirling, A. (2018). Text preprocessing for unsupervised learning: Why it matters, when it misleads, and what to do about it. Political Analysis, 26, 168-189.

Eichstaedt, J. C., Kern, M. L., Yaden, D. B., Schwartz, H. A., Giorgi, S., Park, G., Hagan, C. A., Tobolsky, V., Smith, L. K., Buffone, A., Iwry, J., Seligman, M. E. P., & Ungar, L. H. (2020). Closed and open vocabulary approaches to text analysis: A review, quantitative comparison, and recommendations.

Gladstone, J. J., Matz, S., & Lemaire, A. (2019). Can psychological traits be inferred from spending? Evidence from transaction data. Psychological Science, 30,1087-1096.

Golbeck, J. A. (2016). Predicting personality from social media text. AIS Transactions on Replication Research, 2, 1-10.

Goldberg, L. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216-1229.

Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96.

Gonzalez, M. F., Capman, J. F., Oswald, F. L., Theys, E. R., & Tomczak, D. L. (2019). Where’s the I-O? Artificial intelligence and machine learning in talent management systems. Personnel Assessment and Decisions, 5, 33-44.

Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (in press). Text preprocessing for text mining in organizational research: Review and recommendations. Organizational Research Methods.

Hough, L. M. (1998). The millenium for personality psychology: New horizons or good old daze. Applied Psychology, 47, 233-261.

Iacobelli, F., Gill, A. J., Nowson, S., & Oberlander, J. (2011). Large scale personality classification of bloggers. In S. D’Mello, A. Graesser, B. Schuller, & J. Martin (Eds.), Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction (pp. 568-577). New York, NY: Springer-Verlag.

Jockers, M. (2020). syuzhet: Extracts sentiment and sentiment-derived plot arcs from text [Computer software manual].

Kern, M. L., Park, G., Eichstaedt, J. C., Schwartz, H. A., Sap, M., Smith, L. K., & Ungar, L. H. (2016). Gaining insights from social media language: Methodologies and challenges. Psychological Methods, 21, 507-525.

Kobayashi, V. B., Mol, S. T., Berkers, H. A., Kismihók, G., & Den Hartog, D. N. (2018). Text mining in organizational research. Organizational Research Methods, 21, 733-765.

Kosinski, M., Bachrach, Y., Kohli, P., Stillwell, D., & Graepel, T. (2014). Manifestations of user personality in website choice and behaviour on online social networks. Machine Learning, 95, 357-380.

Lee, K., & Ashton, M. C. (2004). Psychometric properties of the HEXACO Personality Inventory. Multivariate Behavioral Research, 39, 329-358.

Lenhart, A., Duggan, M., Perrin, A., Steepler, R., Rainie, L., & Parker, K. (2015). Teens, social media, & technology overview 2015: Smartphones facilitate shifts in communication landscape for teens (p. 48). Retrieved from

McAbee, S. T., & Connelly, B. S. (2016). A multi-rater framework for studying personality: The trait-reputation-identity model. Psychological Review, 123, 569-591.

Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K.., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Selection, 60, 683-729.

Murphy, K. R. (2020). Performance evaluation will not die, but it should. Human Resource Management, 30, 13-31.

Oswald, F. L., Behrend, T. S., Putka, D. J., & Sinar, E. (2020). Big data in industrial-organizational psychology and human resource management: Forward progress for organizational research and practice. Annual Review of Organizational Psychology and Organizational Behavior, 7, 505-533.

Park, G., Schwartz, A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934-952.

Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015). The development and psychometric properties of LIWC2015. Austin, TX: University of Texas at Austin.

Perrin, A., & Anderson, M. (2019, April 10). Share of U.S. adults using social media, including Facebook, is mostly unchanged since 2018. Retrieved May 27, 2019, from Pew Research Center website:

Sajjadiani, S., Sojourner, A. J., Kammeyer-Mueller, J. D., & Mykerezi, E. (2019). Using machine learning to translate applicant work history into predictors of performance and turnover. Journal of Applied Psychology, 104, 1207-1225.

Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., … Ungar, L. H. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE, 8, e73791.

Seih, Y.-T., Lepicovsky, M., & Chang, Y.-Y. (2020). Your words reveal your thoughts: A two-wave study of assessing language dimensions in predicting employee turnover intention. International Journal of Selection and Assessment, 28, 484-497.

Smith, E., Greco, N., Bosnjak, M., & Vlachos, A. (2015, September). A strong lexical matching method for the machine comprehension test. In Proceedings of the 2015 Conference on the Empirical Methods in Natural Language Processing (pp. 1693-1698).

Society for Industrial and Organizational Psychology (2018). Principles for the validation and use of personnel selection procedures (5th ed.). Bowling Green, OH: The Society for Industrial and Organizational Psychology.

Stachl, C., Pargent, F., Hilbert, S., Harari, G. M., Schoedel, R., Vaid, S., Gosling, S. D., & Bühner, M. (2020). Personality research and assessment in the era of machine learning. European Journal of Personality, 34, 613-631.

Tay, L., Woo, S. E., Hickman, L., & Saef, R. M. (2020). Psychometric and validity issues in machine learning approaches to personality assessment: A focus on social media text mining. European Journal of Personality, 34, 826-844.

Tonidandel, S., King, E. B., & Cortina, J. M. (2018). Big data methods: Leveraging modern data analytic techniques to build organizational science. Organizational Research Methods, 21, 525-547.

Uysal, A. K., & Gunal, S. (2014). The impact of preprocessing on text classification. Information Processing & Management, 50, 104-112.

Welbers, K., Van Atteveldt, W., & Benoit, K. (2017). Text analysis in R. Communication Methods and Measures, 11, 245-265.

Woo, S. E., Tay, L., Proctor, R. W. (2020). Big data in psychological research. Washington: American Psychological Association.

Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112, 1036-1040.