A Deep Learning Ensemble to Classify Anxiety, Depression, and their Comorbidity from Texts of Social Networks
DOI:
https://doi.org/10.5753/jidm.2021.1901Keywords:
Deep learning, Social Networks, Mental healthAbstract
The use of social networks to expose personal difficulties has enabled works on the automatic identification of specific mental conditions, particularly depression. Depression is the most incapacitating disease worldwide, and it has an alarming comorbidity rate with anxiety. In this paper, we explore deep learning techniques to develop a stacking ensemble to automatically identify depression, anxiety, and comorbidity, using data extracted from Reddit. The stacking is composed of specialized single-label binary classifiers that distinguish between specific disorders and control users. A meta-learner explores these base classifiers as a context for reaching a multi-label, multi-class decision. We developed extensive experiments using alternative architectures (LSTM, CNN, and their combination), word embeddings, and ensemble topologies. All base classifiers and ensembles outperformed the baselines. The CNN-based binary classifiers achieved the best performance, with f-measures of 0.79 for depression, 0.78 for anxiety, and 0.78 for comorbidity. The ensemble topology with best performance (Hamming Loss of 0.29 and Exact Match Ratio of 0.47) combines base classifiers according to three architectures, and do not include comorbidity classifiers. Using SHAP, we confirmed the influential features are related to symptoms of these disorders.
Downloads
References
American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. Autor, Washington, DC, 2013.
Amora, P. R. P., Teixeira, E. M., Lima, M. I. V., Amaral, G. M., Cardozo, J. R. A., and Machado, J. d. C. An analysis of machine learning techniques to prioritize customer service through social networks. Journal of Information and Data Management 9 (2): 135—-146, 2018.
Bagroy, S., Kumaraguru, P., and De Choudhury, M. A social media based index of mental well-being in college campuses. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. CHI ’17. Association for Computing Machinery, New York, NY, USA, pp. 1634–1646, 2017.
Becker, K., Harb, J. G., and Ebeling, R. Exploring deep learning for the analysis of emotional reactions to terrorist events on twitter. Journal of Information and Data Management 10 (2): 97–115, 2019.
Benton, A., Mitchell, M., and Hovy, D. Multitask learning for mental health conditions with limited social media data. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, Valencia, Spain, pp. 152–162, 2017.
Chollet, F. What is deep learning? In Deep Learning with Python. Manning, 1, pp. 8–22;94–96,102–104,123;184–185,196–197,202–206,215–216;264–266, 2017.
Cohan, A., Desmet, B., Yates, A., Soldaini, L., MacAvaney, S., and Goharian, N. SMHD: a large-scale resource for exploring online language usage for multiple mental health conditions. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, E. M. Bender, L. Derczynski, and P. Isabelle (Eds.). Association for Computational Linguistics, pp. 1485–1497, 2018.
De Choudhury, M., Counts, S., Horvitz, E. J., and Hoff, A. Characterizing and predicting postpartum depression from shared facebook data. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work amp; Social Computing. CSCW ’14. Association for Computing Machinery, New York, NY, USA, pp. 626–638, 2014.
De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., and Kumar, M. Discovering shifts to suicidal ideation from mental health content in social media. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI ’16. Association for Computing Machinery, New York, NY, USA, pp. 2098–2110, 2016.
De Choudhury, M., Sharma, S. S., Logar, T., Eekhout, W., and Nielsen, R. C. Gender and cross-cultural differences in social media disclosures of mental illness. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. CSCW ’17. Association for Computing Machinery, New York, NY, USA, pp. 353–369, 2017.
Dutta, S., Ma, J., and De Choudhury, M. Measuring the impact of anxiety on online social interactions. In ICWSM. ICWSM’18. The AAAI Press, 2018.
Gama, J., Faceli, K., Lorena, A., and De Carvalho, A. Inteligência artificial: uma abordagem de aprendizado de máquina. Grupo Gen - LTC, 2011.
Giuntini, F. T., Cazzolato, M. T., dos Reis, M. d. J. D., Campbell, A. T., Traina, A. J. M., and Ueyama, J. A review on recognizing depression in social networks: challenges and opportunities. Journal of Ambient Intelligence and Humanized Computing (1868-5145, 2020.
Gkotsis, G., Oellrich, A., Velupillai, S., Liakata, M., Hubbard, T. J. P., Dobson, R. J. B., and Dutta, R. Characterisation of mental health conditions in social media using informed deep learning. Scientific Reports vol. 7, pp. 2045–2322, 2017.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., and Schmidhuber, J. Lstm: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems 28 (10): 2222–2232, Oct, 2017.
Gruda, D. and Hasan, S. Feeling anxious? perceiving anxiety in tweets using machine learning. Computers in Human Behavior vol. 98, pp. 245 – 255, 2019.
Hamilton, M. Development of a rating scale for primary depressive illness. British Journal of Social and Clinical Psychology 6 (4): 278–296, 1967.
Harb, J. G., Ebeling, R., and Becker, K. Exploring deep learning for the analysis of emotional reactions to terrorist events on twitter. J. Inf. Data Manag. 10 (2): 97–115, 2019.
Hirschfeld, R. The comorbidity of major depression and anxiety disorders: Recognition and management in primary care. Prim Care Companion J Clin Psychiatry 3 (244-254), 12, 2001.
Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Comput. 9 (8): 1735–1780, Nov., 1997.
Ireland, M. and Iserman, M. Within and between-person differences in language used across anxiety support and neutral Reddit communities. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. Association for Computational Linguistics, New Orleans, LA, pp. 182–193, 2018.
Ive, J., Gkotsis, G., Dutta, R., Stewart, R., and Velupillai, S. Hierarchical neural model with attention mechanisms for the classification of social media text related to mental health. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. Association for Computational Linguistics, New Orleans, LA, pp. 69–77, 2018.
Kim, Y. Convolutional neural networks for sentence classification. In Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) - ACL. pp. 1746–1751, 2014.
Kowsari, K., Meimandi, K. J., Heidarysafa, M., Mendu, S., Barnes, L. E., and Brown, D. E. Text classification algorithms: A survey. Inf. 10 (4): 150, 2019.
Lin, H., Jia, J., Guo, Q., Xue, Y., Li, Q., Huang, J., Cai, L., and Feng, L. User-level psychological stress detection from social media using deep neural network. In Proceedings of the 22nd ACM International Conference on Multimedia. MM ’14. Association for Computing Machinery, New York, NY, USA, pp. 507–516, 2014.
Lundberg, S. M. and Lee, S. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems : Proc. of the 30th Annual Conf. on Neural Information Processing Systems (NIPS), I. Guyon, U. von Luxburg, and et alli (Eds.). pp. 4765–4774, 2017.
Mann, P., Paes, A., and Matsushima, E. H. See and read: Detecting depression symptoms in higher education students using multimodal social media data. In ICWSM. Proceedings of the International AAAI Conference on Web and Social Media 14 (1): 440–451, May, 2020.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., pp. 3111–3119, 2013.
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., and Gao, J. Deep learning based text classification: A comprehensive review. CoRR vol. abs/2004.03705, 2020.
Murphy, K. P. Machine Learning: A Probabilistic Perspective. The MIT Press, 2012.
Park, S., Kim, I., Lee, S. W., Yoo, J., Jeong, B., and Cha, M. Manifestation of depression and loneliness on social networks: A case study of young adults on facebook. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work Social Computing. CSCW ’15. Association for Computing Machinery, New York, NY, USA, pp. 557–570, 2015.
Pennington, J., Socher, R., and Manning, C. D. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP). pp. 1532–1543, 2014.
Radloff, L. S. The ces-d scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1 (3): 385–401, 1977.
Sharma, E. and De Choudhury, M. Mental health support and its relationship to linguistic accommodation in online communities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI ’18. Association for Computing Machinery, New York, NY, USA, pp. 1–13, 2018.
Shen, J. H. and Rudzicz, F. Detecting anxiety through Reddit. In Proceedings of the Fourth Workshop on Computational Linguistics and Clinical Psychology — From Linguistic Signal to Clinical Reality. Association for Computational Linguistics, Vancouver, BC, pp. 58–65, 2017.
Souza, V. B., Nobre, J. C., and Becker, K. Characterization of Anxiety, Depression, and their Comorbidity from Texts of Social Networks. In Anais do XXXV Simpósio Brasileiro de Banco de Dados. SBC, Porto Alegre, RS, Brasil, 2020.
Tadesse, M. M., Lin, H., Xu, B., and Yang, L. Detection of depression-related posts in reddit social media forum. IEEE Access vol. 7, pp. 44883–44893, 2019.
Tiller, J. Depression and anxiety. The Medical journal of Australia vol. 199, pp. S28–31, 09, 2013.
Tsugawa, S., Kikuchi, Y., Kishino, F., Nakajima, K., Itoh, Y., and Ohsaki, H. Recognizing depression from twitter activity. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. CHI ’15. Association for Computing Machinery, New York, NY, USA, pp. 3187–3196, 2015.
Wongkoblap, A., Vadillo, M., and Curcin, V. Researching mental health disorders in the era of social media: Systematic review. Journal of Medical Internet Research 19 (6), 2017.
Yates, A., Cohan, A., and Goharian, N. Depression and self-harm risk assessment in online forums. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pp. 2968–2978, 2017.
Zhang, C. and Ma, Y. Ensemble Machine Learning: Methods and Applications. Springer Publishing Company, Incorporated, 2012.
Zhang, L., Wang, S., and Liu, B. Deep learning for sentiment analysis: A survey. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 8 (4), 2018.
