DeePsy: Introducing an online feedback tool for psychotherapy
Vol.17,No.1(2023)
Although psychotherapy has been proven to be an effective form of help, its effectiveness has been hitting an imaginary ceiling for several decades. A possible reason for this is the lack of immediate feedback that would allow therapists to detect various difficulties early in the therapeutic process and respond appropriately. Existing research shows that, as therapists, we tend to overestimate our skills and often fail to recognize dissatisfaction and deterioration in our clients in a timely manner. This text aims to introduce DeePsy, an online tool for continuous monitoring of the psychotherapy process and outcome. The application provides therapists with ongoing and systematic feedback on their work based on regularly administered questionnaires and automatic analysis of session recordings. The article explains the basic principles underlying DeePsy and describes its current status. It also offers a reflection on feedback in the broader context of the culture of psychotherapeutic work.
routine outcome and process monitoring; feedback; web application; DeePsy
Aafjes-Van Doorn, K., & Meisel, J. (2022). Implementing routine outcome monitoring in a psychodynamic training clinic: It's complicated. Counselling Psychology Quarterly. Advance online publication. https://doi.org/10.1080/09515070.2022.2110451
Aafjes-van Doorn, K., Porcerelli, J., & Müller-Frommeyer, L. C. (2020). Language style matching in psychotherapy: An implicit aspect of alliance. Journal of Counseling Psychology, 67(4), 509-522. https://doi.org/10.1037/cou0000433
American Psychological Association (2013). Recognition of psychotherapy effectiveness. Psychotherapy, 50(1), 102-109. https://doi.org/10.1037/a0030276
Barkham, M., Bewick, B., Mullin, T., Gilbody, S., Connell, J., Cahill, J., Mellor-Clark, J., Richards, D., Unsworth, G., & Evans, C. (2013). The CORE-10: A short measure of psychological distress for routine use in the psychological therapies. Counselling and Psychotherapy Research, 13(1), 3-13. https://doi.org/10.1080/14733145.2012.729069
Barkham, M., & Lambert, M. (2021). The efficacy and effectiveness of psychological therapies. In M. Barkham, W. Lutz, & L. G. Castonguay (Eds.), Bergin and Garffield's handbook of psychotherapy and behavior change (pp. 135-189). Wiley.
Barkham, M., Mellor-Clark, J., & Stiles, W. B. (2015). A CORE approach to progress monitoring and feedback: Enhancing evidence and improving practice. Psychotherapy, 52(4), 402-411. https://doi.org/10.1037/pst0000030
Bech, P., Olsen, L., Kjoller, M., & Rasmussen, N. (2003). Measuring well-being rather than the absence of distress symptoms: A comparison of the SF-36 mental health subscale and the WHO-Five well-being scale. International Journal of Methods in Psychiatric Research, 12(2), 85-91. https://doi.org/10.1002/mpr.145
Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423-1429. https://doi.org/10.1176/appi.ps.002052011
Brattland, H., Koksvik, J. M., Burkeland, O., Klöckner, C. A., Lara-Cabrera, M. L., Miller, S. D., Wampold, B., Ryum, T., & Iversen, V. C. (2019). Does the working alliance mediate the effect of routine outcome monitoring (ROM) and alliance feedback on psychotherapy outcomes? A secondary analysis from a randomized clinical trial. Journal of Counseling Psychology, 66(2), 234-246. https://doi.org/10.1037/cou0000320
Cao, J., Tanana, M., Imel, Z. E., Poitras, E., Atkins, D. C., & Srikumar, V. (2019). Observing dialogue in therapy: Categorizing and forecasting behavioral codes. https://arxiv.org/abs/1907.00326 https://doi.org/10.18653/v1/P19-1563
Cooper, M., & Norcross, J. C. (2016). A brief, multidimensional measure of clients' therapy preferences: The Cooper-Norcross Inventory of Preferences (C-NIP). International Journal of Clinical and Health Psychology, 16(1), 87-98. https://doi.org/10.1016/j.ijchp.2015.08.003
De Jong, K. Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021). Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review, 85, 102002. https://doi.org/10.1016/j.cpr.2021.102002
De Jong, K., Timman, R., Hakkaart-Van Roijen, L., Vermeulen, P., Kooiman, K., Passchier, J., & Busschbach, J. Van. (2014). The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research, 24(6), 629-639. https://doi.org/10.1080/10503307.2013.871079
De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22(4), 464-474. https://doi.org/10.1080/10503307.2012.673023
Evans, C., Carlyle, J.-A. (2021). Outcome measures and evaluation in counseling and psychotherapy. Sage.
Ewbank, M. P., Cummins, R., Tablan, V., Catarino, A., Buchholz, S., & Blackwell, A. D. (2021). Understanding the relationship between patient language and outcomes in internet-enabled cognitive behavioural therapy: A deep learning approach to automatic coding of session transcripts. Psychotherapy Research, 31(3), 300-312. https://doi.org/10.1080/10503307.2020.1788740
Farber, B. A. (2020). Disclosure, concealment, and dishonesty in psychotherapy: A clinically focused review. Journal of Clinical Psychology, 76(2), 251-257. https://doi.org/10.1002/jclp.22891
Flemotomos, N., Martinez, V. R., Chen, Z., Creed, T. A., Atkins, D. C., & Narayanan, S. (2021). Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations. PLoS ONE, 16(10), e0258639. https://doi.org/10.1371/journal.pone.0258639
Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1-11. https://doi.org/10.1037/cou0000131
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155-163. https://doi.org/10.1002/jclp.20108
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9(3), 329-343. https://doi.org/10.1093/clipsy.9.3.329
Hatfield, D., McCullough, L., Frantz, S. H. B., Krieger, K., Hatfi, D., McCullough, L., Frantz, S. H. B., & Krieger, K. (2010). Do we know when our clients get worse? An investigation of therapists' ability to detect negative client change. Clinical Psychology and Psychotherapy, 17(1), 25-32. https://doi.org/10.1002/cpp.656
Heimerl, F., Lohmann, S., Lange, S., & Ertl, T. (2014). Word Cloud Explorer: Text analytics based on word clouds. Paper presented at 47th Hawaii International Conference on System Sciences (pp. 1833-1842). https://doi.org/10.1109/HICSS.2014.231
Imel, Z. E., Pace, B. T., Soma, C. S., Tanana, M., Hirsch, T., Gibson, J., Georgiou, P., Narayanan, S., & Atkins, D. C. (2019). Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy. Advance online publication. https://doi.org/10.1037/pst0000221
Koole, S. L., & Tschacher, W. (2016). Synchrony in psychotherapy: A review and an integrative framework for the therapeutic alliance. Frontiers in Psychology, 7, 862. https://doi.org/10.3389/fpsyg.2016.00862
Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: A systematic review. General Hospital Psychiatry, 32(4), 345-359. https://doi.org/10.1016/j.genhosppsych.2010.03.006
Ladmanová, M., Řiháček, T., & Timuľák, L. (2022). Client-identified impacts of helpful and hindering events in psychotherapy: A qualitative meta-analysis. Psychotherapy Research, 32(6), 723-735. https://doi.org/10.1080/10503307.2021.2003885
Lambert, M. J. (2010). Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. American Psychological Association. https://doi.org/10.1037/12141-000
Lambert, M. J. (2015). Progress feedback and the OQ-System: The past and the future. Psychotherapy, 52(4), 381-390. https://doi.org/10.1037/pst0000027
Loulová, Š. (2021). Klasifikační systém pro počítačové zpracování terapeutovy řeči v rámci individuálních psychoterapeutických sezení [Diplomová práce]. Masarykova univerzita. https://is.muni.cz/auth/th/f0ux5/Loulova_Diplomova_prace.pdf
Lutz, W., Rubel, J. A., Schwartz, B., Schilling, V., & Deisenhofer, A.-K. (2019). Towards integrating personalized feedback research into clinical practice: Development of the Trier Treatment Navigator (TTN). Behaviour Research and Therapy, 120, 103438. https://doi.org/10.1016/j.brat.2019.103438
Lyon, A. R., Lewis, C. C., Boyd, M. R., Hendrix, E., & Liu, F. (2016). Capabilities and characteristics of digital measurement feedback systems: Results from a comprehensive review. Adm Policy Ment Health, 43, 441-466. https://doi.org/10.1007/s10488-016-0719-4
Meier, S. T. (2015). Incorporating progress monitoring and outcome assessment into counseling and psychotherapy: A primer. Oxford University Press. https://doi.org/10.1093/med:psych/9780199356676.001.0001
Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The Partners for Change Outcome Management System. JCLP/In Session, 61(2), 199-208. https://doi.org/10.1002/jclp.20111
Miller, S. D., Hubble, M. A., Chow, D. L., & Seidel, J. A. (2013). The outcome of psychotherapy: Yesterday, today, and tomorrow. Psychotherapy, 50(1), 88-97. https://doi.org/10.1037/a0031097
Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449-457. https://doi.org/10.1037/pst0000031
Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2), 1-135. https://doi.org/10.1561/9781601981516
Prescott, D. S., Measchalck, C. L., Miller, S. D. (Eds.). (2017). Feedback-informed treatment in clinical practice: Reaching for excellence. American Psychological Association. https://doi.org/10.1037/0000039-000
Rennie, D. L. (1994). Clients' deference in psychotherapy. Journal of Counseling Psychology, 41(4), 427-437. https://doi.org/10.1037/0022-0167.41.4.427
Řiháček, T., Elliott, R., Owen, J., Ladmanová, M., Coleman, J. J., & Bugatti, M. (2023). Session Reactions Scale-3: Initial psychometric evidence. Psychotherapy Research. Advance online publication. https://doi.org/10.1080/10503307.2023.2241983
Řiháček, T., & Matějka, P. (2021). Deep learning v psychoterapii: Strojová analýza nahrávek terapeutických sezení. E-psychologie, 15(3), 35-37. https://doi.org/10.29364/epsy.414
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298-311. https://doi.org/10.1037/a0019247
Solstad, S. M., Castonguay, L. G., & Moltu, C. (2019). Patients' experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature, Psychotherapy Research, 29(2), 157-170. https://doi.org/10.1080/10503307.2017.1326645
Straková, J., Straka, M., & Hajič, J. (2014). Open-source tools for morphology, lemmatization, POS tagging and named entity recognition. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations (pp. 13-18). Association for Computational Linguistics. https://doi.org/10.3115/v1/P14-5003
Swift, J. K., & Greenberg, R. P. (2014). A treatment by disorder meta-analysis of dropout from psychotherapy. Journal of Psychotherapy Integration, 24(3), 193-207. https://doi.org/10.1037/a0037512
Walfish, S., McAlister, B., O'Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110(2), 639-644. https://doi.org/10.2466/02.07.17.PR0.110.2.639-644
Widyassari, A. P., Rustad, S., Shidik, G. F., Noersasongko, E., Syukur, A., Affandy, A., & Setiadi, D. R. I. M. (2022). Review of automatic text summarization techniques & methods. Journal of King Saud University - Computer and Information Sciences, 34(4), 1029-1046. https://doi.org/10.1016/j.jksuci.2020.05.006
Zhang, J., Filbin, R., Morrison, C., Weiser, J., Danescu-Niculescu-Mizil, C. (2019). Finding your voice: The linguistic development of mental health counselors. https://arxiv.org/abs/1906.07194 https://doi.org/10.18653/v1/P19-1089
Copyright © 2023 Tomáš Řiháček, Jan Nehyba, Michal Čevelíček, Alexandr Polok, Pavel Matějka, Petr Doležal