The NICE Guideline for Depression in Adults: What is all the brouhaha about methodologies?
In the original symposium- ‘Using Research to Impact Counselling and Psychotherapy Practice and Mental Health Policy’ Felicitas talked about her work leading a coalition of almost 50 mental health and psychotherapy organizations to challenge NICE on the development of the NICE Guideline for Depression. Since then, the revised guideline draft (the third) has been released. For an update on the argument the coalition are making see the related blog.
In her symposium talk Felicitas outlined that the original motivation behind the NICE Guideline coalition was not an objection to the treatment recommendations but an objection to the methodologies being used by NICE to make their recommendations. And even though there have been some tweaks in the methodologies used to develop the third guideline draft, the coalition remains dubious.
Brouhaha # 4: NICE methods are not nice
NICE states on its website that: “NICE guidelines are evidence-based recommendations for health and care in England” (NICE Guidelines, 2021[N1] ) so it is problematic when the ranking of recommended treatments is not actually evidence-based. However, even if you can get past that, there are a long litany of problems with the methodological approach taken by NICE to the guidelines that the coalition have raised. To name just three:
- The NICE methodology manual (which dictates how NICE develops its guidelines) prioritises evidence from randomised control studies (RCTs) over all other kinds of ‘evidence’ and so did the depression guideline development team. This means the guideline analyses rest on mostly smaller studies, typically conducted in research settings, mostly outside the UK (e.g. 6% of the 534 RCTs included in the evidence review for more severe depression were conducted in the UK). Yet, the aim of the NICE guidelines is to improve outcomes for NHS patients and there is a huge wealth of information available about the outcomes of NHS psychological therapies patients. Given that the depression guideline directly shapes delivery of psychological therapies in NHS primary care, it seems frankly perverse that NICE is not considering any of the data about actual practice in the actual NHS, let alone the data that shows how the 2009 guideline recommendations have played out over the last decade plus.
- NICE and the NHS both place considerable emphasis on the importance of listening to patient experience. The initial update of the 2009 depression guideline failed in any meaningful way to consider patient experience by failing to update the section on patient experience. The 3rd consultation draft does include a systematic review of qualitative studies of patient views on treatment choice – this is a good thing. However, there is still nothing on how patients /service users experience depression treatment. The coalition has argued all along that if NICE is serious about listening to service users/patients then it needs to actually include research on what people who have had psychological therapy for depression say about it (e.g. Finazzi & MacBeth, 2021). Doing so is critical to help patients and GPs make decisions about what treatment might be right for them, e.g. to support individualised care and reduce inequalities.
- The primary method used by the depression guideline development team is network meta-analysis (NMA). The coalition have consistently argued that the NMAs as conducted evidence numerous violations of statistical assumptions and other methodological shortcomings, which suggest that the conclusions drawn have to be viewed with absolute caution/may not be valid. This implicates also the health economic analyses, as these are based on the results derived from the NMA.
Beyond the problems with the NMAs conducted for the draft guideline, the coalition is arguing a broader problem with the methodology – namely that the validity or trustworthiness of statistical evidence derived from NMA is controversial (Faltlinsen et al., 2018; Leucht et al., 2016). It is the view of the coalition that the lack of formal expert consensus on NMAs and that it has never been used on such a complex dataset (i.e. so many different kinds of treatments) means that their use in the depression guideline is basically experimental and that a national health treatment guideline should not be based on an experimental technique.
Watch this space: NICE’s response to the feedback from stakeholders will be released later this year along with – probably – the final version of the guideline. Until the guideline is released it is unknown whether NICE will make further revisions to the guideline. It is also not yet clear how the organizations in the stakeholder group will respond – whether they will continue to protest or accept a guideline they have said publicly is flawed.
[N1]National Institute for Health and Care Excellence (2021). NICE Guidelines. https://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-guidelines
Please see below the original blog about the symposium.