| Summary: | <p><strong>Background</strong></p>
<p>Clinical care is often guided by prognosis. Understanding the chances of future outcomes associated with a health condition can inform decisions around support and treatment for individuals, and decisions at a clinical service or policy level. There is wide interest in refining individualised prognostic understanding to personalise care and inform stratified treatment, where interventions are targeted for maximum benefit. Prediction models are a key part of this progress. These models combine information about predictors of an outcome into a statistical model to estimate the risk of that outcome for an individual, and have had a substantial impact on clinical practice in areas such as cardiovascular medicine. However, many models that are developed are never adopted into practice. This can be due to limitations in development study methodology, or neglect of the stepwise process of translating into a usable clinical tool. Ideally, this should involve validating the accuracy of the model in a clinical setting, and considering practical issues such as its acceptability, feasibility and role.</p>
<p>An important adverse outcome in psychiatry that may benefit from a prediction model approach is violence perpetration. Violence can have significant implications for patients and others, and is a target for prevention at a public health level. There is a small but significant increased risk of violence perpetration associated with many mental disorders, including schizophrenia spectrum disorders. First episode psychosis has been identified as a higher risk phase of illness, and so this outcome is of particular relevance to community mental health teams called Early Intervention in Psychosis (EIP) services who engage individuals presenting at this early stage. These services do not specialise in violence risk assessment and management however. Currently there are no established ways to support this risk assessment, and how it is undertaken in practice is not well understood.</p>
<p>Whilst many tools to assess violence risk exist in other settings such as forensic psychiatric services or justice settings, there are limitations in how they have been developed, and barriers to their application in EIP services such as the resource required to use them. However, a potentially more scalable model, called “OxMIV”, was transparently developed using contemporary prediction modelling approaches. This may be more suited to supporting violence risk assessment in EIP services, but whether and how it could be translated into practice is yet to be explored.</p>
<p><strong>Aims</strong></p>
<p>The aims of this thesis are to first understand current clinical approaches to the assessment and management of violence risk in EIP services, and what the difficulties and clinical needs associated with this are. Whether the OxMIV violence risk prediction tool can meet some of these needs will then be examined. Preliminary clinical use will address practical aspects of its feasibility and acceptability, develop its optimal clinical role, and establish the potential impact of its implementation. The predictive performance of the model in this setting will be examined by externally validating it in individuals assessed by EIP services, and updating the model as necessary. Finally, a framework for how the tool could be integrated into a stratified approach to violence risk, and how the impact of this could be tested, will be developed. Through this work, an overarching aim is to develop methodological approaches that address the gap between the development of a prediction model and its translation into clinical practice.</p>
<p><strong>Methods</strong></p>
<p>Qualitative methodology (Chapter 2) is used to examine current practice and challenges through individual interviews with 18 multidisciplinary clinicians working in EIP services (Chapter 3). Data is analysed thematically, informed by the constant comparative method.</p>
<p>Mixed methods (Chapter 2) are then used to examine the feasibility, acceptability and role of the OxMIV tool when made available to clinicians in the electronic health record (Chapter 4). Individual interviews are conducted with 20 clinicians and 12 patients and carers. Quantitative data on the use of the tool is also collected, in a convergent parallel design. Quantitative and qualitative data is integrated at the level of interpretation and reporting, using a joint display matrix to draw conclusions that incorporate their combined meaning.</p>
<p>A separate external validation study (Chapter 5) uses routine electronic health record data and police data on violent occurrences to examine the performance of OxMIV in a retrospective cohort of 1,145 individuals consecutively assessed by EIP services between 2012 and 2018. Multiple imputation is used to address missing data, and measures of discrimination, calibration and overall performance are reported. A stepwise process of model updating is undertaken. Data from the electronic health record on the unstructured clinical judgement made by the original assessing clinician is used to further consider the potential role and benefit of OxMIV.</p>
<p><strong>Results</strong></p>
<p>Violence is viewed as an important outcome by clinicians working in EIP services, though relevant in a minority of patents (Chapter 3). Currently assessment is a brief screen, with fuller exploration if the assessing clinician deems it relevant, involving unstructured information-gathering focussed on the content of psychotic symptoms and individual circumstances. Any linked responses are practical and focus on crisis situations rather than modifying risk in the longer term. Challenges include low clinician confidence, perceived subjectivity, limitations of training and knowledge, wariness around stigma, limited collaboration with patients, issues around obtaining forensic history, and difficulties in how to describe and document risk. Clinicians have varied views on the importance of different risk factors, particularly static factors.</p>
<p>In Chapter 4, OxMIV was found to be a practically feasible approach to address some of these difficulties in EIP services. It was used in a considerable proportion of assessments over 12 months, and clinicians found it quick and straightforward. Required information was available, and the output typically aligned with clinical judgement. In most cases it was deemed to helpfully support the overall assessment, particularly if there were already concerns about violence risk. Clinicians did not require OxMIV to alter their judgement to find it helpful and they reported broad impacts on their practice, including confidence, knowledge and consistency. The summary percentage score provided by OxMIV was felt to provide richer clinical information to focus and clarify the clinical view. Patients and carers felt that using a structured tool was an acceptable approach, and encouraged clinicians to be straightforward in their discussion of violence risk. Modifications in careplans were more likely when risks were increased. OxMIV did not affect whether clinicians collaborated with patients and families. Linked treatment planning was identified as the process on which to focus efforts to improve collaboration. Adding therapeutic suggestions to the tool was deemed a palatable way for a future iteration to improve the linking of risk assessment to approaches to reduce risk.</p>
<p>In the external validation study (Chapter 5), violence perpetration occurred in 11% of the cohort. OxMIV’s discriminative performance was robust (area under a receiver operating characteristic curve [AUC] of 0.75), demonstrating its transportability. Calibration was adequately corrected by updating the intercept of the model, without changing any of the predictor coefficients. Unstructured clinical assessment identified violence risk with low sensitivity in the external validation cohort, with only 40% of those individuals who perpetrated violence having had any violence risk identified in their clinical risk assessment. Using a 10% cutoff for designating increased risk, sensitivity of OxMIV was 71%, specificity 66%, positive predictive value 22% and negative predictive value 95%. OxMIV showed net benefit over clinical assessment and default clinical strategies.</p>
<p><strong>Conclusions</strong></p>
<p>Violence perpetration is an important but challenging adverse outcome for EIP services to assess and manage within the limits of their remit and resource. OxMIV has been externally validated and updated as a scalable risk prediction tool to support these challenges. Its practical use is feasible within EIP services, and acceptable in a range of domains to clinicians, patients and carers. Its optimal clinical role is as a complement to routine new assessments, to improve the currently low sensitivity with which needs around violence risk are identified and supported. An updated model is presented in Chapter 6 along with a guide for its use and linked therapeutic suggestions. This includes prompts to shift emphasis toward considering risk more longitudinally, improve collaboration with patients and families, and support the seeking of collateral history about past convictions where appropriate. A framework for how this can be integrated and studied in future work is provided.</p>
<p>Through this work, methodological progress has been made in an important gap in the prediction model literature, that of the stepwise translation of a model to a clinical tool. This includes the use of routine electronic health record data to examine external validity, and novel mixed methods approaches to study the clinical feasibility, acceptability and role of a prediction model. Future work should develop and apply these methods to other prediction models in EIP services, psychiatry and beyond, and seek to examine in comparative clinical trials the impact of prediction models when they are integrated into clinical systems as complex interventions.</p>
|
|---|