THE MacARTHUR VIOLENCE RISK ASSESSMENT STUDY

September 2005 Update of the Executive Summary

A study funded by the National Institute of Mental Health employed the public-assess data from the MacArthur Violence Risk Assessment Study to develop violence risk assessment software, and also validated that software on independent samples of patients. The results of this validation study have been published:

Monahan, J, Steadman, H., Robbins, P., Appelbaum, P., Banks, S., Grisso, T., Heilbrun, K., Mulvey, E., Roth, L., and Silver, E. (2005). An actuarial model of violence risk assessment for persons with mental disorders. Psychiatric Services, 56, 810-815.

The software, called the Classification of Violence Risk (COVR) is available from Psychological Assessment Resources, Inc. 16204 N. Florida Avenue, Lutz, FL 33549. Phone: 800 331 8378. Web: www.parinc.com

May 2004 Update of the Executive Summary

The principal findings of the MacArthur Violence Risk Assessment Study were summarized and integrated in Monahan, J., Steadman, H., Silver, E., Appelbaum, P., Robbins, P., Mulvey, E., Roth, L., Grisso, T., & Banks, S. (2001). Rethinking Risk Assessment: The MacArthur Study of Mental Disorder and Violence. New York: Oxford University Press.

Since the publication of Rethinking Risk Assessment in 2001, Network members have published a number of articles that have reported further analyses of the dataset:

Robbins, P., Monahan, J., & Silver, E. (2003). Mental disorder and violence: The moderating role of gender. Law and Human Behavior, 27, 561-571.

Skeem, J., Monahan, J., & Mulvey, E. (2002).Psychopathy, treatment involvement, and subsequent violence among civil psychiatric patients. Law and Human Behavior, 26, 577-603.

Skeem, J., Miller, J., Mulvey, E., Tiemann, J., & Monahan, J. (submitted for publication). Using a Five Factor lens to sharpen focus on the relation between personality traits and violence in psychiatric patients.

Skeem, J. L., & Mulvey, E. P. (2001). Psychopathy and community violence among civil psychiatric patients: Results from the MacArthur Violence Risk Assessment Study. Journal of Consulting & Clinical Psychology, 69, 358-374.

Skeem, J. L., Mulvey, E. P., & Grisso, T. (2003). Applicability of traditional and revised models of psychopathy to the Psychopathy Checklist: Screening Version. Psychological Assessment, 15, 41-55.

In April of 2001, the entire dataset from the MacArthur Violence Risk Assessment Study was made publicly available on the web at www.macarthur.virginia.edu/risk.html

Since that time, the dataset has been downloaded by over 500 people using it for their own re-analyses.

Questions regarding the forthcoming availability of violence risk assessment software developed under a National Institute of Health grant from the publicly-available data set of the MacArthur Violence Risk Assessment Study should be directed to John Monahan (jmonahan@virginia.edu).

Executive Summary

April 2001

Beliefs about the causes of mental disorder have shifted over the centuries, but the belief that mental disorder predisposes many of those suffering from it to behave violently has endured. Indeed, this belief appears to have increased in intensity in the past several decades, despite many educational campaigns designed to allay public apprehension.

These perceptions are reflected both in formal policies toward people with mental disorders and in the public’s expectations about the role of mental health professionals in insuring the safety of the community. Violence risk assessment now is widely assumed by policy makers and the public to be a core skill of the mental health professions and plays a pivotal role in mental health law throughout the world. “Dangerousness to others” is now a principal standard for inpatient commitment, outpatient commitment, and commitment to a forensic hospital. The imposition of tort liability on mental health professionals who negligently fail to anticipate and avert a patient’s violence to others has become commonplace.

Despite the pervasiveness of violence risk assessment in mental health law, research continues to indicate that the unaided abilities of mental health professionals to perform this task are modest at best. Many have suggested that making available to clinicians statistical (“actuarial”) information on the empirical relationships between various risk factors and subsequent violent behavior is the only way to reduce the disconnect between what the law demands and what clinicians currently are able to provide.

General Research Strategy

The MacArthur Violence Risk Assessment Study[1] had two core goals: to do the best “science” on violence risk assessment possible, and to produce an actuarial violence risk assessment “tool” that clinicians in today’s world of managed mental health services could actually use. From these initial intellectual commitments, the Study evolved in six stages over the decade it took to plan, execute, and analyze the research.

Identifying Gaps in Methodology.  Almost all existing studies of violence risk assessment suffer from one or more methodological problems:  they considered a constricted range of risk factors, often a few demographic variables or scores on a psychological test; they employed weak criterion measures of violence, usually relying solely on arrest; they studied a narrow segment of the patient population, typically males with a history of prior violence; and they were conducted at a single site.  Based upon this critical examination of existing work, we designed a piece of research that could, to the greatest extent possible, overcome the methodological obstacles we had identified.  We studied a large and diverse array of risk factors.  We triangulated our outcome measurement of violence, adding patient self-report and the report of a collateral informant to data from official police and hospital records.  We studied both men and women, regardless of whether they had a history of violence.  And we conducted our study at several sites rather than at a single site.

Selecting Promising Risk Factors.    Although we lacked any comprehensive theory of violence by people with mental disorder from which we could derive hypothesized risk factors, recent studies suggested that a number of variables might be potent risk factors for violence among people with a mental disorder.  We assessed personal factors (e.g., demographic and personality variables), historical factors (e.g., past violence and mental disorder), contextual factors (e.g., social support and social networks), and clinical factors (e.g., diagnosis and specific symptoms).   We chose what we believed to be the best of the existing measures of these variables, and where no instrument to adequately measure a variable was available, we commissioned the development of the necessary measure.

Using Tree-Based Methods. We developed violence risk assessment models based on the “classification tree” method rather than the usual linear regression method. A classification tree approach reflects an interactive and contingent model of violence, one that allows many different combinations of risk factors to classify a person at a given level of risk.  The particular questions to be asked in any assessment grounded in this approach depend on the answers given to prior questions.  Factors that are relevant to the risk assessment of one person may not be relevant to the risk assessment of another person. This contrasts with a regression approach in which a common set of questions is asked of everyone being assessed and every answer is weighted to produce a score that can be used for purposes of categorization.

Creating Different Cut-Offs for High and Low Risk. Rather than relying on the standard single threshold for distinguishing among cases, we decided to employ two thresholds – one for identifying higher risk cases and one for identifying lower risk cases.  We assumed that inevitably there will be cases that fall between these two thresholds, cases for which any actuarial prediction scheme is incapable of making an adequate assessment of high or low risk.  The degree of risk presented by these intermediate cases cannot be statistically distinguished from the baserate of the sample as a whole (therefore, we refer to these cases as constituting an “average risk” group).

Repeating the Classification Tree.    To increase the predictive accuracy of a classification tree, we re-analyzed those cases designated as “average risk”.  That is, all people not classified into groups designated as either high- or low-risk in the standard classification tree model were pooled together and re-analyzed.  The logic here was that the people who were not classified in the first iteration of the analysis might be different in some significant ways from the people who were classified, and that the full set of risk factors should be available to generate a new classification tree specifically for these people who were not already classified as high or low risk.  We referred to the resulting classification tree model as an “iterative” classification tree).

Combining Multiple Risk Estimates. Finally, we estimated several different risk assessment models in an attempt to obtain multiple risk assessments for each case.  That is, we chose a number of different risk factors to be the lead variable upon which a classification tree was constructed. In attempting to combine these multiple risk estimates, we began to conceive of each separate risk estimate as an indicator of the underlying construct of interest, violence risk.  The basic idea was that patients who scored in the high risk category on many classification trees were more likely to be violent than patients who scored in the high risk category on fewer classification trees. (And analogously, patients who scored in the low risk category on many classification trees were less likely to be violent than patients who scored in the low risk category on fewer classification trees). 

Specific Research Methods

Admissions (n=1,136) were sampled from acute civil inpatient facilities in Pittsburgh, PA, Kansas City, MO, and Worcester, MA. We selected English-speaking patients between the ages of 18 and 40, who were of White, African American, or Hispanic ethnicity,  and who had a chart diagnosis of thought or affective disorder, substance abuse, or personality disorder. The median length of stay was 9 days. After giving informed consent to participate in the research, the patient was interviewed in the hospital by both a research interviewer and a research clinician in order to assess him or her on each of the risk factors.

Three sources of information were used to ascertain the occurrence and details of a violent incident in the community.  Interviews with patients, interviews with collateral individuals (i.e., persons named by the patient as someone who would know what was going on in his or her life), and official sources of information (arrest and hospital records) were all coded and compared. For the analyses reported here, the patients and collaterals were interviewed twice (every 10 weeks) over the first 20 weeks -- approximately 4-5 months -- from the date of hospital discharge.

Violence to others was defined to included acts of battery that resulted in physical injury; sexual assaults; assaultive acts that involved the use of a weapon; or threats made with a weapon in hand.

Results

At least one violent act during the first 20 weeks after discharge from the hospital was committed by 18.7 percent of the patients we studied. Of the 134 risk factors we measured in the hospital, approximately half (70) had a statistically significant bivariate relationship with later violence in the community (p<.05). Some examples of specific risk factors that were  – or were not – significantly related to violence:

 •       Gender.  Men were somewhat more likely than women to be violent, but the difference was not large. Violence by women was more likely than violence by men to be directed against family members and to occur at home, and less likely to result in medical treatment or arrest.

•        Prior violence. All measures of prior violence – self-report, arrest records, and hospital records – were strongly related to future violence.

 •         Childhood experiences.  The seriousness and frequency of having been physically abused as a child predicted subsequent violent behavior, as did having a parent – particularly a father – who was a substance abuser or a criminal.

 •         Neighborhood and race. While there was an overall association between race and violence, African Americans and whites who lived in comparably disadvantaged neighborhoods had the same rates of violence.

 •      Diagnosis.  A diagnosis of a major mental disorder -- especially a diagnosis of schizophrenia -- was associated with a lower rate of violence than a diagnosis of a personality or adjustment disorder. A co-occurring diagnosis of substance abuse was strongly predictive of violence.

 •         Psychopathy. Psychopathy, as measured by a screening version of the Hare Psychopathy Checklist,  was more strongly associated with violence than any other risk factor we studied. The “antisocial behavior” component of psychopathy, rather than the “emotional detachment” component, accounted for most of this relationship.

•      Delusions.  The presence of delusions – or the type of delusions or the content of delusions – was not associated with violence. A generally “suspicious” attitude toward others was related to later violence.

•      Hallucinations. Neither hallucinations in general, nor “command” hallucinations per se, elevated the risk of violence. If voices specifically commanded a violent act, however, the likelihood of violence was increased.

•        Violent thoughts. Thinking or daydreaming about harming others was associated with violence, particularly if the thoughts or daydreams were persistent.

 •         Anger. The higher a patient scored on the Novaco Anger Scale in the hospital, the more likely he or she was to be violent later in the community.

These are only bivariate relationships between risk factors measured in the hospital and violence during the first 20 weeks after discharge into the community, however. How do the risk factors perform when combined in a multivariate way? Recall that we did not use regression-based methods to combine variables, but rather employed the classification tree technique.

Rather than pitting different tree-based risk assessment models against one another and choosing the one model that  appears “best,” we used an approach that integrates the predictions of many different risk assessment models, each of which may capture a different but important facet of the interactive relationship between the risk factors and violence. Using this approach, we ultimately combined the results of five prediction models generated by the Iterative Classification Tree methodology.  This combination of models produced results not only superior to those of any of its constituent models, but superior to any other actuarial violence risk assessment procedure reported in the literature to date. Using only the 106 risk factors commonly available in hospital records or capable of being routinely assessed in clinical practice, we were able to place all patients into one of five risk classes for which the prevalence of violence during the first 20 weeks following discharge into the community varied between 1 percent and 76 percent. The risk factors that emerged most frequently on the various models are presented in Table 1, and the five risk groups that materialized from the use of these risk factors -- along with the numbers of patients who fell into each risk group -- are presented in Figure 1.

 

Conclusions

The approach to risk assessment developed in the MacArthur Violence Risk Assessment Study appears to be highly accurate when compared to other approaches to assessing risk among people hospitalized in acute-care psychiatric facilities. But it is also much more computationally complex than other approaches. Five tree-based prediction models need to be constructed, each involving the assessment of many risk factors. It would clearly be impossible for a clinician to commit the multiple models and their scoring to memory, since different risk factors are to be assessed for different patients, and using a paper-and-pencil protocol would be very unwieldy. Fortunately, however, the administration and scoring of multiple tree-based models lends itself to software. In clinical use, the risk assessment instrument we have developed would consist simply of a series of questions that would flow one to the next on a computer screen -- through the various iterations of each of the models as necessary -- depending on the answer to each prior question.  Under a grant from the National Institute of Mental Health, we are currently in the process of testing a prototype of such “violence risk assessment software.” The software should be available in 2003. Further information will be available at this website.

Publications

Appelbaum, P., Robbins, P., & Monahan, J. (2000). Violence and delusions: Data from the MacArthur Violence Risk Assessment Study. American Journal of Psychiatry, 157, 566-572.

Appelbaum, P., Robbins, P., & Roth, L (1999). A dimensional approach to the assessment of delusions. American Journal of Psychiatry, 156, 1938-1943.

Banks, S., Robbins, P., Silver, E., Vesselinov, R., Steadman, H., Monahan, J., Mulvey, E., Appelbaum, P., Grisso, T., & Roth, L. (in press). A multiple models approach to violence risk assessment among people with mental disorder. Criminal Justice and Behavior.

Grisso, T., Davis, J., Vesselinov, R., Appelbaum, P., & Monahan, J. (2000). Violent thoughts and violent behavior following hospitalization for mental disorder. Journal of Consulting and Clinical Psychology, 68, 388-398.

Monahan, J., & Appelbaum, P. (2000). Reducing violence risk: Diagnostically based clues from the MacArthur Violence Risk Assessment Study. In S. Hodgins (Ed.), Effective prevention of crime and violence among the mentally ill.  Dordrecht, The Netherlands:  Kluwer Academic Publishers (pp. 19-34).

Monahan, J., Appelbaum, P., Mulvey, E., Robbins, P., & Lidz, C. (1994). Ethical and legal duties in conducting research on violence: Lessons from the MacArthur Risk Assessment Study.  Violence and Victims, 8, 380-390. 

Monahan, J., & Steadman, H. (eds) (1994). Violence and mental disorder: Developments in risk assessment. Chicago: University of Chicago Press. 

Monahan, J., Steadman, H., Appelbaum, P., Robbins, P., Mulvey, E., Silver, E., Roth, L., & Grisso, T. (2000).Developing a clinically useful actuarial tool for assessing violence risk. British Journal of Psychiatry, 176, 312-319.

Monahan, J., Steadman, H., Silver, E., Appelbaum, P., Robbins, P.,  Mulvey, E.,  Roth, L., Grisso, T., &  Banks, S. (2001). Rethinking risk assessment: The MacArthur study of mental disorder and violence. New York: Oxford University Press.

Robbins, P., Monahan, J., & Silver, E. (2001). Mental disorder,  violence, and gender. Submitted for publication.

Silver, E. (2000).  Race, neighborhood disadvantage, and violence among persons with mental disorders: The importance of contextual measurement. Law and Human Behavior, 24, 449-456.

Silver, E. (2000). Extending social disorganization theory: A multilevel approach to the study of violence among persons with mental illnesses. Criminology, 38, 301-332.