One of the most frequent questions parents ask educational consultants, other referral agents, and the schools and programs themselves is, "What is your success rate?"  That's about the same as outcomes.   Although some outcome studies are beginning to appear, there are too few.   Our first concern is that schools and programs be continuously revaluating themselves on the basis of many factors, not the least of which is studying the effect they have had on their students and clients.   Without that information, schools and programs cannot properly evaluated their own performances. 

The studies that do appear, however, appear to have more of a marketing purpose than a self-evaluation focus.  We hope that will change.

In most cases, the studies look at too short a time after discharge /graduation/ withdrawal and are not based on sufficiently objective data.  We don't want to be hypercritical when we really want to applaud these beginning steps.  But we also want to be clear that we need to progress from the present point. 

Some studies have used a format and methodology designed that precisely matches outcome studies used in the past to attempt demonstrate that residential and boarding programs are no more effective than outpatient services.  Accordingly those studies use identical or nearly identical questions and methodology so there would be a direct comparison.  We understand and applaud those studies, but hope for further studies that look at outcomes five and ten years later, validated by information sources that are likely to be more objectives than parents contacted by phone. 

In particular, we would like to see studies that compare the relationship of length of stay and adding on transition services and effective family therapy into the mix of services to outcomes.  With any given set of presenting facts, what lengths of stay produce best results? Are outcome enhanced by transition services?  By family therapy? 

 There was a time that programs considered it an invasion of client privacy to contact discharged clients to learn about outcomes.  Fortunately, that trend is changing. 

Currently it is our understanding that NATSAP has created a system for  gathering and storing data to facilitate outcome research.  Whether or not that is true, our guidelines call for schools and programs to gather outcome data that matches data in well known studies according the methodology of those studies so that people and institutions doing research can access this data. We call on the larger for-profit organizations such as Aspen, CRC Health, and Universal Health Systems to create and fund a blind trust to study collected data to answere some of the questions raised above. 

Our criterion for Outcomes is that all programs should either participate in NATSAP's information gathering or develop their own outcome data collection system. There is some question as to whether or not  NATSAP will permit its data to be used for program  specific studies, but rather intends the data to be used for classes of programs only.  If that is correct, we see value in studies by class of facility, but we also think individual programs need to be accountable for delivering what they claim and that they have have the opportunity to learn their own strengths and weakness from these studies, for purposes of quality improvement.

While our initial guidelines only call for data collection so that the data will be research-ready we hope to see the time that it will be practical to have guidelines that call for all schools and programs to have long term outcome studies that are an essential part of their quality improvement programs. 

Last updated 1-17-09;