The debate over psychotherapy as a science, set off by Eysenck’s criticisms of psychoanalysis halfway through the last century, has been extended to cover in general the scientificness of all treatments that try to modify the psyche by using psychic methods. This latter specification seems to be essential because treatments that modify the psyche through chemical means (pharmacological therapies are the most common today) are not questioned in terms of scientificness, since the validity attributed to clinical trials on the effectiveness of pharmaceuticals, while taken for granted, is (in the opinion of many) not always deserved.
With reference to psychotherapy, it must first of all be asked whether the criteria of scientificness required for the control of these treatments can be the same as those used in other sciences.
In general, one can define a research practice scientific if it is open to intersubjective control, if it gives clear definitions of concepts and postulates and uses interpretable, repeatable procedures, using a rationally grounded method for the validation of the theoretical hypotheses. According to Ajdukiewicz’s well-known definition, knowledge is scientific when it can be intersubjectively tested and communicated.
Does psychotherapy respond to the requirements of the other scientific disciplines? To do so, it should:

• give clear definitions of concepts and postulates;
• use interpretable, repeatable procedures;
• be open to intersubjective testing;
• use a rationally grounded method for the validation of hypotheses. 

Psychotherapy as a science should consequently be based on the following epistemological presuppositions:

• psychotherapy is a process generally aimed at changing the components of the mind that create suffering and/or maladjustment, with specific objectives deriving from the theoretical model adopted by the therapist;
• the objectives set are pursued in each theoretical model using specifically  useful techniques, i.e. the most suitable and economical in view of the subject’s situation and the context;  these techniques can be interpreted and repeated by the outside world so as to ensure intersubjectivity of control;
• the change is produced within a relation between ‘subjects’ (therapists/clients) whose aspects can be evaluated and explained both in theoretical and in factual terms.

As far as the method of validating hypotheses is concerned, the scientific approach can take three main modes:

• checking whether the new knowledge generated by the hypotheses is consistent with all the knowledge present in the same theory or research field (logical validation, typical of the ‘formal’ sciences);
• ascertaining, through a series of data drawn from experiences in which the variables are as far as possible kept under control, if and to what degree the hypotheses ‘work’ and are useful for the aim set (empirical pragmatic validation);
• testing hypotheses that set out to explain, always through research controlling the variables involved, why a certain process works (empirical explicative validation).

Similarly to the three types of validation outlined above for science in general, in psychotherapy it is also possible to use a threefold procedure:
1. Logical validation. An evaluation is made of the congruency between the theory of the pathology, the objectives of the psychotherapy, and the techniques actually used. A scientific model of therapy deduces objectives and techniques from a theory of the pathology which is in turn scientifically validated.
Typical examples of this approach are: the use of interpretation of dreams in  psychoanalytic theory, congruent with a theory which holds that dreams bring out latent contents which cannot otherwise be expressed; the modification of dysfunctional ideations in the cognitivist model, which considers them the source of pathologies; ‘modeling’ of behaviour in behaviorism, which considers this technique the basis of new functional learning opposed to disadaptive learning, and so on.
2. Empirical pragmatic validation. Here the result obtained by means of a therapeutic technique is evaluated in the light of criteria of socially agreed efficacy/usefulness. The efficacy of the method of treatment constitutes a validation of the theoretical statements and the benefits obtained are assessed by comparing the results of the initial and final assessments, and if possible also the follow-up stage. This is what, in psychotherapy research literature, is called testing effects.
The aim is to ascertain whether a therapeutic intervention actually reduces the symptoms and produces in the client a more integrated and adjusted personality balance.  On the empirical plane this is the approach typical of outcome research, which was dominant in the first studies of the efficacy of psychotherapy and today is still used to demonstrate the social utility of psychotherapy.
3. Empirical explicative validation. Here the hypotheses on the psychopathological roots of the client’s problems and on the strategies of change are periodically tested in phases of assessment, at times t1, t2, t3, ... tn; these checks are linked both with  the effects and with the theoretical reference-points, so as to systematically bring out the   ‘sense’ of what happens in therapy in relation to the operations carried out there.
From this process research standpoint, the monitoring of the therapy enables us to ascertain:

• what is therapeutic and for whom;
• not only if and to what degree, but also why a treatment works with a certain type of patient but does not work with others;
• what psychic processes are activated and what aspects of personality are modified.

The parallel study of both the outcome and the psychotherapy process may allow us to get beyond the choice, often unjustly exaggerated, between the nomothetic  - i.e. the objective, the quantitative, the rule-setting – and the idiographic, i.e the subjective, the qualitative, the single and unrepeatable.
In psychotherapy with a person reporting a malaise or symptoms of depression, the first step, involving a nomothetic approach, is to ascertain whether the person is actually depressed in terms of the diagnostic criteria and how far removed he is from the threshold of normality according to a psychometric depression test; thus establishing or ruling out – always on psychometric grounds – the co-occurrence with other nosographic categories (anxiety, psychosis, etc.). The idiographic approach on the other hand serves to describe, in the initial assessment, the type of depression, the causes leading to it, the ideative and emotional aspects characterising it, in short what makes that specific person a subject of psychotherapy, the plan of which must take these particular characteristics into account. There is an appropriate set of tools – from particular categories of specially designed tests, to the clinical conversation, to systematic observation – that can guarantee this second part of the clinical-diagnostic approach.
To assess how well the therapy works, a nomothetic, quantitative approach is also required in order to establish to what degree the client’s symptoms and perceptions change after treatment; whether on the basis of the tests repeated and compared to the threshold values, the client can be deemed to have re-entered the area of normality, or at any rate whether the difference from the norm is significantly lower than in the pre-test; whether these changes are maintained over time. But a qualitative approach is also needed to follow the transformation of the peculiar variables of that particular patient and the progress of the relation that this transformation produces and maintains over time.
In this parallel use of quantitative and qualitative evaluations, the criteria for scientific validation are different but convergent.
The scientific assessment of the therapy must necessarily take into account the criterion of overcoming – or significantly reducing – the problems that led the client to undertake it or that emerged during the therapy (criterion of effectiveness)2.
However the unrepeatability of what happens during the intervention makes it necessary to place, alongside the normative criteria, different, qualitative and process-based criteria of evaluation. Regularity and co-occurrence (which can also be established statistically) may emerge from the cumulative analysis of repeated checks on the individual therapeutic processes.
The aspects of regularity and uniqueness may be combined as long as methodological tools are found for the rigorous study of the diachronic nature of the process of change, the tendencies found in them are monitored and therefore the sense of what happens in the psychotherapy is reconstructed.
Quantitative tools, like psychometric tests or observation grids, are needed to obtain scores ‘calibrated’ to specific populations, but also qualitative techniques to follow the process-based variables of the intervention: grids to evaluate what happens in the therapy, to be compiled by the subject him/herself and/or by the therapist or other observer; reports or transcripts of what happens in therapy, of the verbalizations, of the non-verbal and relational components; techniques based on imagination, drawing, games and role-playing for children and teenagers. The qualitative approach needs tools designed for the in-depth examination of the semantic elements. To study the working of an instance of psychotherapy it may be useful to collect information through a conversation at the end of the treatment, on what happened during the treatment itself, regarding variables such as the relationship with the therapist, the feeling of the therapy’s effectiveness, the satisfaction of expectations, etc.3 At the same time, the report on the experience can also be elicited from the therapist so as to compare the two evaluations and bring out the sense of the treatment just ended.

To illustrate the discourse presented so far, it can be applied to the central stage of any therapeutic approach: the relationship that is established between the ‘therapist’ subject and the ‘client’ subject (which can also be a group or a family).
Is the relationship the ineffable element in psychotherapy? Or can it be the object of a systematic scientific study that is coherently communicable? With what criteria, and using what tools?
Recent research disproves the old commonplace that says whatever happens in the setting of the therapeutic relationship is inevitably inexpressible in logical terms and therefore substantially impossible to communicate at a scientific level. Certainly, if what happens in the mental and experiential space underlying the relational setting were truly ‘ineffable’, being placed on a level of elaboration that logical thought cannot express without devastating distortions, there would be no way of avoiding the risks of self-referentiality and hermeneutic circularity that have often been attributed to psychotherapy. However, the scholars who have explored the impervious regions of the therapeutic relationship believe that they can be visited by avoiding both the dogmatic listing of principles and rules and the way of transmitting knowledge typical of   the art workshop (the logic of “look and learn”: hardly scientifically based training experiences!).
But can the therapeutic relationship be an issue – not only of expressible reflection – but of real scientific research, i.e. of the acquisition of knowledge, from the viewpoint of testing hypotheses and of intersubjective debate?
Also from this angle there is valid evidence concerning the hypothesis that the relational setting is accessible to the confrontation and sharing of knowledge, in that it is the fundamental node in scientific research – even purely theoretical – into  therapeutic action, the meeting-point between subject-therapist and subject-client, and between these and the universe of socially-determined meanings containing their relationship. 
With these premises, it is legitimate to ask a more delicate question: can the therapeutic relationship be the subject of empirical research, in both the pragmatic and explicative sense?
Again, recent studies on the issue confirm that there are possibilities, although many of them seem still to be waiting to be brought to fruition. To return to the metaphor of the journey of exploration, it is like glimpsing fertile fields that are underused and imagining the products that could be harvested from their intensive cultivation.  But one must ask oneself why this productive effort is still underdeveloped. What are the reasons that the practice of empirical testing of hypotheses about the relationship, and more generally about the therapeutic process, is not common and is viewed with diffidence, if not with suspicion, both by experimentalists and clinicians?
One reason lies in the tendency to identify the empirical viewpoint with the experimental one and the latter with the collection and analysis of quantitative data. Criticising this tendency, Jaspers almost a century ago wrote that from the natural sciences, which are exact sciences, we get the prejudice that only what can be proven in a quantitative way can constitute scientific work, while what is studied from the  qualitative viewpoint will always remain something subjective and arbitrary (from It. transl. 1964).
The empirical = experimental = quantitative chain of identifications is certainly inappropriate, since:

• the non-experimental methodologies (such as observative or self-observative, provided the criteria are standardized and acceptable to other scholars) are strategies of empirical research just as valid – if used properly – as those requiring the manipulation and control of independent variables;
• the facts supporting empirical testing, and the data obtained, are not necessarily confined only to the so-called objective aspects (for example, related to the ‘set’ as the organizational space of the relationship or to the words that are uttered in the dialogue) but may also concern the subjective and relational aspects: the variables, or at least some of them, of the ‘setting’ as the mental space of the relationship. This is naturally providing that a way is agreed upon to translate these variables into indicators - not fetichistically ‘numerical’ but above all qualitative -  which, as we have said, provide data that can be analysed with techniques that are just as reliable as the traditional statistics.
• Another reason for the resistance to empirical research into psychotherapy, and in particular into the relationship, derives from the assumption that the psychotherapy intervention is so complex that it does not leave space for deductions or predictions of a mechanistic and causal type. All attempts to analyse the linear relation between observable data would be misleading and reductionist.

Theories of complexity and the study of chaotic systems are closely connected. The more complex a system, the more unpredictably it evolves; the slightest variations of the initial conditions produce effects, even highly significant, with no deterministic connection to the conditions themselves: this development is called ‘chaotic’ precisely because of its unpredictability. These theories severely challenge the traditional concepts of science, laying the basis for a new epistemology centred on three elements:

• a ‘holistic’ approach to reality, i.e. global and non-reductionist;
• basic indeterminism: initial conditions create chain reactions with unpredictable outcomes;
• the circularity of causality: A influencing B, but being in turn influenced itself, is true for many of the relations within a system.

The latter condition was already known in classical science, which was well aware of the effects of feedback; but combined with the other two, it totally undermines the possibility of predicting that a particular action by A always and in any case leads to a certain effect on B. In certain cases, not even the prediction with a certain degree of probability, common in statistical analyses, can be sufficient or useful.
In actual fact, the impossibility of prediction deriving from the crisis of linear, deterministic thinking can lead, in the case of psychotherapies, to the decision-making paralysis feared by many, or to the application of criteria and methods typical of the artistic-literary disciplines, instigated by others. We are left to wonder then how these principles can be transmitted and taught profitably (in every sense …) in psychotherapy schools, whose legal recognition depends – and I say this as a member of the ministerial commission responsible – on the use of scientific parameters.
But the crisis of traditional linear science in no way excludes the search for rules within the chaos: to this end, in empirical research there must be the exact indication of the confines of the necessary reduction of complexity and the methodological and data analysis criteria appropriate to the complexity itself.
In many situations one can choose to avoid the  testing of hypotheses in ‘transversal’ research where control between groups and within groups, typical of classical experimental design, is impossible or unreliable, turning instead to longitudinal monitoring of phenomena evolving after specific interventions. And again, one can avoid generalizing on statistical-probabilistic grounds starting from a single research,  relying instead on testing hypotheses through the cumulative analysis of several pieces of research, as happens in the meta-analytic logic so common today4.
The essential problem of clinical research is actually the need to deal with the complexity of the object under study and its dynamicity over time.  The segmentation of complexity, for the purpose of effecting a strict control of the variables, can be at the expense of the external validity of the research, i.e. of its relation to the reality which it attempts to reflect. This problem was raised long ago by the ‘ecological’ approach and is particularly felt in research associated with the transformation of reality.
While purely cognitive research leaves sufficient margins for the breakdown and control of variables (and therefore has no problems going ahead according to the classical rules of experimentation), the effecting of change and the scientific study of it while it is underway must be placed in a perspective that is necessarily holistic.
In the applied sciences, which include clinical research, one may find coexisting a kind of research that poses specific questions each time and on which ‘universal’ knowledge is developed (for instance in the diagnostic field or about psychopathological theory), alongside a form of research centred on the intervention, which has to answer multiple, indivisible questions and try to understand ‘all and now’ of the problem being dealt with.
In response to the criticisms of the experimentalists who claimed that psychoanalysis could not be verified, Freud answered that psychoanalysis is not an academic research but a therapeutic action; its purpose is not to prove, but to bring about change. It is well known that these views have, often unjustly, been taken to extremes by psychoanalysts claiming that the clinically positive outcomes of analytical theories constitute in themselves conclusive proof of the validity of Freudian theories,  and that many equally radical critics have denied the psychoanalytical approach any scientific credibility in that it is not falsifiable according to the thinking of the experimental method.   
In Kurt Lewin’s well-known theory of ‘action research’, knowledge of the facts and acting on them are structurally connected: reality must be studied by changing it and monitoring the effects of the change. In action research, the testing of the hypothesis on a research object and the intervention designed to modify the object itself proceed in parallel in a circular process of planning - intervention - testing – further planning, a process in which the targets of the intervention are fully involved along with the researcher. Although Lewin’s action research originated in the psycho-social field, there are clear similarities between this strategy and clinical therapeutic practice.
On the epistemological plane, clinical psychology has no need to adjust itself to the level of  experimental laboratory methods in order to gain a scientific stature. When it is dealing with an object that is not suited to a reductive breakdown and with a purpose not merely of acquiring knowledge but of effecting a transformation, its methodology must be different and unique, following the example of action research.
In conclusion it must be repeated that the scientific research space in clinical psychology is defined neither by predictive power on a deterministic basis, nor – as has been said – by a purely quantitative logic.
The complexity of clinical and therapeutic work shows regularities that must be inferred by means of the systematic study of the indicators ‘representing’ the essential aspects. Admittedly, working with indicators and not with the actual reality is in itself reductionist: all smart researchers (and clinicians) are convinced that the variables quantified by a psychometric test do not tell the whole truth about a person’s psychic distress; that there is a very fine line dividing the normal discomfort caused by daily stress from the perception of ‘pathology’ on the part of the person himself or his social context (including those who have to diagnose the type and the degree of this presumed pathology); that the medical-nosographic model explains only a small part of the variability and the dynamicity innate in the therapeutic relationship; that empirical research into psychotherapy cannot merely consist of weighing up the quantities of what has been changed after the intervention.
But all sensible researchers (and clinicians) also know that from the profile revealed by  tests together with a clinical conversation, precise hypotheses can be drawn to understand the problems of the person being examined and to plan the treatment; that the pathology often – though not always – coincides with the degree of subjective suffering that psychotherapy is called upon to significantly alleviate; that this degree of significance is not only statistical but that which comes from the ‘meanings’ emerging  during the therapy; that to define these meanings and the global sense of the ‘treatment’ one can take into account not only the therapist’s intuition or the client’s subjective sense, but also precise parameters accepted by the scientific community.
It is one thing to seek an acceptable balance between the requirements of methodological rigour and the constraints of reducing complexity, and quite another to risk returning to the clouds of mystic-magic intuitionism that enveloped therapy for so long, and that still envelop it in some sectors connected to training. This is a risk of regression, albeit shrouded in the best epistemological intentions, that we cannot afford to take.

References

American Psychiatric Association, Commission on Psychotherapies (1982). Psychotherapy Research: Methodological and Efficacy Issues, Washington DC: Author.

Anderson, T., & Strupp, H.H. (1996).  The ecology of psychotherapy research. Journal of Consulting and Clinical Psychology, 64, 776-782.

Bergin, A.E., & Garfield, S.L. (Eds). (1994). Handbook of psychotherapy and behavior change: An empirical analysis. (4th ed.). New York: Wiley.

De Coro, A., & Andreassi, S. (2004). La ricerca empirica in psicoterapia. Roma: Carocci.

Denzin, N.K., & Lincoln, Y.S. (Eds). (2000). Handbook of qualitative research (2nd ed.). Thousand Oaks: Sage.

Di Nuovo, S.(1995). La metanalisi. Roma: Borla.

Di Nuovo, S. (2000). Strumenti qualitativi per la ricerca sulla psicoterapia: l’intervista sul cambiamento terapeutico di R. Elliott. Ricerca in Psicoterapia, 3 (2-3), 135-146.

Di Nuovo, S., Lo Verso, G., Giannone, F., & Di Blasi, M. (Eds.). (1998). Valutare le Psicoterapie. La Ricerca Italiana. Milano: FrancoAngeli.

Di Nuovo, S., & Lo Verso, G. (Eds.). (2005). Come funzionano le psicoterapie. Milano: FrancoAngeli.

Elliott, R. (Ed.). (1999). Qualitative psychotherapy research [Special Issue]. Psychotherapy Research, 9, 251-404.

Elliott, R. (2002). Hermeneutic Single Case Efficacy Design. Paper presented at 31st Annual Meeting of Society for Psychotherapy Research, Chicago, Ill, 2000. Psychotherapy Research, 12 (1) 1-22.

Fava, E., & Masserini, C. (2002). Efficacia delle psicoterapie nel servizio pubblico: Il contributo della ricerca valutativa alla clinica. Milano: FrancoAngeli.

Greenberg, L.S., & Pinsof, W.M. (Eds.). (1986). The psychotherapeutic process: A research handbook. New York: Guilford Press.

Hansen,  N. B., Lambert, M.J., & Forman, E. M. (2002). The psychotherapy dose – response effect and implications for treatment delivery services. Clinical Psychology, 9, 329 – 343.

Horvath, A. O., & Greenberg, L.S. (1994). The working alliance: Theory, research and practice. New York: Wiley.

Jaspers, K. (1913). Allgemeine Psychopathologie. Berlin: Springer Verlag (trad. it. Psicopatologia generale, Il Pensiero Scientifico, Roma, 1964).

Kazdin, A.E., & Weisz, J.R. (Eds.). (2003). Evidence-based psychotherapy for children and adolescents. New York: Guilford Press.

Lambert, M.J. (Ed.). (2003). Handbook of psychotherapy and behavior change: An empirical analysis (5th ed.). New York: Wiley.

Lingiardi, V. (2002). L’alleanza terapeutica. Milano: Raffaello Cortina.

Martin, D., Garske, J., & Davis, K. (2000).  Relation of therapeutic alliance with outcome and others variables: A metaanalytic review. Journal of Consulting and Clinical Psychology, 68, 438-450.

Nathan, P. E., & Gorman, J. M. (2002). A guide to treatments that works. (2nd ed.). New York: Oxford University Press.

Roth, A., Fonagy, P., Parry, G., Target, M., & Woods, R. (1996). What works for whom? A critical review of psychotherapy research. New York: Guilford.

Snyder, C.R., & Ingram, R.E. (Eds.). (2000). Handbook of psychological change: Psychotherapy processes and practice for the 21st century. New York: Wiley.

Speer, D. C. (1998). Mental health outcome evaluation. San Diego: Academic Press.

Vidotto, G., & Ciuffi, R. (2004). La verifica della psicoterapia. In A. Galeazzi & P. Meazzini (Eds.), Mente e comportamento (pp. 563-588). Firenze: Giunti.

Wampold, B.E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah: L. Erlbaum.

Notes

* Full Professor, Chair of Psychology, University of Catania. Top

1. This article is a reworking of the paper presented at the Congress of the Italian section of the Society for Psychotherapy Research (Reggio Calabria, 2006). Top

2. This type of evaluation is of interest not only to the psychotherapist wanting to test whether what s/he does has externally documentable effects, but also the public client (the local health service manager who has to quantify the efficacy of the psychology service performance) or the private client (the insurance company that funds psychotherapy: it happens in other countries and will perhaps happen here too). These aspects, often considered too pragmatic, are frequently the butt of jokes, but they would help to extend the accessibility of psychotherapy to users who at present are excluded for purely financial reasons. Top

3. I am referring to the interview developed by Robert Elliott in the context of his ‘hermeneutic’ approach: Elliott R. (2002) Hermeneutic Single Case Efficacy Design. Paper presented at 31st Annual Meeting of Society for Psychotherapy Research, Chicago, Ill. Published in Psychotherapy Research, 12, 1, 1-22 (2002) and translated in Di Nuovo S. (2000). Strumenti qualitativi per la ricerca sulla psychotherapy: l’intervista sul cambiamento terapeutico di R. Elliott, Research in Psychotherapy, 3 (2-3) pp. 135-146. Top

4. On the issue of cumulative analysis I am sorely tempted to mention meta-analysis which is so commonly talked about, often with no knowledge of its theoretical and methodological premises,  but am able to control myself.  I will simply point out that every statistical tool is like a hammer; one needs to know how to use it if one wants to avoid crushing one’s fingers. For those with the patience to find out more about the usefulness and limitations of meta-analytical techniques, see the book from some time ago  (Di Nuovo S., La metanalisi, Borla, Roma 1995). Top