 |
|
|
|
|
|
|
|
|
|
| search | feedback | |
|
|
| thriving organisations |
|
| governance | |
| planning | |
| evaluation | |
| quality improvement |
|
| evidence based practice |
|
| risk management | |
| writing
policy & organisational manuals |
|
| questionnaires | |
| data analysis | |
| social capital | |
| facilitation | |
| training workshops | |
| ideas & articles | |
| about us | |
|
|
| disclaimer | copyright | |
Management |
|
Finding cause and effect in human services
It is not a simple matter to establish that one event causes another*. We can observe one event following another. We can observe one event occurring with another. We cannot observe one event causing another. We can only infer one event causes another.
Key ingredients to inferring one event (X) causes another (Y) include:
Y must vary with X
The inference that X causes Y must make sense:X must take place before Y
Y must be capable of change
The explanation that X causes Y must be theoretically plausible. For example is there a mechanism that makes sense in this context?We need to be able to make comparisons between X causing Y and other situations in order to test the logic of the model for example:
X causes Y in context A but not in context B
‘not X’ not causing Y.
2X causing 2Y
There are ways we are prone to errors in inferring one thing causes another:
We see Y vary with X but don’t notice that Y and X are both varying with Z (and Z is the cause of both X and Y).
We don’t have an adequate theory for our explanation.
We don’t make comparisons between different situations to explore the logic of our cause and effect model.
Some of the errors in our inferences arise because of the nature of our being human**:
We jump to conclusions - we see more than is there; we are good at creating order out of random data.
We see what we expect to see - especially when the evidence is ambiguous.
We see what we want to see - our motivations affect our inferences.
We believe what we are told - what we have been told in the past affects our inferences in the present.
We see what we think others would like us to see - social support affects our inferences.
In human services additional difficulties in inferring that X causes Y arise because:
Individuals are unique with unique life stories within unique families within unique neighbourhoods and unique communities.
Mechanisms for causes and effect often differ from one context to another.
The causes and effects are difficult to define and describe.
The causes and effects may themselves be connected with the values of clients, staff and other stakeholders.
The causes and effects hard to measure.
There are multiple causes and multiple effects (outside the control of service providers).
The intended effects may not be know in advance (for example in community development).
There are conflicting demands between providing services and researching clients and service processes to show cause and effect relationships.
There are conflicting demands between client confidentiality and the data collection necessary for research.
While the services are provided short term the effects we want may be long term.
There is often a lack of research on which to build.
So if we want to show that X causes Y in human services what can we do?
Firstly we need to ask ourselves:
How certain do we need to be?
When do we need to know?
What resources have we got to find out?
If we need to be very certain, have time on our side and have resources - we probably need to undertake some rigorous research.
If we need to be moderately certain, don’t have time on our side and have minimal resources then what we can do is:***
Approach thinking about inferences in two ways:
Look for evidence both for and against the inference that X causes Y.
Look for evidence both for and against any important alternative inferences eg U causes Y or W causes Y.
We can look for the evidence by:
1. Ask observers - we can ask clients and staff about what causes what.
2. Ask wether what happened is consistent with what was intended to happen? For example if a program teachers parenting skills to improve parenting and parenting has improved we can ask whether the parenting skills that were taught are being used.
3. Check the timing of causes and effects makes sense. Are the causes before the effects?
4. Check whether the amount of ‘cause’ is related to the amount of ‘effect’. Does, for example more thorough teaching of parenting skills improve parenting more than less thorough teaching of the skills. Does an 8 week program work as well as the condensed four week version?
5. Identify what we think are the underlying causal mechanisms and the contexts in which they operate and check for patterns consistent and inconsistent with these mechanisms and the side effects of these mechanisms. For example we can develop a service model of the outcomes hierarchy and processes and think through the implications and see if we notice the patterns we would expect to see.
6. Make comparisons with other groups, for example a ‘comparison’ group or a ‘control’ group. To be most effective this step would build on the point above about developing the model of the underlying causal mechanisms and the contexts in which they operate.
7. Use statistical analysis to separate out the effects of extraneous variables on the inferred cause and effect relationship.
8. Make statistical models of how we expect the cause and effect relationships to work and see if they provide a plausible explanation for the data we observe (about X, Y and other related variables).
9. Design a rigorous research model to do the above with more rigour than we would be able to do in the normal course of our work.
In summary:
Think things through.
Understand the limitations to our thinking.
Be aware of the difficulties of inferring causation in human services.
Look for evidence for and against the inference that X causes Y in a given
context.
Look for evidence for and against any important alternative inference eg
Z causes Y in the same context.
Do the best looking and thinking that can be done with the resources available
in the time available.
Three things worth reading:
* David A de Vaus “Causation and the Logic of Research Design” Chapter 3 in Research Design in social Research Sage 2001 (19pp)
** Thomas Gilovich How we know what isn’t so. The fallibility of human reasons in everyday life. The Free Press 1993. (216pp)
*** E. Jane Davidson "Dealing with the causation issue" Chapter 5 in Evaluation Methodology Basics The Nuts and Bolts of Sound Evaluation, Sage 2005 (18pp)