Data Loading...
Recognising patterns: health systems research beyond ... Flipbook PDF
Health Systems Development Programme Recognising patterns: health systems research beyond controlled trials Authors: Bar
113 Views
59 Downloads
FLIP PDF 316.36KB
Health Systems Development Programme
Recognising patterns: health systems research beyond controlled trials Authors: Barbara McPake, Duane Blaauw & Rod Sheaff HSD/WP/10/06
Draft: not for citation. This paper has been submitted to the Milbank Quarterly and is subject to future revision.
The authors are members of the Health Systems Development Programme, which is funded by the UK Department of International Development. The UK Department of International Development (DFID) supports policies, programmes and projects to promote international development. DFID provides funds for this study as part of that objective but the views and options expressed are those of the author(s) alone.
Draft: not for citation
Abstract Experimental approaches such as randomised controlled trials have been successful in identifying and testing medical technologies. There is enthusiasm for use of similar approaches for health systems policies and programmes. This enthusiasm is a current manifestation of the attempt to model social science on physical science. It fails to appreciate the social nature of health system intervention where everything depends on how people interpret and implement policy, and users respond to new programmes and services. Health systems research needs to build more effectively on thinking from social science. This paper suggests a model through which to conceptualise the health systems research problem, identifies some methods that are consistent with studying its inherent complexity and shows, using a case study, how this approach can inform policy.
Introduction The health system has been an increasing focus of enquiry over the last decade. A search of PubMed reveals the term ‘health-system’ to have occurred more than once every 40 cited articles in which the term ‘health’ occurs in 2004, but only about once every 90 papers in 1994. The term ‘health systems research’ has been coined by some authors (Frenk et al., 1988), distinct from ‘health services research’ although it is not a recognised MeSH term in PubMed. The term has been increasingly used as debates over the international health sector reform movement intensified over the 1990s. For example, the first of a series of OECD reports in 1992 was titled ‘The Reform of Health Care’ (OECD, 1992). This report proposed a typology of health care systems and its follow-up in 1994 was entitled ‘The reform of health care systems’ (OECD, 1994). In 2000, the World Health Organisation’s annual report was titled: ‘Health systems: improving performance’. The increasing popularity of the term systems over services research, may be viewed as a recognition of the complex nature of the factors that determine health care access and quality characteristics. For example, Varkevisser, Mwaluko and Le Grand (2001) distinguish health systems from health services research as a broader scope of inquiry: ‘Health systems are an
1
Draft: not for citation
important component of the health system, but they are never regarded in isolation.’ (p282). In contrast, for Frenk et al. (1988) ‘health services research is restricted to that subset of health systems research concerned with the study of the system’s primary products’ (p65), implying an understanding of system as output-producing machine. For others, however, it is the research process that has systemic characteristics, implying that the researcher’s interactions with the subject of study have to be part of the research itself (Checkland, 1999). Von Bertalanffy is usually credited as the founder of general systems theory in particular and with ‘systems thinking’ as a general intellectual approach that can be applied in any discipline (Checkland, 1993). He suggests (1973) that the main departure of systems thinking from other kinds of thinking is the notion of the whole being greater than the sum of the parts, a departure from Descartes scientific method of reductionism, or dividing up the problem to be examined into separate parts. However general system theory as a whole has fallen out of vogue (although many of its ideas and concepts remain in use). This may be because its belief in the power of mathematical representations to capture complex systems relationships and generate accurate predictions is now viewed as naïve in the context of the theories of chaos and complexity that have emerged from the physical sciences and that have rapidly traversed the breadth of science (Lewin, 1993; Prigogine, 1996; Barabasi, 2002; Buchanan, 2002; Urry, 2003). This paper aims to explore what a health systems research project implies, where the term ‘systems’ is interpreted to signal a recognition of the complex (a term we use throughout this paper in the complexity theory sense) processes by which access and quality characteristics of health care are determined. We argue that the widely held belief that health systems research should seek to emulate epidemiological research by extending its use of randomised controlled trial methodologies, is misguided. That belief represents a current manifestation of a largely discredited school of thought that seeks to model social science on an inaccurate understanding of the methods of physical science. We illustrate the problems of a focus on measurement of effect, rather than on understanding mechanisms of effect, from a recent well known evaluation. We propose an alternative conception of ‘rigour’ in health systems research, that recognises
2
Draft: not for citation
both the value and limits to applicability of methods used in physical sciences by focusing on: (1) what can be learned from current approaches in the physical sciences; (2) the essential difference between physical and social science: the reflexive human agent; (3) a method that theorises and seeks evidence of mechanisms of effect in observational data. We review the techniques available to analyse and test for mechanisms of effect and other approaches that might be adapted to health systems research purposes. Finally, we provide a case study in which use of these approaches can be shown to provide understanding and information that we consider has critical importance for policy makers.
The randomised controlled trial
‘Don’t think, Jenner, but try.’ : John Hunter’s advice to Edward Jenner (Riedman, 1974)
When Jenner injected matter from a cowpox pustule into 8-year old James Phipp, he was practising the experimental method. While the method has undergone considerable development in the intervening 200 years, its essential logic has not changed: infer general laws of cause and effect by introducing the cause and inferring effect from measurement of subsequent change, relative to subjects in which the cause has not been introduced. The method can be credited with the discovery of the cause of smallpox, permitting the eradication of a disease which in Jenner’s day was responsible for one in every ten deaths in England. Through the subsequent work of Pasteur, Koch, Salk, Sabin, Fleming and others, it can be credited with the development of modern medicine in general and with the dramatic improvements in child survival, life expectancy and the disappearance of diseases largely forgotten across most of the world. Without the experimental method, polio sufferers would languish in iron lungs and leaches and ass’s milk would fill pharmacies and Gladstone bags. It is hardly surprising that those interested in curing social ills sought to emulate the experimental method in the hope of similar successes. The founders of modern social science conceived of the discovery of social laws: Comte’s law of social dynamics (Compte, 1854),
3
Draft: not for citation
Marx’s ‘laws of motion of modern society’, and law of capitalist accumulation (Marx, 1863), Durkheim’s law of social gravity (Durkheim, 1893) among other examples (Ritzer, 2003). Yet it is apparent that social science has failed in establishing such laws (Bloor, 1976) and that social problems have not proved responsive to solutions generated by (social) science, producing the often observed paradox of moon landings, internet and genome mapping concurrent with poverty, crime and warfare persisting at unabated levels. Condorcet and others have faulted social science for doing natural science badly lacking precision, consistency, cumulation, unifying theory, and objectivity (Gerring, 2001). Weber, writing in 1879 took issue with this perspective, arguing that:
‘the thesis that the ideal of science is the reduction of empirical reality to “laws,” is meaningless.’ (Weber translated by Heydebrand, 1994)
Modern expression of a similar critique has been voiced by Hayek (1942): ..in the hundred and twenty years or so, during which this ambition to imitate Science in its methods rather than its spirit has now dominated social studies, it has contributed scarcely anything to our understanding of social phenomena, not only does it continue to confuse and discredit the work of the social disciplines, but demands for further attempts in this direction are still presented to us as the latest revolutionary innovations which, if adopted, will secure rapid undreamed of progress.’ (p268)
More recent critiques have also attributed the failure of social science to offer easy solutions to social problems to the impossibility of the scientific ideal in social contexts. Social determination pathways, mediated by human agency, are inherently mutable and even liable to mutate in response to their very discovery. A different type of science altogether is required (Byrne, 1998; Gerring, 2001).
4
Draft: not for citation
The experimental approach in health systems research
Gonzalez-Block and Mills (2003) have defined health (policy and) systems research (HPSR) as: ‘knowledge generation to improve how societies organise themselves to achieve health goals, including how they plan, manage and finance activities to improve health, as well as the roles, perspectives and interests of different actors in this effort. HPSR contributes to sound, socially relevant and ethically acceptable guidelines for more effective, efficient and sustainable health policies and systems’ (2003: 1). This definition – focusing on how societies organise themselves – classifies health systems research as an essentially social science endeavour and contrasts with broader conceptions of the system to be researched as encompassing all dimensions influencing health and therefore requiring bio-medical and public health methods of enquiry (for example Priya, 2004). Experimental approaches are common in health systems research by either definition. Within the social sciences realm – that part of health systems research focused on how societies organise themselves to achieve health goals, experimental approaches are used extensively to test ‘interventions’, and this approach is held by some schools of thought to represent the gold standard of health systems research. For example, the Cochrane Collaboration1 and the Cochrane Reviewers’ Handbook2 “focus particularly on systematic reviews of randomised controlled trials (RCTs) because they are likely to provide more reliable information than other sources of evidence on the differential effects of alternative forms of healthcare” 3. The Handbook cites Kunz, Vist and Oxman (2003) on this point although their paper only compares randomised with non-randomised experimental approaches. Implicitly, all sources of evidence are experimental. Cochrane reviews are largely conducted to determine the appropriateness of clinical interventions, but they also encompass comparisons of alternatives ways that societies organise themselves to achieve health goals. For example, there is a review on ‘strategies for
1
http://www.cochrane.org (accessed 23rd May, 2005) http://www.cochrane.dk/cochrane/handbook/hbook.htm (accessed 23rd May, 2005) 3 http://www.cochrane.dk/cochrane/handbook/hbook.htm (accessed 23rd May, 2005) 2
5
Draft: not for citation
integrating primary health services in middle- and low-income countries’ which seeks to understand the effects of bringing together inputs, organisation, management and delivery of service functions (Briggs, Capdegelle and Garner, 2005); one on ‘capitation, salary, fee-forservice and mixed systems of payment: effects on the behaviour of primary care physicians’ (Gosden et al., 2005); and one on ‘healthcare financing systems for increasing the use of tobacco dependence treatment’ which considers whether or not paying smokers to accept cessation treatment increases uptake of that treatment (Kaper et al., 2005). This approach is variously described as ‘evidence-based policy’; ‘evidence-based management’; ‘evidence-based purchasing’ (i.e. commissioning of health services); and ‘evidence-based health care’ (MuirGray, 1997; Lohr, Eleazer and Mauskopf, 1998). The authors of these reviews are often disappointed both by the quality of evidence they find in relation to these questions, and by the extent that review of evidence leads to conclusive policy recommendations. For example, for Gosden et al. (2005) ‘It is noteworthy that so few studies met the inclusion criteria’ and ‘There is some evidence to suggest that the method of payment of primary care physicians affects their behaviour, but the findings’ generalisability is unknown’. For Briggs Capdegelle and Garner (2005) ‘Few studies of good quality, large and with rigorous study design have been carried out to investigate the evidence to support integration as a style of service delivery’, and ‘(p)olicy makers and planners considering integration could introduce strategies, using rigorous study design, to allow further evaluation and increase the base of studies from which to draw evidence’. Pawson and Tilley (1997) have highlighted the enthusiasm for similar approaches in areas of social research in the UK, and similar disappointment with the extent to which evidence lives up to this idea. It seems that the ideas voiced by Weber in 1879 and labelled ‘scientistic’ by Hayek in 1943 are alive and well at the beginning of the 21st century. As both would have predicted, attempts to follow this advice have failed to provide the guidance such approaches would seem to offer. A good case study of the difficulties can be found in the recent set of evaluations of WHO’s ‘integrated management of childhood illness’ (IMCI) strategy. IMCI has three components: (1) interventions that seek to improve the skills of
6
Draft: not for citation
health workers through training and supervision; (2) interventions that seek to improve the support systems for child health service delivery including drug availability and supervision systems; (3) community and household-based interventions that seek to improve the care of children at home (Bryce et al., 2004). On the whole the evaluations suggest that this set of interventions ‘works’. A cluster randomised evaluation of IMCI in Bangladesh suggested that it increased appropriateness of treatment and uptake of services (El Arifeen et al., 2004). Further evaluations in Tanzania suggested that it reduced the costs associated with childhood illness and reduced mortality (Armstrong-Schellenberg et al. 2004; Adam et al., 2005). This is hardly surprising. Under IMCI, a set of interventions of known biological efficacy is supported organisationally and financially. Some impact might be regarded as inevitable if it were not that a descriptive study in Haiti concluded that little impact had been achieved (Arrive, Perez and Pierre, 2004). The explanation in Haiti’s case is suggested to be limited economic and human resources – in other words the set of interventions was not in effect implemented (much). Exactly what is being tested by these studies? Recognition that experimental methods were not producing the knowledge required for policy and management prompted thoughtful contributions from members of the team involved in the multi-centre evaluation from which many of these reported results were outputs (Victora, Habicht and Bryce, 2004). These recognised the additional value (to randomised controlled trial evidence) of what were labelled ‘adequacy’ evaluations and ‘plausibility’ evaluations. Respectively these document ‘time trends in the expected direction, following introduction of an intervention’ and ‘attempt to document impact and to rule out alternative explanations by including a comparison group – historical, geographical, or internal – and by addressing confounding variables’ (Victora, Habicht and Bryce., 2004). An evaluation that sought all three types of evidence was developed in which it was recognised that ‘IMCI is a complex strategy, incorporating numerous interventions that affect child health through a variety of pathways’ (Bryce et al., 2004). The ultimate focus of the evaluation was to measure an overall net effect of the evaluation, together with an associated cost.
7
Draft: not for citation
It seems to us that what policy makers and programme managers might want to learn from evaluations of a programme such as this – where efficacy of intervention is established and what is at stake is the social organisation of implementation – is not the overall net effect. Given the complexity of the strategy and the multiplicity of pathways through which effects are produced, any overall level of net effect is almost certainly specific to a particular context. It is not clear that even reporting net effects in a range of contexts gives managers or policy makers helpful guidance about the likely level of effect in any specific context. Multiple pathways are likely to work synergistically in ways that multiply or dampen effects to extreme degrees. The actual effect achieved in an untried context might be any unpredictable value, as is apparent from the Haitian example. A study of IMCI interventions in Mexico (Lopez-Flores, Barquera-Cervera and Kageyama-Escobar, 2004) found that in one locality (Acaponeta), 3.3% of children received all interventions (which were intended for universal coverage) whereas in another (Santiago Ixcuintla), this percentage was 1.6%. In both cases these percentages seem low. Managers and policy makers seek strategies to produce high levels of effect with their intervention. To do this they need to better understand the constraints to IMCI reaching its target population, what kinds of children the programme works well for, which ones less well, and most importantly why. However for between 1/30 and 1/50 children, the initiative seems to have been very successful. It would be equally interesting to those trying to design a high performing programme to understand more about the characteristics of these children and whether there are particular parts of the programme that are associated with this success. The Mexican evaluation however focused on how variation in application of interventions affected childhood stunting, rather than on what determined the difference in application of interventions in the first place. Realistic evaluation It would seem that any social programme interacts with different people in different circumstances differently. According to Pawson and Tilley (1997): ‘Programs work by introducing new ideas and/or resources into an existing set of social relationships. A crucial task of evaluation is to include (via hypothesis
8
Draft: not for citation
making and research design) investigation of the extent to which these preexisting structures ‘enable’ or ‘disable’ the intended mechanism of change.’ (p70) Pawson and Tilley label their approach ‘realistic evaluation’, recognising parallels in the search for research frameworks that will help those seeking to design social programmes design them better, and wider epistemological debates in social science. Realism (Lakatos, 1970; Hesse, 1974; Bhaskar, 1975, Sayer, 2000) responds critically to both epistemological poles of positivism and relativism. It is neither law seeking (nomothetical) nor interested only in the documentation of the unique (idiographic), emphasising the mechanics of explanation (Keat and Urry, 1982; Pawson and Tilley, 1997, Fox, 1997). ‘Social programs are merely a special case of social change, which always comes in a spiral of new ideas and transforming social conditions. … Policy makers try to engineer episodes of social change, and the success (or otherwise) of these initiatives depends upon the extent to which the program theory has been able to predict and control this interpretative spiral of ideas and social conditions.’ (Pawson and Tilley, 1997, p56) ‘Causal outcomes follow from mechanisms acting in contexts’ (Pawson and Tilley, 1997, p58) Taking the IMCI programme as an example, this realist approach would imply a different understanding of the purpose of an evaluation – focusing on the mechanisms of change rather than the net effects. The three components of IMCI outlined above have separate target populations. The first component (improving the skills of health workers) targets front-line health workers and seeks to change their behaviour when they attend to a child. The second (improving health system supports) targets the workforce engaged in providing services that support front-line care. This workforce might include front-line health workers for example in their role in managing drug stocks at facility level, but also includes workers in medical stores or wherever drugs are processed before they reach facilities, district health teams, or others responsible for the primary supervision role and managers at successive levels responsible for
9
Draft: not for citation
the supervision of tasks such as these. The third component targets communities and individual community members and seeks to change community and household behaviours. Each of these populations contain members with quite different susceptibilities to changing their behaviour following an intervention. The front-line health workers population contains younger and older members who have undergone different initial training, have longer and shorter histories of engagement with programmes designed to improve their performance respectively, have different roles in their own households that may make their job and job performance more or less important to them, have different peer groups with different attitudes to the social change intended. It contains more junior and more senior cadres of staff who have been oriented differently in their initial training, have different levels of authority, social and economic status, and different peer groups. Besides these examples liable to affect the success a programme seeking to change behaviour will meet, the population contains health workers of different ethnic groups and gender, who work with health systems users from the same or different ethnic groups and gender, who live further or nearer their work place, who are committed to health work or are looking for alternative employment, who have optimistic or pessimistic, entrepreneurial or conservative, miserly or generous, personalities. The same divergence within the target population groups will apply to those of the other two components. Ultimately, the programme’s success will depend not primarily on its success in any one of these components but in all three components acting together. Success in measures of outcome such as child deaths may depend for example on appropriate recognition of symptoms requiring the attention of frontline health staff in households and communities, response to that recognition by households in effective accessing of health services; the correct use of empirically valid algorithms proceeding from symptom identification to diagnosis and prescription by front-line health workers, the availability of those materials (such as drugs) necessary to fulfil the prescription, made possible by those responsible for the functioning logistics of the system and the adherence to health workers’ advice within households. Multiple interactions between households, communities, front-line health workers and successive layers of management are necessary to achieve this result. Within these interactions, effective
10
Draft: not for citation
programme components will operate. For example a training programme delivered by a trainer who conveys inspiration as well as information may result in an effective interaction between a mother of a sick child and front-line health worker where information alone may not have sufficed, in those specific circumstances. Generally, an effective programme can achieve positive outcomes in a range of circumstances (each component with a proportion of the target populations; overall in a proportion of scenarios in which the target populations interact), but not in all. Documenting with whom and in what combinations of circumstances a programme achieves success, and seeking explanatory mechanisms as to how, would seem to take precedence over measuring an overall effect, or comparing that effect with a different context, with a different mix of susceptibilities among target populations, in which no programme was implemented. It is practically impossible that the full set of relevant factors can be controlled for. The description of the actions of confounding variables in a range of circumstances substitutes for attempts to control them, in realist methodology. Rather, multiple comparisons of the programme in operation in different circumstances are likely to be informative because they will expose differences in what confounding variables are present, and why and how they operate. Comparison with sites in which no intervention is occurring is likely to be uninformative about these differences. It is equally impractical to document every possibly relevant characteristic of a programme as implemented in multiple contexts. A selected set of features to document and examine has to be identified. Realistic evaluation proposes the process of theory development as capable of identifying the selected set: ‘(T)heories must be framed in terms of propositions about how mechanisms are fired in contexts to produce outcomes.’ (Pawson and Tilley, 1997, p84-85) How might this have been done in the context of the IMCI evaluation? The model used – and unusually, explicated - by the evaluators posited direct causal links between programme inputs (such as measures intended to improve organisation at health facilities) to (intermediate) outputs such as reduced missed opportunities for vaccination, to ultimate outcomes such as reduced mortality (World Health Organisation, 1999; Bryce et al., 2004). Realistic evaluation
11
Draft: not for citation
would focus further on what mechanism(s) might underlie any links between inputs, outputs and outcomes in order to help focus the enquiry, and most importantly on the circumstances, or contexts in which such mechanisms will be triggered. For example, the IMCI evaluation model proposes that as measures to improve service quality and organisation at health facilities, review and revision of the distribution of IMCI related tasks might, among other measures, be associated with reduced waiting times and increased patient satisfaction. This link could be theorised by postulating the following alternative (though not all mutually exclusive) explanatory mechanisms, and many possible others.
(i)
Rationalisation of tasks will reduce the load of some front-line health workers who will have more time for interaction with service users whose satisfaction will improve.
(ii)
Rationalisation of tasks will reduce the load of some front-line health workers who will experience more job satisfaction. This will improve the quality of their interaction with service users whose satisfaction will improve.
(iii)
Rationalisation of tasks will reduce the load of some front-line health workers. This will reduce waiting time.
(iv)
Rationalisation of tasks will concentrate appropriate tasks on those health workers most capable of delivering effectively. This will increase user satisfaction.
(v)
Rationalisation of tasks will concentrate appropriate tasks on those health workers most capable of delivering time-efficiently. This will reduce waiting times and patient satisfaction.
(vi)
The process of task rationalisation will give health workers greater say in planning their own work. This will result in higher levels of motivation, improved quality of interaction with users and higher user satisfaction.
(vii)
The process of task rationalisation will give health workers greater say in planning their own work. As understanding of how to achieve more efficient use of time is concentrated among health workers, this will reduce waiting lists.
12
Draft: not for citation
Any of these mechanisms might be expected to be triggered in one context but not in another. For example, it might be easier to shift the job satisfaction and motivation levels of younger health workers or those who have been a shorter time in a particular post and hence the programme might be expected to achieve greater effect in a newly constructed or designated health facility than an older, well established one. Or any of these mechanisms might be considered more plausible in contexts in which other components of the health system are working well, health workers are generally better supported and have the other tools necessary to do their jobs than in contexts where the reverse is the case. By identifying possible mechanisms, the evaluators focus on those variables of interest for documentation and exploration – the time taken for each consultation (under mechanism (v), this would be expected to fall whereas under mechanism (i) it will increase); the expressed motivation/job-satisfaction levels of staff; the pattern of task distribution before and after rationalisation. Hence the researcher’s task is not to describe, document and explore everything and anything, but to test for evidence of the operation of specific mechanisms of effect that have been pre-specified and that can be compared across sites of intervention, and to compare across contexts. Unlike in a relativist account, where all observations however unique are of equal interest, because generalisability is not of concern, the ultimate goal of a realistic evaluation is to recognise patterns or regularities: common conjunctions of variables that because common, can be considered relevant to a wide range of settings. Pattern recognition The complexity literature provides another critique of experimentalism in social science that has been very effectively martialed by Byrne (1998). Byrne defines complex situations as ones in which the impact of a change in policy cannot be predicted with certainty, but in which what will happen will be drawn from a set of alternatives that is larger than one, but less than too many to cope with. That is, the range of predicted outcomes is under-determined; it lies within a constrained range whose limits are determinate. This suggests that if health systems are characterised by complexity, the role of
13
Draft: not for citation
health systems research will be to map out the set of alternatives (or the ‘phase space’ in the language of complexity theorists) and to pay particular attention to common solutions within that available set. We classify such attempts as attempts at pattern recognition, but define these to include intuitive diagnosis of patterns in qualitative data describing processes and outcomes following health system and policy change, as well as formal quantitative data analysis techniques, such as cluster analysis, that test for quantifiable patterns within complex data sets. The concept of pattern recognition has been developed within the scientific literature on complexity. Hayek (1967) may have been the first to use this term in the context of social science and its inherent complexity. Pattern recognition is defined by Hayek as an inductive approach, albeit one which can underpin deductive investigation: ‘We shall first have to invent the pattern before we can discover its presence in the phenomena – or before we shall be able to test its applicability to what we observe’ (Hayek, 1967: p24) The process of pattern recognition has been unpacked to consist of sensing, segmentation and grouping, feature extraction, classification and post processing. The statistical approach, that deals with vectors under assumptions about their distribution has been contrasted with the syntactical approach, that deals with non-vectorial data and can be used for example to recognise genes in a DNA sequence, or a face in a photo. A distinguishing characteristic of these approaches is that they are not theory driven in the sense that data are tested for their support for pre-hypothesised relationships, but rather the data themselves are used to generate patterns – that is they apply inductive logic. After Hayek however, once the pattern is ‘invented’ deductive approaches may be applied, comparing patterns with those predicted by existing theory, or testing further sets of data from the one in which the pattern was identified for the presence or not of the pattern. Quantitative and qualitative data can equally contribute to pattern recognition and pattern seeking. It would be mistaken to counterpose the inductive capacity of qualitative approaches and the deductive capacity of quantitative approaches. Rather, quantitative procedures contribute to the inductive stage, and where verbal description represents those
14
Draft: not for citation
patterns, qualitative methods may deductively test for patterns recognised elsewhere or predicted by theory. Examples of data analysis strategies across a range of social science endeavours that might be classified as ‘pattern recognition’ according to that definition include Lichtman’s ‘algorithm of recognition’ for predicting the next President of the United States which parameterizes qualitative information using a kernel method (Lichtman and Keilis-Borok, 1981; Lichtman, 2000); Marsh’s approach to interpretation of survey data in which ‘theory serves as a basis of the organisation of the model but the data itself is also used to generate ideas in an exploratory way which are then taken back for further review’ (Byrne, 1998: p67); various spatial analysis techniques such as point pattern analysis, network analysis and visualisation of data associated with geographic information systems and approaches that use cohort data to identify patterns in individual trajectories. For example, Calistri and Galbraith (1999) use a combination of cluster and discriminant analysis to identify patterns in the paths of wage change across countries. Such approaches can recognise evolving patterns. These processes would seem then to have multiple disciplinary roots. Theories from different social science disciplines can potentially provide inspiration in the creative process of inducing a pattern and as points of comparison for recognised patterns. Some theory itself constitutes recognised pattern in the sense proposed here. For example, the theories of transaction cost economics are comparable to those of theoretical anthropology in the sense that they make (falsifiable) claims of incompatibility between particular institutional forms. Roemer’s Law is a ‘law’ of cause and effect –that supply of hospital beds creates its own demand. While this assumes to be constant, more than we consider tenable in any social context, it can still provide the inspiration for building theory about mechanisms of cause and effect that are investigable at both inductive and deductive stages of enquiry. What are the social processes that might provide a chain linking supply and demand? In what contexts are those social processes triggered? Do data provide evidence of the operation of those mechanisms? Methodological techniques that support the pattern recognising and seeking processes will draw on a range of disciplines including and going beyond the social sciences, and spanning
15
Draft: not for citation
quantitative and qualitative elements, and there is apparent scope for considerable development and adaptation. Use of explanatory mechanisms and context in health systems research These perspectives have been little applied in the field of health systems research. A preliminary framework which we think captures the essence of the health systems scenario is described by Figure 1.
Dynamic responses: Informal structures eg, community groups networks of friends Informal behaviours eg. (offering or demanding) under-the-counter payment professionalism Informal relationships eg. clientelism political alliance
De-facto system: Services as experienced by people For example: access; quality
De-jure system: Organisational structures Intended incentives Management procedures Training courses
Figure 1 A conceptual model for health systems research
Dynamic responses are the lynchpin between formal arrangements (the de-jure system) and the de-facto system as experienced by people who use it. The interplay of these components is then the focus of enquiry. Whilst formal arrangements offer the entry point for intervention, the ability of people to access services responsive to their needs, crucially depends on how those implementing the de-jure system respond. The development of programmes and policy that improve outcomes for users requires direct engagement with context and with how formal organizational structures, intended incentives and management procedures interact with
16
Draft: not for citation
informal structures, behaviours and relationships. These dynamic responses may have effects considered positive or negative through circumventing formal arrangements that might be judged adverse by some criterion, or subverting intentions of formal arrangements that by the same criterion, are helpful. The model characterises the elements of dynamic responses as being ‘informal’ behaviours, meaning to describe those behaviours that are not governed by the rules of the formal system which take place in the de-jure box. The bi-directional arrows between the two left-hand boxes represent the equally dynamic interaction between formal and informal behaviours. For example, informally constituted community groups may form to resist a proposal to close a hospital, made by a formally constituted committee. This may have effects both in the de-jure system – changes in the constitution of the committee, in the proposal emerging from the committee, ultimately in whether or not the hospital is officially designated to close, but also in further dynamic responses – how committee members form alliances to support or oppose this closure and others, alliances of community group members and hospital staff, off-record media briefings; all of which produce the emergent de-facto system. The critical feature of the model is that it draws no direct link between the de-jure and defacto systems. The only health system that can be experienced by its users is the one that emerges from multiple human interactions. Hence the de-facto system can be considered the system’s ‘emergent properties’, the product of complex, essentially social interactions (Byrne, 1998). The task of health systems research can be viewed as the analysis of the emergence of the de-facto system from the relationship between the de-jure system and the dynamic responses of human agency. Like realistic evaluation, the model emphasises the mechanisms of effect between a social programme (component of the de-jure system), and its impact (in the defacto system), and recognises these mechanisms as being social: involving the interaction of human beings. The model seeks to capture several of the preceding ideas. As realistic evaluation, it focuses on the explanatory mechanisms that relate a social programme (a change to the de-jure system) and its results. The model provides a starting point for theorising relevant features of
17
Draft: not for citation
context, found in both de facto and dynamic response boxes and in environmental features outside those that influence the multiple relationships inside and between them, and with the defacto box. This suggests that the relevance of the types of components identified in each of the boxes in the initial state requires consideration in theory building. In emphasising dynamism in its centre, it seeks both to capture the notion of the potential of reflexive human agency to drive the outcomes that emerge in the de-facto system, and the characteristics of complexity that result from multiple inter-relationships between large numbers of variables. It is intended to suggest that the resulting system may be ‘dissipative’, producing evolutionary characteristics, capable of disturbing itself without external stimulus, and far from equilibrium (Byrne, 1998). The types of patterns that might be identifiable from the study of this system will not be simple laws, or cause and effect mechanisms. Rather they are likely to contain multiple variables in a structured set of relationships between micro and macro phenomena, and ‘algebraic’ (Hayek, 1967) in the sense that measured values may not be available for some variables whose actions can only be inferred from the measurement of others. To demonstrate the ideas put forward in this paper, we provide one case study of a research project in Uganda that we think has been able to provide useful advice for policy on the basis of this kind of approach. The study is by no means unique, and we use an example from our own work only because we are best able to explore it from this perspective. There are many other examples of good health systems research. What is lacking is the clarification of principles that distinguish it from the bad, that has a consensus among health systems researchers. Informal markets and formal health financing policy in Uganda In the early 1990s, Uganda was in the early stages of recovery from two decades of civil war and misrule, reflected in the collapse of the institutions of the state. Stability seemed to have returned under the government of Yoweri Museveni and the National Resistance Movement, and with it a host of international agencies offering advice as to how to rebuild the health sector. The policy of introducing user fees for health services was widely advocated at the time, for health systems whose symptoms appeared similar to those of Uganda – low utilisation of public
18
Draft: not for citation
services, poor public perception of those services, frequent stock-outs of drugs and other supplies, and a poorly performing health workforce. Those advocating user fees did so on the basis of a model, widely disseminated by a World Bank Publication (World Bank, 1987), which had in the intervening years been amended and reinterpreted by the Bamako Initiative (UNICEF, 1990), and research evidence that highlighted the importance of quality (largely interpreted as drug availability) in the mechanisms by which user fees produced change in the de-facto health system (McPake, 2002). By the early 1990s, those advocating user fees largely did so on the basis that the disincentive to utilisation inherent in the charging of a price would be outweighed by the positive effect of improved quality brought about by the use of revenues from charges primarily to finance increased drug supplies (McPake, 2002). One might identify the following assumptions within the ‘programme theories’ underpinning the policy: -
drug supply was inadequate
-
drug supply was the critical constraint to health service utilisation
-
those administering user fees would use the revenues generated to alleviate that constraint
However, in 1992, the Ugandan government produced a document entitled ‘The Three Year Plan: financing for health for all’ (Republic of Uganda, 1992). Features of the environment identified in this document might have called these theories into question: The Danish Red Cross donates drugs to health units on the basis of reported volume of out-patients and in-patients. While the objective of this procedure is to introduce equity in the distribution of drugs it provided loop-holes for the manipulation of records, as those units that show a high work load receive a high quota of drugs. (Republic of Uganda: 6) The medical staff have also responded to a decline in demand and a fall in real wages by spending less time at their work and by engaging in other activities to
19
Draft: not for citation
support themselves and their families, including illicit private practice in government premises (Republic of Uganda: 10) [A]lthough the government does not charge user fees, the staff do, and that too without any authority. What is recommended here is not the introduction of user charges but the formalization of already existing … charges. (Republic of Uganda 27-8) These observations speak of an ‘informal economy’ in which health workers were engaged, and which could only be understood through an understanding of the recent history of the country and the collapse of public institutions. Salaries in public services had been decimated in value and unpaid altogether for periods. A default privatisation of the public sector, including the public health sector had occurred. International agencies, such as the Danish Red Cross, channelled their inputs through this privatised system without always recognising it as such, with consequences for the treatment of those inputs. User fees were introduced in the Ugandan system in a gradual manner during the first half of the 1990s. A health systems research project (Asiimwe et al., 1997; McPake et al, 1999) sought to evaluate their impact on the system. The conceptual model underpinning this study (see McPake et al., 1999), recognised that the de-facto health system’s access and quality characteristics were dependent on the interactions that occurred among health workers, and between health workers and users in the informal economy, in turn dependent on contextual characteristics such as the sociodemographic, psychological and professional characteristics of health workers, and other environmental factors such as the local economy, employment opportunities and the range of competing health facilities. A change to the de-jure health system, or formal health policy, for example the introduction of user fees would operate via a series of dynamic responses in the informal economy, on quality and access. Theorising the mechanisms by which the policy might change the social reality (the existing equilibrium in the informal market in health services), suggested among others, the following potential mechanisms:
20
Draft: not for citation
-
user fees would be viewed by staff as competitive to their own informal charges. Their introduction would be resisted, passively or actively and would fail;
-
user fees would be successfully introduced but their impact on health workers’ incomes from public sector work would lead them to increase their private sector work and their absenteeism from their public sector jobs;
-
user fees would be successfully introduced but their impact on health workers’ incomes would lead them to sell the health facility’s drug supply privately instead. Drug stock-out problems would increase;
-
The distribution of revenues from user fees would replicate that of existing informal charges. They would replace the latter with no impact on quality of care;
-
The informal economy would not intervene in the universal model of user fee implementation and the net effect would be to improve drug availability, quality of care as perceived by the population and utilisation of public health services.
This by no means comprehensive list, suggested that research should focus on measuring and analysing the following characteristics, among others, of the post user-fee situation. -
The relative importance of informal charges, re-selling drugs and private sector work for health workers incomes;
-
The relative size of formal and informal charges - the extent to which formal charges would be likely to impact on the incentive environment;
-
The adequacy of the initial drug supply for the current level of demand - whether or not there was a real shortage of drugs to be bridged with user fee revenue;
-
how elements of the informal economy entered the utility function of health workers – informal income generating strategies would be more easily replaced if they were perceived as unpleasant, demeaning, risky, or otherwise negatively by health workers
-
how revenues from formal charges were allocated It is apparent that the exploration of this list of characteristics would not be the preserve of a
single discipline. Economics perspectives were useful in analysing incentive environments associated with the informal economy, and how, and the extent to which, these would be
21
Draft: not for citation
reshaped by the introduction of formal fees. Organisational theory was useful in considering the autonomy of health workers in their workplace, and the features of the workplace that contributed to, or controlled that. A sociological appreciation of the roles of the competing professions in the health workplace contributed to an understanding of the processes that might differentiate the distribution of formal and informal fee incomes. And a psycho-social approach to understanding the perceptions of, and attitudes to informal income generating strategies on the part of health workers was helpful. A number of evaluations of user fee policy conducted concurrently (Kamugisha, 1993; Wamai, 1992; Bizimana and Liesener, 1993; Jitta et al., 1996), were largely predicated on a linear model that looked for the effects of the policy in utilisation levels and quality characteristics of public health services. This produced a mixed set of conclusions, suggesting that the relationship between fees and outcomes varied across settings and times – in other words that context mattered. However it produced very little basis for understanding why the effects of policy were different in the different settings studied and therefore little advice as to how to modify policy to offer the greatest probability of more positive outcomes. In contrast, this research suggested that user fees were rarely the dominant factor in the processes that were producing utilisation and quality outcomes. The user fee policy model had failed to engage with the underlying problem system of the Ugandan situation. In most of the settings studied, drugs supplied by the Danish Red Cross were in fact sufficient to deal with the actual patient load, and this was also true of the resource levels in terms of staff numbers and other supplies. The problem experienced by patients in the de-facto health system was that those resources de-jure provided were not available to them. Most drugs were used by health workers in their private businesses, or sold on, a 76% leakage rate in the median of the 10 facilities studied. Health workers worked on average for approximately 36% of a full working week in the median facility. The result was that patients encountered health facilities with inadequate drug and staffing levels for their needs and voted with their feet by seeking services in the private sector.
22
Draft: not for citation
The dominant factors in this outcome were judged to be the collapse in trust in public institutions on the part of both patients and staff, the recent history of abnegation of responsibility for public services that had filtered from the top of the governance structure of Uganda through to its roots in the frontline workforce, the struggle for basic survival that had confronted all Ugandan citizens over a prolonged period, public health workers not excepted, and the consequent atomisation of individuals’ economic strategies, detached from dysfunctional social structures. The path of recovery from this situation could be conceptualised as the phase space in which policy sought to bring about social change. The research was conducted in two districts that appeared to constitute two areas of that phase space. In the first, a district in the South West of the country, ethnically aligned with the new government, recovery from the nadir of trust, governance and social cohesion appeared more rapid. In general, the extent of retreat from more damaging economic strategies, and outcomes in terms of the utilisation rates of public health facilities were greater there. In the second district in the South East, progress appeared slower. At the level of the individual facility, specific factors were found to be associated with the extent and impact of different economic strategies. For example, a less prosperous catchment population seemed to make informal charging and drug pilfering more viable strategies than operating a private clinic. Individual health worker characteristics were also shown to be associated with choice of strategy. For example, charging informal fees was shown to be chosen by less well established health workers who had lower salaries and allowances and lower household expenditures. It was also role associated, more likely to be used by midwives than by medical assistants despite their similar status in a facility. In summary, there appeared to be individual and group level causes (in the probabilistic sense) that given a set of contextual factors - the set of possibilities embodied in the phase space, could trigger the resort to particular economic strategies. Finally, the choice of economic strategy, and its implications for quality and access, were not points in a linear process. Strategies have implications on a number of levels. By induction,
23
Draft: not for citation
in the facilities studied, a number of variables seemed to be found together: High informal charges and incomes earned from them; higher numbers of hours worked and high outpatient attendances (pattern 1) or high rates of drug leakage, low drug availability, low numbers of hours worked and low utilisation (pattern 2). These associations between quantities were reflected in correlations of marginal statistical significance. Inducing further on the basis of qualitative data, under pattern 1, health worker morale was higher, and community members were more likely to make positive comments related to staff attitudes and behaviour towards them than under pattern 2. This pattern might result from a feedback loop whereby health workers' main incomes were earned from informal charging, their incentives were to encourage high utilisation by treating well those patients who attended and paid the expected charge levels, ensuring that there was some drug availability and that informal charge levels were set so as to maximise their incomes (neither too high nor too low). Pattern 2 might be explained by a feedback loop in which drug leakage left little income to be earned from within the facility and little interest in attending the facility on the part of the population. Once health workers’ livelihoods were established external to the public facility, especially in private clinics and drug shops, health workers would be more likely to channel any drugs supplied to the facility in that direction, rather than to use the supply as the basis for income generation strategies internal to the facility. Formal user charges would be largely irrelevant to the business of providing health care in that locality. Complexity theory suggests that paths predicated on such feedback loops can be dislodged, sometimes by quite small external shocks. Where learning and reflexive human agents further complicate the set of relationships, shocks can also be internally generated. Ten years after this research, reports from Uganda suggest that pathways towards more positive outcomes have been travelled (Kirunga-Tashobya, Ssengooba and Oliveira-Cruz, 2006). The relative roles in supporting progress along this path, of plausibly more appropriate policy measures since introduced (including the removal of user charges and a host of measures to better support health workers’ livelihood security), and a changing phase space (of increased security, improved governance and developing social institutions) cannot be discerned on the
24
Draft: not for citation
basis of existing research. This is partly a problem of the irrelevance of the dominant paradigm of health research to understanding processes of change in health systems, and partly due to the inherent difficulty of any project that sought to understand the process. As the phase space in Uganda looks to be shifting again, there are new opportunities for understanding. Conclusions The call to randomise, control and experiment in health systems research, in imitation of epidemiological study design, is misguided. It fails to identify the essentially social nature of the health system, or to recognise the systemic implications of that nature as complex and not reducible to the simple unilinear relationships that epidemiological methodologies are designed to isolate and test. Understandings associated with complexity theory point to different types of approach in physical, natural and social science. Realistic evaluation focuses on understanding mechanisms of effect rather than quantifying the size of effect. Identifying what is feasible requires a pattern recognition approach. Theories and investigative techniques for the project of pattern recognition are under-developed, but a few promising approaches are beginning to emerge and realist methodology provides important components of what is needed. Experiment may play a role in that project, but its contribution will be to allow the social and organisational processes described above to be observed in real time, so as to enlarge the range of possibilities from which data can be collected, rather than to provide a method that controls for context. The interactions between human actions in health activities, human intentions in health related social programmes, and the contexts in which they occur are the proper focus for health systems research. Identifying contrasting theoretical and methodological approaches to pattern recognition more clearly identifies how specific disciplines can make both discrete and complementary contributions to a potentially cumulative understanding of the key issues in securing health system development. Even while methods to support this approach to health systems research are poorly developed, research that has taken this approach and made what it could of available methods has proved capable of generating important insights into the processes by which the de-facto health system emerges that have relevance for a policy focused audience. These
25
Draft: not for citation
insights cannot be packaged into solutions for perceived health system ills. Rather, they provide those attempting to shape the de-facto health system with a basis for continual reactive responsiveness to its emerging conditions.
26
Draft: not for citation
References Adam, T., Manzi, F., Schellenberg, J.A., Mgalula, L., de Savigny, D., Evans, D.B.(2005) Does the integrated management of childhood illness cost more than routine care? Results from the United Republic of Tanzania, Bulletin of the World Health Organisation, 83 (5): 369-77 Armstrong-Schellenberg, J.R., Adam T., Mshinda, H., Masanja, H., Kabadi, G., Mukasa, O., John, T., Charles, S., Nathan, R., Wilczynska, K., Mgalula, L., Mbuya, C., Mswia, R., Manzi, F., de Savigny, D., Schellenberg, D., and Victora, C., (2004) Effectiveness and cost of facility-based integrated management of childhood illness (IMCI) in Tanzania, The Lancet, 5; 364(9445): 158394. Arrive, E., Perez, F., Pierre, L.M. (2004) The integrated management of childhood illness: Haiti’s example, Santé, 14 (3): 137-42 (French). Asiimwe, D., McPake, B., Mwesigye, F., Ofoumbi, M., Ortenblad, L., Streefland, P., and Turinde, A. (1997) The private-sector activities of public-sector health workers in Uganda, in Bennett, S., McPake, B., and Mills, A. Private Providers in Developing Countries: Serving the Public Interest?, London, Zed Books. Atkinson, S., Medeiros, R., Oliveira, P. & de Almeida, RD. (2000) Going down to the local: incorporating social organisation and political culture into assessments of decentralised health care. Social Science and Medicine, 51, 619-636. Atkinson, S. (2002) Political cultures, health systems and health policy. Social Science and Medicine, 55(1), 113-124. Atkinson, S. & Haran, D. (2004) Back to basics: does decentralization improve health system performance? Evidence from Ceará in north-east Brazil. Bulletin of the WHO, 82(11), 822-827. Barabasi, A.L. (2002) Linked: the new science of networks, Perseus Books Group Bertalanffy, L. Von (1973) General System Theory, Harmondsworth: Penguin Books. Bhaskar, R. (1975) A realist theory of science, Brighton: Harvester Bizimana, D. and Liesener, L. (1993) Financing of health services in Kapchorwa, Mbale and Pallisa, Kampala: Care International Uganda Blaauw, D., Gilson, L., Modiba, P., Erasmus, E., Khumalo, G. & Schneider, H. Governmental relationships and HIV/AIDS service delivery. Final project report. 2003. Johannesburg, Centre for Health Policy. Blaauw, D. (2004) State transformation, health sector reform and HIV/AIDS policy implementation in post-apartheid South Africa: the tensions of public sector decentralisation [in French]. In: Afflictions. L'Afrique du Sud, de l'apartheid au sida (D. Fassin, Ed.), p. 111-138, Karthala, Paris. Bloor, D. (1976) Knowledge and Social Imagery. London: Routledge. Bossert, T. & Beauvais, J. (2002) Decentralization of health systems in Ghana, Zambia, Uganda and the Philippines: A comparative analysis of decision space. Health Policy and Planning, 17(1), 14-31. Briggs, CJ., Capdegelle, P., Garner, P. Strategies for integrating primary health services in middle- and low-income countries: effects on performance, costs and patient outcomes, The
27
Draft: not for citation
Cochrane Library, Issue 2, 2005, http://www.cochrane.org/cochrane/revabstr/ab00318.htm (accessed 23rd May 2005) Bryce, J., Victora, C.G., Habicht, J-P., Vaughan, J.P., and Black, R.E. (2004) The multi-country evaluation of the integrated management of childhood illness strategy: lessons for the evaluation of public health interventions, American Journal of Public Health, 94, 3: 406-14 Buchanan, M. (2002 ) Small World: uncovering nature’s hidden networks, Phoenix, London and New York Byrne, D. (1998) Complexity and the Social Sciences, Routledge, London and New York Calistri, A.D and Galbraith, J.K. (1999) Inter-industry wage structures: new evidence from the OECD, UTIP working paper no. 13, University of Texas at Austin, http://utip.gov.utexas.edu Checkland, P. (1999) Systems thinking, systems practice, John Wiley and sons. Checkland, P. and Scholes, J. (1999) Soft systems methodology in action, John Wiley and sons. Collins, C. (1996) Decentralization. In: Health policy and systems development: An agenda for research (K. Janovosky, Ed.), p. 161-178, WHO, Geneva. Comte, A. (1854) On the Positivistic Approach to Society, in The Positive Philosophy of Auguste Comte (translator: Martineau, H.), New York: Calvin Blanchard, Vol 2. Duda, R.O. (2000) Pattern classification: Pattern classification Part I, John Wiley, New York. Durkheim, E. (1893) The division of labour in society, Free Press, 1984. El Arifeen, S., Blum, L.S., Hoque, D.M., Chowdhury, E.K., Khan, R., Black, R.E., Victora, C.G., Bryce, J. (2004) Integrated Management of Childhood Illness (IMCI) in Bangladesh: early findings from a cluster-randomised study, The Lancet, 364 (9445): 1595-602 Fox, G. (1997) Reason and realities in the methodologies of economics, Edward Elgar. Frenk, J., Bobadilla, J.L., Sepulveda, J., Rosenthal, J., Ruelas, E. (1988) A conceptual model for public health research, Bulletin of the Pan American Health Organisation, 22, 1, 60-71. Gerring, J. (2001) Social Science Methodology: A criterial framework, Cambridge University Press Gonzalez-Block, M. and Mills, A. (2003) Assessing capacity for health policy and systems research in low and middle income countries, Health Research Policy Systems 2003; 1: 1, http://www.pubmedcentral.org/articlerender.fcgi?artid=151554 Gosden, T., Forland, F., Kristiansen, I.S., Sutton, M., Leese, B., Guiffrida, A., Sergison, M., and Pedersen, L. Capitation, fee-for-service and mixed systems of payment: effects on the behaviour of primary care physicians, The Cochrane Library, Issue 2, 2005, http://www.cochrane.org/cochrane/revabstr/ab002215.htm (accessed 23rd May 2005) Hayek, F.A. von (1942) Scientism and the study of society: Part 1, Economica, new series, volume9, 35, 1942: 267-91 Hayek, F.A. von (1967) Studies in Philosophy, Politics and Economics, London: Routledge and Kegan Paul
28
Draft: not for citation
Heydebrand, W. (1994) Max Weber, Sociological Writings, Continuum, http://www.marxists.org/reference/subject/philosophy/works/ge/weber.htm (accessed 12th April, 2006) . Jitta, J., Kawesa-Kisutu, D., Tereka, S., Babikwa, D. and Magezi, A. (1996) Evaluation of health financing reforms in Uganda: a document review of user fees: Child Health and Development Centre, Makerere University, Kampala. Kamugisha, J. (1993) Health cost sharing in Kabarole District, Kampala: Tropical Health Consult. Kaper, J. Wgaena, E.J., Severens, J.L., Van Schayck, C.P. Healthcare financing systems for increasing the use of tobacco dependence treatment, The Cochrane Library, Issue 2, 2005, http://www.cochrane.org/cochrane/revabstr/ab004305.htm (accessed 23rd May 2005) Keat, R. and Urry, J. (1982) Social theory as science, International Library of Society. Kirunga-Tashobya, C., Sssengooba, F., and Oliveira-Cruz, V. (2006) Health Systems Reforms in Uganda: Processes and Outputs, Kampala and London: IPH,Makerere University and London School of Hygiene and Tropical Medicine. Kunz, R., Vist, G., and Oxman, A.D. (2003) Randomisation to protect against selection bias in healthcare trials, In: The Cochrane Library, Issue 1, 2003. Oxford: Update Software Lewin, R. (1993) Complexity: Life at the edge of chaos, London Phoenix, 1993 Lichtman, A.J. and Keilis-Borok, V.I. (1981) Pattern recognition applied to presidential elections in the United States, 1860-1980: Role of integral social, economic, and political traints, Proceedings of the National Academy of Science, USA, 78, 11, 7230-4 Lichtman, A.J. (2000) The Keys to the White House, Lexington Books, Lanham MD and Oxford (originally Madison Books, 1996) Lohr, K.N., Eleazer, K., and Mauskopf, J. (1998) Health policy issues and applications for evidence-based medicine and clinical practice guidelines, Health Policy, 46: 1-19 Lopez-Flores, F., Barquera-Cervera, S., and Kageyama-Escobar, ML (2004) Evaluación de un antencida integrada contra la desnutrición infantile en areas urbanas de Nayarit, México, Salud Pública de México, 46, 2, 158-63 McPake, B. (2002) The globalisation of health sector reform policies: is ‘lesson drawing’ part of the process? In Lee, K., Buse, K., and Fustukian S. (eds) Health Policy in a Globalising World, Cambridge University Press McPake, B., Asiimwe, D., Mwesigye, F., Turinde, A., Ofumbi, M., Ortenblad, L. and Streefland, P. (1999) Survival strategies of public health workers in Uganda: implications for quality and accessibility of care, Social Science and Medicine, 49, 849-865 Marx, K. (1863) Capital: A critique of political economy, Pluto Press, 2003 Muir Gray, J.A. (1997) Evidence-based healthcare, Edinburgh: Churchill Livingstone OECD (1992) The reform of health care: a comparative analysis of seven OECD countries, Health Policy Studies, No. 2, OECD, Paris. OECD (1994) The reform of health care systems: a review of seventeen OECD countries, Health Policy Studies No. 5, OECD, Paris. 29
Draft: not for citation
Pawson, R., and Tilley, N. Realistic Evaluation, Sage Publications, 1997 PHRplus. Decentralisation and health system reform. Issue in brief. No 1, September, 1-11. 2002. Bethesda, Abt Associates. Prigogine, I. (1996) The end of certainty: time, chaos and the new laws of nature, Free Press, 1997 Ritu Priya, Health Policy and Systems Research: Informing the HIV and AIDS Research Agenda, Presentation for the Global Health Research Forum 8, Mexico City, November 2004, http://www.globalforumhealth.org/forum8/forum8-cdrom/OralPresentations/Priya%20R%20F8615.doc (accessed 23rd May 2005) Republic of Uganda (1992) The three year health plan: Financing health for all, Entebbe: Ministry of Health Planning Unit. Riedman, S.R. (1974) The story of vaccination, Folkeston, Bailey and Swinfen Ritzer, George (2003) Classical Sociological Theory, 4th edition, New York, McGraw Hill Rondinelli, DR. & Nellis, JR. (1986) Assessing decentralization policies in developing countries: a case for cautious optimism. Development Policy Review, 4, 3-23. Shawe-Taylor, J. and Cristianini, N. (2004) Kernel methods for pattern analysis, Cambridge University Press UNICEF (1990) Revitalising primary health care/maternal and child health: the Bamako Initiative, progress report, Executive Board Document E/ICEF/1990/L.3, 20 February, New York Urry, J. (2003) Global complexity, Cambridge, Polity Press Varkevisser, C. M. Mwaluko, G.M.P., Le Grand, A. (2001) Research in action: the training approach of the Joint Health Systems Research Project for the Southern African Region, Health Policy and Planning, 16, 3: 281-91 Victora, C., Habicht, J-P., Bryce, J. (2004) Evidence-based public health: moving beyond randomized trials, American Journal of Public Health, 94, 3: 400-5 Wamai, G. (1992) Community financing in Uganda: Kasangati Health Centre Cost Recovery Programme, Kampala, UNICEF. World Health Organisation Department of Child and Adolescent Health and Development, Health Systems and Community Health (1999) Modeling IMCI impact on mortality: planning and implementing IMCI interventions, February, World Health Organisation, www.who.int/imcimce/Publications/im_p1.pdf, accessed 8th August 2005 World Bank (1987) Financing health services in developing countries: an agenda for reform, A World Bank Policy Study, Washington DC.
30