Intended audience
This brief provides information and ideas for analysts of policy and arguments, policy-makers, decision-makers and advocacy.
Topic: How to study the effectiveness of arguments
Effects of arguments are easiest to observe within the context of where they are used, and also considering the interactions between different arguments. The analysis of argumentation can be conducted through observational investigation of case-studies where different arguments are used or by observation of chains of arguments. Argument effectiveness can be addressed also at a more general level, by considering the logic of the argument and making causal inferences. These two approaches may be broadly referred to as observed effectiveness and potential effectiveness, respectively.
Observed effectiveness can be analysed through empirical work; it requires observations of relations, between e.g. events, decision-making levels or stakeholders. Effects take place when the arguments in one event, at one level or by one stakeholder produce a change in behaviour or in arguments used in another event, level or by other stakeholders. Analysing policy processes, we observe effectiveness across stages of the policy cycle, across governance levels and across actor types.
Potential effectiveness can be studied empirically or by applying methods of logic, as used by philosophy, semantics and linguistics. Potential effectiveness can be analysed as an exercise of logical inference, which is used, for example, in legal analyses and assessments of new laws or other argumentation referring to standards, such as biodiversity policy. Empirically, potential effectiveness can be sought from people’s evaluations of how different arguments might contribute to policy outcomes. The analysis of potential effectiveness relies on recording the sequence of inference by the analyst or informants’ views on effectiveness.
Arguments at different stages of policy cycle
BESAFE took the policy process to consist of five phases: 1) Agenda setting, 2) Policy formulation, 3) Policy adoption, 4) Policy implementation 5) Policy evaluation (Figure 1).
The BESAFE analyses showed that different types of arguments were used at different stages of the policy cycle. Judging by the frequency of different types of arguments at each stage, it appears that the intrinsic value of nature type arguments dominate at the early stages of the process, while different types of benefit arguments are used in the middle of the process and legal obligation arguments are most frequent at the implementation and evaluation stages.
Potential effectiveness
Analysing potential effectiveness, the BESAFE case studies found that:
- Logic or a sound scientific basis is not a sufficient condition for an argument to be effective; duty or benefit related arguments support their effectiveness.
- Linking several benefits and using general language can increase the effectiveness of an argument.
- Tailoring the argument to the audience increases its effectiveness.
Observed effectiveness
Most of the case studies detected signals of effectiveness through empirical analysis. The effectiveness of arguments could be observed in their:
- Persistence: in several cases similar arguments (e.g. sustainable development or inherent value arguments) persisted or evolved over long periods, sometimes also to new stages in the policy cycle;
- Accumulation: sometimes the arguments were used increasingly and with growing emphasis and importance in the process;
- Diffusion: in several cases particular arguments diffused to broad use, various policy arenas and audiences; in particular, several cases witnessed livelihood arguments being taken up by advocates of conservation;
- Level-crossing: arguments were taken up by stakeholders at lower or higher governance levels than where they originated;
- Replacing: several case studies show situations where new arguments replaced old ways of expressing concern or placing importance on some phenomenon; for example, ecosystem service arguments replaced arguments focusing on rarity.
Usefulness
Understanding how arguments generate effects is important for analysts of policy processes and for anyone who wants to improve the effectiveness of their arguments.
Transferability
The approach on how to study effectiveness is general and widely transferable. The empirical lessons represent generalisations across all the BESAFE case studies from varying contexts and may thus also be considered of wider applicability (see Deliverable D2.3 for an overview of the case studies).
Lessons learned
- Persistence of biodiversity conservation arguments through different policy processes and against the counter-arguments is a pre-condition for their effectiveness.
- Diffusion and accumulation of arguments originally used by a limited group of actors signals effectiveness.
- Operational planning channels scientific arguments to new audiences, and increases effectiveness.
- Livelihood arguments coupled with biodiversity related arguments can increase the effectiveness of conservation, by allowing more dialogue between different types of actors.
- Livelihood arguments originating at the local level can be effective at higher levels, by widening the scope of debate and engaging different actors.
- Appealing to high level legal arguments is effective in concrete tight argumentation at the local level, particularly in deadlock situations.
- Deliberation across levels improves the effectiveness of arguments.
- Broad concepts and complex reasoning can easily be replaced by arguments that refer to concrete benefits or duties.
- Arguments that people personally relate to often replace scientific and inherent value arguments that have to do with biodiversity in isolation from society.
- Context of the argument is crucial. Arguments change with time and move across governance levels, depending on context and different strategies.
- The effectiveness of arguments depends also on the positive or negative framing of the argument.
Looking for more information on effective arguments for biodiversity?
For more BESAFE results, including separate briefs focusing on other case studies and various aspects of argumentation, see http://www.besafe.pensoft.net
Final report synthesizing the analysis on effectiveness in case studies. BESAFE. Deliverable D2.3. Available at:
This brief is a result of research carried out under the BESAFE project. This brief was written by Eeva Primmer, Pekka Jokinen and Malgorzata Blicharska.
The BESAFE project is an interdisciplinary research project funded under the European Community’s Seventh Framework Programme, contract number: 282743.