A live reconstruction at the future conference of the German green party.

How can argument maps be used for debate moderation? In this post three short “live reconstruction” case studies from 2007, 2011 and 2014 are presented.

In a “live reconstruction” the arguments of the statements made during a live discussion are reconstructed and visualised in real-time during the event. The most impressive “live reconstruction” technology so far comes from ARG-tech at the University of Dundee. ARG-tech used a huge touchscreen, the “AnalysisWall”, and a large team of analysts to transcribe, segment and analyse an episode of the BBC radio show “The Moral Maze” in 2012.

The technology we have used so far has been much simpler. But the following use cases show that the problems are not so much of technical but of a methodological nature. Even if working perfectly, it is an open question whether the technology is just a nice gimmick or if it produces real benefits for the participants of live discussions. The following cases show that such benefits are difficult to achieve if analysts, moderators, clients and participants have conflicting aims and expectations.

What is a “live reconstruction”?

Our original concept of “live reconstruction” can be characterized in the following way:

• participants argue about a concrete controversial thesis or question.
• three to four analysts work on the reconstruction of the debate with Argunet. Because Argunet is a client-server software they can work on the same map from different computers,
  • inserting important theses,
  • summarizing the arguments in short descriptions and giving them telling titles (there is not enough time for detailed logical reconstruction),
  • sketching the relations between arguments and theses,
  • layouting the argument graph.
• the emerging argument map is projected live on a big screen. Arguments ‘magically’ pop up one by one.
• the moderator uses the resulting map to structure the ongoing debate and to summarize results, e.g. she
  • asks participants whether the reconstruction is adequate,
  • directs attention to arguments that were neglected or to implicit assumptions that were revealed,
  • shows what the central claims are and what dialectical role they play in different arguments,
• participants can refer to specific arguments in the debate in order to
  • express their agreement or disagreement,
  • add new rebuttals or support, or
  • clarify their previously made statements.
• the argument map can be used to document the discussion results.

First case study: Long night of the sciences 2007

2007 was the first time we gave the “live reconstruction” a try at the “Lange Nacht der Wissenschaft” (long night of the sciences) a popular event in Berlin, where universities and other science institutions open up for the general public.

Overall this has been a very positive experience:

• the argument map was a useful moderation tool that made it easier for the moderator to keep an overview,
• participants used the map in their statements to refer to arguments,
• participants collaborated on making the map better and more representative instead of pushing and clinging to their points of view,
• this led to structured, fair and non-confrontative discussion.

At the time we were quite enthusiastic about the prospects of the method. Time has shown that this success depended crucially on the non-typical “ideal” conditions of this use case:

• it was a pro bono experiment, there were no costs to consider,
• the moderator was part of our team and an expert in argumentation theory,
• guests were expecting an experiment and came to learn about the method of argument analysis and reconstruction,
• the topic of the debate was secondary and was decided upon by the audience at the beginning of the experiment.

Second case study: Future Congress of the German Green Party 2011 (Zukunftskongress der Grünen)

Our team at work at the future conference.

In 2011 the German Green Party organized a “Future Congress” as an open and innovative forum for discussions about their party platform. The congress was organized in 14 workshops, some of which were moderated by the party leaders Claudia Roth and Cem Özdemir. We were hired to conduct live reconstructions of two workshops, one on “social justice” and one on a “common European defence policy”. Both workshops were moderated by Cem Özdemir. Our team consisted of one coordinator, two “reconstruction analysts” and one co-moderator. The boundary conditions were challenging, to say the least:

• a huge audience and speakers list,
• limited time,
• the first topic was very broad and there was no clear focus of the discussion,
• many statements from the audience were very emotional and vague,
• the moderator was not an expert in argumentation theory, though we had the opportunity to introduce him to our method beforehand,
• the moderator was a party leader, and had to meet many expectations that were not easily reconcilable with using argument maps to structure the debate and keep the participants focused on specific points,
• because of this and the time restrictions our co-moderator could only play a small and very limited role.

Not all conditions were unfavorable. The Green Party agreed to pay for a team of four experts which is quite extraordinary, considering these were moderation expenses alone. Even with such a large team the reconstruction was very exhausting, stressful and difficult for the team; and it was hard to assess whether the live projection brought any benefits for the discussion, because of the lack of interaction. Though the feedback we got was very positive, we were somewhat disillusioned.

Third case study: Annual convention of the German ethics council 2014

In 2014 we were hired by the German Ethics Council for its annual convention and gave the concept another chance: In a collaboration with Ralph Groetker from explorat we tried to develop ideas to overcome the weaknesses of our concept. We changed the process to make it financially more feasible, decrease the workload for the analysts and increase the benefits of the resulting maps. This process was tested in three parallel workshops on “reproductive ethics”. The changes were the following:

• Due to financial limitations we could only use one analyst per workshop.
• This made it impossible to create a new reconstruction from scratch live; instead we prepared maps beforehand that were based on an online survey of the invited guests and additional research.
• These maps were given to the members of the Ethics Council and their staff beforehand. In several rounds they were revised and enhanced.
• Every guest got a conference binder that included these argument maps; additionally the maps were presented on big posters at the conference.
• These maps were than augmented live with simple comments that described important claims and additional arguments of the participants, without reconstructing the precise relations of these arguments.
• Directly after the workshops the new arguments were added with post-its to the posters.
• Additionally the maps were used to document the results.
• To simplify the process even more we agreed to use the mindmapping software XMind instead of Argunet. Though this made it possible to use icons as mnemonic devices there were some limitations we think speak against using such generic software in the future (for example, the graph has to be hierarchic and argument groups have to be sub-trees).

We think that these changes proved largely sucessful. It made a huge difference for the reconstruction process that the analysts could build on already existing maps. Even though we reduced the team from three analysts to one single analyst, the reconstruction of the new arguments was feasible. The feedback was once again very positive.

Even so, this was a mixed experience, because in two of three workshops the moderators didn’t want to get distracted or restricted by the maps and refused to use live projections. In these workshops the live reconstruction had no influence on the discussion (though it was used for documentation purposes). But maybe this didn’t make such a huge difference, because in all workshops the participants had read our arguments beforehand and we got the impression that some of them used the maps to formulate their statements.

Argument map of the session on “mitochondrial transfer” at the annual convention of the German ethics council. Circular boxes were added live during the event. Rectangular boxes were reconstructed beforehand.

Conclusion:

Live reconstructions can have real benefits if the boundary conditions are just right. But in real-world deployment it is difficult to find and establish these conditions. The most important ones are:

• Preparation: The live reconstruction can only augment an already existing map, so it has to be possible to prepare such a map beforehand.
• Preparation: Clients and moderators have to be included in the reconstruction process as early as possible.
• Moderation: The moderator has to be willing and able to use the map; realistically, this will only be the case if the moderator has been trained in argument reconstruction and analysis.
• Topic: The questions to-be-discussed must be concrete and controversial.

In many cases moderators, clients and participants have conflicting expectations and aims that make it impossible to create these conditions. Under such circumstances further revisions to the process are needed to make argument maps useful for debate moderation.

The post Going live: Using argument maps for debate moderation appeared first on Argunet.

	[1] Tracy Bowell and Gary Kemp: Critical Thinking: A Concise Guide. Routledge: London 2010 (3rd edition).
	[2] John Butterworth and Geoff Thwaites: Thinking Skills. Cambridge University Press: Cambridge 2005.
	[3] Alec Fisher: Critical Thinking: An Introduction. Cambridge University Press: Cambridge 2011 (2nd edition).
	[4] Alec Fisher: The Logic of Real Arguments. Cambridge University Press: Cambridge 2004 (2nd edition).
	[5] Joe Y. F. Lau: An Introduction to Critical Thinking and Creativity: Think More, Think Better. Wiley: Hoboken 2011.
	[6] Anne Thomson: Critical Reasoning: A Practical Introduction. Routledge: London 2009 (3rd edition).

In a nutshell, my favourite is Bowell/Kemp [1], closely followed by Lau [5].

Note, however, that I review the books from a particular perspective and that I don’t necessarily consider each book in its entirety. So, first of all, I focus on those parts that pertain to argument reconstruction only. Secondly, and more importantly, I assess the textbooks against a couple of key beliefs, which I shall state upfront:

Explicitness

Argument analysis makes explicit the informal judgments involved in natural language reasoning and argumentation. In particular, a good reconstruction uncovers all the hidden assumptions an argument relies on (to make them amenable to critique) and shows, in the same time, which premisses are actually unneeded.

Interpretation

To reconstruct an argument means to interpret a text. Reconstruction is guided by the principle of charity (make the reconstructed argument as strong as possible!). Consequently, one cannot separate sharply reconstruction and evaluation.

Inference

Argument reconstruction involves the assessment of deductive and non-deductive inferences and hence builds on (basic) formal logic and a theory of non-deductive inference schemes.

These statements are part of the ideal that guides our own reconstructions (see, e.g., here or here). Accordingly, this post (as well as the reviews to come) explores to which extent a textbook teaches you to reconstruct arguments in a similarly detailed and Argunet-compatible way.

Overview

As stated above, the textbook by Bowell/Kemp [1] provides, from my view, the best instruction to argument reconstruction. Lau’s [5], which covers more theoretical material but is a bit poor in examples, is also an excellent book. The following table summarizes my evaluation.

I will detail this assessment below.

Illustrative reconstructions

Let’s have a look at some example reconstructions to get a flavour of the books’ different approaches.

[1] provides the following reconstruction (pp. 136-138):

P1) Tuna catches have been decreasing significantly for the past nine years.

P2) If Tuna catches have been decreasing significantly for the past nine years, then, if the Tuna industry is not regulated more stringently, the Tuna population will vanish.

---

C1) If the Tuna industry is not regulated more stringently, the Tuna population will vanish.

P3) If the Tuna population vanishes, then the Tuna industry will collapse altogether.

---

C2) If the Tuna industry is not regulated more stringently, it will collapse altogether.

Comment: This is impeccable. The inferences in the argument are deductively valid. (That’s formally obvious, but [1] explains it in a non-formal way.) All the reasoning’s assumptions are hence made explicit. In addition, [1] nicely shows that the reconstruction is the result of a hermeneutic process involving earlier and preliminary versions of the reconstruction.

[2] analyses a complex argument about traffic rules as follows (p. 29):

R1 In a number of countries cars drive on the left.

R2 This can result in accidents involving drivers and pedestrians from other countries who are used to traffic being on the right.

(therefore)

IC: R3 Roads would be safer if in all countries the rule was the same.

R4 Countries where cars keep to the left are in a very small minority.

Therefore

C Those countries should change to the right.

Comment: The inferences in this reconstruction are not valid. Critical implicit assumptions of the argument are, moreover, not made explicit. While [2] discusses the concept of logical validity and provides a list of inference schemes, arguments are not systematically reconstructed in a deductively valid (or inductively strong) way.

[3] reconstructs an argument against genetic engineering as (p. 41):

R1<Most prospective parents would prefer to have sons>. So C1 [if people can choose the sex of their child, it is likely that there will eventually be more males than females in the population] and R2<This could produce serious social problems>; therefore C2 [we should prohibit the use of techniques which enable people to choose the sex of their children].

Comment: Basically, the ‘reconstruction’ is just a markup of the original text. No premisses are added, no text is deleted, no sentences are logically streamlined. As a consequence, the reconstructed arguments are not necessarily deductively valid or inductively strong, and implicit premisses are not uncovered by means of the reconstruction.

[4] analyses a (sub-)argument advanced by US Defence Secretary Caspar Weinberger (in a letter to NATO partner in 1982) as follows (p. 65):

1. We must take the steps necessary to match the Soviet Union’s greatly improved nuclear capability.

and

2. The Soviet Union has a capability for a survivable response.

therefore

C We [the US] must have a capability for a survivable and endurable response.

Comment: The premisses and the conclusion are direct quotes from the reconstructed letter. Also, the reconstructed argument is neither deductively valid nor inductively strong, and major assumptions (e.g., a principle of practical reasoning or a statement about the side-effects of having a capability for a survivable response) remain implicit.

[5] unpacks and visualizes the structure of arguments as inference diagrams, e.g. (p. 96):

Comment: This reconstruction is absolutely fine. The inferences are deductively valid. And it’s straightforward to translate the argument map into the standard form used in Argunet, which is also introduced in [5].

A typical argument analysis in [6] reads (pp. 28-29):

The policemen gives three reasons which, taken together, are intended to support the conclusion that the burglar must have left by the fire escape:

Reason 1: This person is not in the building now

supports the claim that the burglar must have left the building. But

Reason 2: (the person) has not been seen leaving, and

Reason 3: there are guards posted at each entrance

do not entitle us to conclude that the burglar must have left by the fire escape unless we assume that Reason 3 supports an intermediary conclusion to the effect that no one could leave undetected except by the fire escape.

Comment: [6] doesn’t present reconstructed arguments in a standard form at all. The reasons stated above are direct quotes from the original text (e.g., no logical streamlining, no substantial reformulations so as to repair the inferences). Accordingly, the argument is not reconstructed in a way such that its inferences are deductively valid (or inductively strong). As a result, tacit assumptions of the reasoning are not systematically uncovered.

Detailed Assessment

Realistic examples

One of the main challenges in argument reconstruction consists in handling argumentatively opaque and logically confused texts. Simplistic cases of natural language reasoning may have a role to play in formal logic instructions, but they’re clearly insufficient if you want to learn how to deal with real argumentation. So it’s pivotal that a textbook makes use of realistic, sufficiently complex examples.

Concerning realistic examples, all books reviewed do fairly well: they discuss many and sufficiently long reasonings.

[1], [2] and [6], in particular, contain sections or chapters devoted to extra-long examples (entire pieces rather than single paragraphs). [5], however, contains somewhat fewer examples than the other books and doesn’t illustrate analysis techniques equally comprehensively.

Examples are clearly most prominent in [4]. In terms of didactic concept, [4] differs significantly from all the other textbooks: It starts with a brief introduction of basic methods and then unfolds, in eight chapters, detailed illustrative analyses of complex arguments. In this regard, [4] is certainly a valuable supplement to the other books.

Exercises and answers

Argument reconstruction is an art and involves as much knowing-how as knowing-that. You learn it by doing. All textbooks pay tribute to this fact in providing exercises and questions (as well as answers, except [4]). That makes them suitable for self-study.

Identifying arguments and their conclusions

Reconstruction starts with identifying arguments and their conclusions. All books devote a chapter or section to the question how to determine whether a text contains an argument at all and, if so, what the argument is supposed to show. They explain the basic technique of using conclusion- and premiss-indicators.

Identifying implicit premisses and the reconstruction of enthymemes

It’s here where substantial differences between the textbooks emerge. As spelled out above, a key function of argument reconstruction is to uncover hidden assumptions. A good textbook tells you how to find such implicit premisses.

Now, while [1], [2] and [5] discuss this issue in depth, [3], [4] and [6] touch upon this question only briefly and, more importantly, don’t provide a method for uncovering implicit assumptions. Quite the opposite, [3], [4] and [6] instruct the reader to identify hidden premisses on a purely intuitive basis only. But this is no advance whatsoever to our everyday practice. By resorting to informal judgement, these books fail to acknowledge that we often err as to the implicit assumptions of an argument.

[1], [2] and [5], in contrast, rightly explain that, when reconstructing an argument, implicit assumptions are added so as to make the inferences deductively valid (or inductively strong). [1] and [5] in particular discuss this pivotal problem of argument reconstruction in detail, and [1] introduces, in this context, the helpful concept of “connecting premisses” (pp. 132-133).

Employing deductive inference schemes to reconstruct arguments

Many (if not all) arguments can and should be reconstructed as deductively valid. To assess an argument’s validity is an integral part of its reconstruction: Inconclusive inferences indicate that hidden premisses have not been uncovered yet. The textbooks reviewed differ substantially in terms of the space devoted to deductive reconstruction.

[1] and [5] possess exclusive chapters on deductive validity. Both list and illustrate the most important deductive inference schemes. [1] especially motivates the logical study of inference by the fact that argument reconstruction is essentially an interpretative activity, guided by the principle of charity. Overall, logic is a bit more prominent in [5] than in [1].

[2] and [3] introduce briefly the concepts of validity and soundness but don’t come up with deductive inference patterns. More importantly, though, the connection between assessing an inference’s validity and argument reconstruction is not established (at most, [2] vaguely hints at such a link). Argument reconstruction and inference evaluation are depicted as two independent and separable procedures.

Finally, while [4] doesn’t engage in logical considerations when reconstructing natural language arguments and merely contains an appendix on formal logic, [6] doesn’t even mention the concept of logical validity — let alone apply it to assess inferences and reconstruct arguments.

Employing non-deductive inference schemes to reconstruct arguments

Deductive validity is no prerequisite for justificatory strength. There are good arguments which rely on non-deductive, or inductive inferences. Reconstructing non-deductive arguments in a charitable way is at least as challenging as analysing deductive arguments.

The chapters on deductive validity in [1] and [5] are directly followed by entire chapters on inductive, i.e. non-deductive, inference. Both books introduce various non-deductive inference schemes, including schemes for practical reasoning. Moreover, [5] discusses argument schemes for analogical reasoning and inference to the best explanation.

The other textbooks fare rather poorly. [2] only touches upon non-deductive inference in a superficial (and slightly mistaken) way (p. 62-63). [3] introduces “proved beyond reasonable doubt” as non-deductive standard of good inference, but doesn’t set forth argument schemes or inference patterns that satisfy this criterion. [4] and [6] don’t bring up inductive inference at all.

Assumptions for the sake of the argument and the reconstruction of suppositional reasoning

In a reductio ad absurdum, or indirect proof, you assume the contrary of what you want to demonstrate and then derive a contradiction. Arguments which represent such suppositional reasoning contain — besides premisses, intermediary and final conclusions — so-called assumptions for the sake of the argument. Employing assumptions f.t.s.o.t.a. in argument reconstructions is an advanced technique that also helps you to analyse justifications of conditional statements.

No book introduces assumptions f.t.s.o.t.a. in detail. [5] has a short, example-free section on reductio ad absurdum as deductively valid inference pattern. And [1] briefly discusses the technique of conditional proof. But in both books assumptions f.t.s.o.t.a. are not systematically treated as part of an argument reconstruction (like premisses, intermediary conclusions and final conclusions).

[2] mentions that, in suppositional reasoning, one spells out the consequences of hypothetical assumptions (pp. 104-105); but it gives no clue whatsoever how to reconstruct such a reasoning.

Fallacies

Natural language reasoning is full of typical, common errors. In argument reconstruction, it’s very helpful to be able to recognise such fallacies.

[1] and [5] explore fallacies in depth, not only in dedicated chapters, but throughout the entire text. The books present and systematize more than two dozens different fallacies, most of which are illustrated by examples. (In terms of illustrations, [1] does a better job than [5].)

The detailed discussions in [1] and [5] dwarf [2]‘s treatment of fallacies (three fallacies, informally discussed on 5 pages).

The other books don’t consider fallacies, understood as typical mistakes in reasoning, at all. (Although they may discuss individual cases of flawed reasoning on an illustrative basis.)

Tips and practical guidance for argument reconstruction

Argument reconstruction is an art which one has to practice so as to master it. While there’s no algorithm to follow when reconstructing an argument, rules of thumb and practical tips may nonetheless provide helpful guidance for novices.

As regards such practical guidance, [1] clearly does best. It contains a valuable chapter dedicated to “the practice of argument-reconstruction”. Topics covered include: the need to focus and to leave aside unrelated material when reconstructing and argument; the handling of ambiguities; the clarification of the logico-semantical structure of premisses and conclusions (“logical streamlining”); the use of general principles as connecting premisses; semi-formal strategies for logical assessment.

Both [2] and [5] fall short of [1]‘s detailed advice, but give some helpful tips on their own, e.g.: [2] rightly recommends to reconstruct an argument backwards from its conclusion; [5] highlights a couple of typical mistakes made in argument reconstruction and suggests useful rules of thumb, such as the rabbit rule: “Every key term appearing in the conclusion of an argument must also appear in at least one of the premises.” (p. 101)

All books stress the need to use precise language, e.g. to resolve ambiguities in an argument.

The post Six Critical Thinking Textbooks Reviewed (Textbook Reviews Series, #1) appeared first on Argunet.