Clusters, Clades, and the Chimera of Objectivity

Entering any science involves both mastering its accepted methods and developing a characteristic way of seeing things. What is so surprising about the connection between technical procedures and worldview? Here I will discuss some of the simplest procedures for processing biological data and, at the same time, the conceptual framework that underlies these procedures.
Three Approaches to Systematics
Let me begin from afar. In which branches of statistics has the contribution of biologists been most visible? If probability theory was developed mainly by gamblers, then the key ideas of so-called "variational" statistics were articulated by biologists. Francis Galton, Karl Pearson, Ronald Fisher, and others learned to identify common causes hidden beneath the veil of individual variation. Cluster analysis — a set of methods for building hierarchical classifications — also belongs to biology's statistical domain. This method, whose name derives from the English word "cluster," was described in 1939 by the psychologist Robert Tryon. The American systematist-biologists Robert Sokal and Peter Sneath became the classics of this approach: following their "Principles of Numerical Taxonomy" (1963), cluster analysis came to be widely applied both in biology and in other sciences (from astronomy to economics, from philology to data mining). What drove systematist-biologists to venture into the territory of statistics?
There is nothing surprising in the fact that biologists developed classification methods. The idea of hierarchical classification entered science through biology. The systematist Carl Linnaeus built on a tradition going back to the very "father" of biology (and a whole cluster of other sciences) — Aristotle.
Constructing a system became a high art. It was believed that to classify some group one had to deeply sense the regularities of its diversity. But some prefer one classification, others prefer another. How can the classification being developed be made independent of the individual systematist? Tensions between relationships of similarity and of kinship gave rise to two new scientific schools. A comparison of traditional (though ever-changing) systematics and the two new schools is presented in the table.
The founders of numerical systematics (or phenetics) — Sokal and Sneath — decided that only the assessment of organismal similarity could be objective. They had to refine cluster analysis precisely in order to expel subjectivism from biology. A different solution was proposed by phylogenetic systematics, which developed the approach of the German biologist Willi Hennig. Classification, according to Hennig, must be built solely on the basis of kinship, without considering similarity at all! Since phylogenetic systematists were interested only in the branching of the evolutionary tree (cladogenesis) and not in the change of individual branches (anagenesis), their opponents derisively began to call this school "cladistics" {whereas my former scientific supervisor called this school "egg-cladistics"}. Hennig's followers themselves adopted this "taunt" to designate their views.

Authorities

[IMG_1]

[IMG_2]

[IMG_3]

[IMG_4]

Ernst Mayr (1904–2005)

George Gaylord Simpson (1902–1984)

Peter Sneath (pictured) and Robert Sokal (b. 1926)

Willi Hennig (1913–1976)

Self-designation

Evolutionary Systematics

Numerical
Systematics

Phylogenetic Systematics

Name given by opponents

"Eclectic Systematics"

Phenetics

Cladistics

Classification by…

Similarity (including similarity by descent, i.e. kinship)

Observable similarity only

Kinship only

Primary instrument

Informalized selection of optimal compromise

Cluster analysis

Cladistic programs

"Objectivity"

An unattainable quality of the system

The primary ideal

It might seem that the views of pheneticists and cladists are diametrically opposed. But do you know what unites them? They are both convinced that their views are objective. Those who believe that similarity by kinship is one component of overall similarity and should be taken into account as such are constantly accused by their opponents of holding unscientific views. These views seem outdated, as does their defender, Ernst Mayr — a classic of systematics and evolutionary theory who recently died at the age of one hundred.
To compare the three named schools of systematics, we will attempt to classify some objects using "objective" methods. Perhaps such an exercise will allow us to reach some substantive conclusions of a general nature. What shall we classify? Let us take the covers of "Computerra" magazine!
Let us begin with a typical (agglomerative) cluster analysis with the construction of a classificatory "tree" — a graph.
First Stage of Clustering: Selection of Objects for Analysis
Defining the initial set of objects to be classified is a complex task. In the case of serious work, it must be delimited by some a priori (with respect to the investigation being conducted) criteria. These selection rules predetermine essential features of the clustering result.
Here I will not justify my choice of material. The set with which we are working is before you {taken from https://inside.computerra.ru/}. Why exactly seven covers, and why from issue #753 to #759? Stop asking silly questions — the method is objective!
The set of objects to be classified:

#753

#754

#755

#756

#757

#758

#759

[IMG_5]

[IMG_6]

[IMG_7]

[IMG_8]

[IMG_9]

[IMG_10]

[IMG_11]

Second and Third Stages of Clustering: Selection of Characters and Description of Objects
No, no, I will not conduct an art-historical analysis — if only because of my own incompetence. But I must provide an example of comparison.
The "rectangular" matrix (objects × characters)

Metaphor

No. of
people

Heading
at top

Obscene
hint

Color of "…terra"
and background

Color of "…terra"

#753

Not coordinated

183

#754

Print media

Yes

Not coordinated

105

#755

Print media

Coordinated

215

#756

Table

Yes

Partially coordinated

176

190

#757

Wall

Yes

Coordinated

#758

Table

Yes

Partially coordinated

102

#759

Wall

Yes

Partially coordinated

177

So, we select characters (characteristics by which the objects being compared differ from one another) and determine the states of these characters for each object. The first character is the metaphor underlying the image. In two cases it is something on a table, in two cases people at a wall, in two cases images of people in some print media, and in one case it is a menu. The second character is the number of people (as individual units) depicted on the covers.
Here we can comment on some of the difficulties associated with the procedure being applied. Why do I choose these or those characters? In the tongue-in-cheek classification I am constructing now — quite arbitrarily. And in a serious one? The old idea of using all possible characters is in principle unrealizable. Any given set of characters at a researcher's disposal reflects the researcher's (and their predecessors') notions of what is important and what is inessential. There is no hint of "objectivity" here.
Very well, we have selected our characters. But then we must unambiguously specify their states for the objects under study. Yet the diversity of reality and even of the objects of classification does not always fit without distortion into the Procrustean bed of our schematic representations. Should the girl on a playing card be counted as a depiction of a person, and should the monster on another card also be counted as such (in the case of #753)? And is the hand in #758 a person or not? In any case, the decisions I have inserted into the table will be used.
Do you suppose that in "serious" systematics the descriptions of organisms by accepted characters are unambiguous? No! A great deal depends simply on interpretations. To simplify the task of character interpretation, many biologists try to shift to classifying fragments of genetic texts. First we choose some segment of the genetic text, then we read it, then we "align" the sequences and compare them. The necessity of alignment is connected to the fact that genetic text can be interrupted by insertions, and segments can shift from one location to another, etc. To compare segments of common origin, they must be identified. Of course, an entirely "objective" procedure is applied for this purpose, but when different algorithms are used it yields different results.
The title of the issue's topic may be written at the top in an empty field, or it may be inserted into some element of the image. In two covers there is a hint at obscene language. In terms of color, there is both agreement between the color in which part of the journal's title is set and the color fills of the announcement panels at the bottom of the page with the background, and greater or lesser disagreement between them.
I (and probably the reader as well) am already growing tired of this. Well then, let us add more objectivity and finish. We can quantitatively assess the color of "...terra." The artist most likely drew in the CMYK color model {which describes the color space of printing with four inks (cyan, magenta, yellow, and black)}, while the covers downloaded from the web have been converted to RGB {intended for images formed by luminous points of three colors (red, green, and blue)}, but that is not so important. I open the cover images in a graphics editor and write down the values of the three color channels.
Done! Seven objects have been described by eight characters; the "rectangular" matrix (so called because the number of objects may or may not correspond to the number of characters) has been filled in.
Fourth Stage of Clustering: Comparison of Objects
We need to determine the similarity-dissimilarity relationship for all pairs of objects, i.e., to construct a "square" matrix (objects × objects). There are many ways to solve this problem. In fact, we cannot always even give a firm answer to the question of which objects are more similar to each other and which differ more strongly. Alas, for the choice of a metric of similarities and differences between the classified objects, there simply are no universally accepted (let alone "objective") criteria.

Which object is object A more similar to: B or C? If distance is used as the similarity metric, then to C: |AC|<|AB|. But if one relies on the correlation between the characters shown in the figure (which can be described as the angle between the vector pointing to the object from the origin and the x-axis), then to B: [IMG_12]. And which is correct? There is no single correct answer. On the one hand, an adult toad is more similar to an adult frog (both are adults); on the other, to a young toad (both are toads)! The correctness of the answer depends on what we consider more important.

[IMG_13]

In our example with covers, we have no need for excessive complexity, and we can simply use the sum of differences between objects for each of the 8 characters (calculating these differences as fractions of one). Let us compute the differences between #753 and #754. The metaphors are different — difference of 1. The number of people differs by 3; this is 0.6 of the maximum difference (between #756 and #758). The headings are in different positions — 1. There are no obscene hints in either — 0. The background and color scheme are not coordinated in either case — 0 (in other pairs, differences may also be 0.5 or 1). The difference in the red channel (R) is 177 units out of 255 possible, i.e. (rounded to tenths) 0.7. Similarly, the difference for G is 0.4, and for B — 0.2. In total, covers #753 and #754 differ by 1+0.6+1+0+0+0.7+0.4+0.2 = 3.9 units.
So, I present to you the "square" matrix showing the differences between objects. Imagine how much effort it took me to fill it in!
The diagonal of this matrix need not be filled in (each cover is identical to itself). Of the remaining cells, only half need be filled, since the matrix is symmetric about its diagonal.
The "square" matrix (objects × objects):

#753

#754

#755

#756

#757

#758

#759

#753

—

3.9

2.5

3.4

4.2

4.3

4.5

#754

—

3.6

3.3

4.8

2.2

3.4

#755

—

4.2

2.8

3.9

4.2

#756

—

5.3

1.6

3.4

#757

—

3.1

2.1

#758

—

2.6

#759

—

Could the relationships between objects have been calculated differently? Of course! We could have treated the character states as coordinates and computed the Euclidean distance between objects in such a space. We could also have used the square or the square root of the Euclidean distance. The metric we used resembles the sum of differences along each coordinate — the Manhattan distance, or "city-block distance" — only normalized as fractions of one. And there are also various measures of correlation and association. How does one choose among them? Either for convenience (as in our example), or on the basis of taste and tradition.
Fifth Stage: The Clustering Itself
Our task is to unite the most similar (i.e., least different) objects into clusters. Which pair of objects is most similar? Of course, #756 and #758, which differ by only 1.6 units out of 8 possible (with the main contribution to this difference being the different numbers of people on the cover). So we unite these two objects into one: (#756+#758).
But now we must answer a very important question. How do we determine the distance from a single object to a cluster? For instance, #753 is 3.4 units from #756 and 4.3 units from #758. What is the distance from #753 to (#756+#758)?
We could say that the distance from a point to a cluster is the distance to the nearest point of that cluster (this variant is called single linkage). Or perhaps to the farthest one (complete linkage)? Or perhaps we should compute the average? In some situations one option is more logical, in others — another. Single linkage gives rise to chain-like, elongated clusters. Complete linkage splits the set into relatively small lower-level clusters. Average linkage provides a certain compromise. And there is also Ward linkage, in which the variant that minimizes the increment of the within-cluster sum of squares of deviations is chosen for uniting an object with a cluster. How does one choose among these options? There is no "objective" criterion, and the choice, as in the previous case, is a matter of taste and tradition. One can only remind the reader that the super-idea behind using cluster analysis in biology was to free science from the influence of taste and tradition.
In zoological research, good results are often obtained using Euclidean metric and either Ward linkage or average linkage. Computing sums of squares is very laborious, so we will use average linkage. We can now reconstruct the "square" matrix for six objects:
The "square" matrix after the first clustering step

#753

#754

#755

(#756+#758)

#757

#759

#753

—

3.9

2.5

3.85

4.2

4.5

#754

—

3.6

2.75

4.8

3.4

#755

—

4.05

2.8

4.2

(#756+#758)

—

4.2

3.0

#757

—

2.1

#759

—

For the newly formed cluster, each cell contains the arithmetic mean of the distances that separated the remaining objects from its constituent elements.
What is the next step? Unite #757 and #759 at a level of 2.1 units. We reconstruct the "square" matrix again.
The "square" matrix after the second clustering step

#753

#754

#755

(#756+#758)

(#757+#759)

#753

—

3.9

2.5

3.85

4.35

#754

—

3.6

2.75

4.1

#755

—

4.05

3.5

(#756+#758)

—

3.6

(#757+#759)

—

We unite #753 and #755 at the level of 2.5 units. Probably there is no longer any need to present the reconstructed matrix — an attentive reader understands how I obtain it. The next step is the attachment of #754 to (#756+#758) at a level of 2.75. If I have not erred in my calculations, (#754+#756+#758) is united with (#757+#759) at a level of 3.6, and then (#754+#756+#757+#758+#759) is clustered with (#753+#755) at a level of approximately 3.84. Done!

Sixth Stage: Analysis of the Obtained Clusters

[IMG_14][IMG_15]

Alongside the tree in which the covers stand in order of their issue numbers, the result of rotation at its nodes is shown (observe how the tree "shuffled its feet," like a spider that has become entangled in its own web and has barely disentangled itself). As a result of this rearrangement, the resulting classification became far easier to perceive.
And is such a rearrangement of the obtained classification result permissible? Yes. The cluster structure reflects only those similarity and difference relationships of the objects that are explicitly represented in it. The resulting tree contains no information about the level of similarity between #754 and #755; it contains only information about the level of similarity of the clusters that include them.
Well, the outcome is interesting. The covers divided into "brownish-wall-obscene," "greenish-(table- and dollar-themed)," and "reddish" groups. I do not know whether the reader will manage this, but having clustered these covers, I myself was able to see them anew.
How reliable is the obtained result? There are several approaches to answering this question. I will mention only one. We drop one of the objects or add another (cover #760). Has the result changed significantly? We drop one of the characters or add another (the number of words in the title of the issue's topic). How does it look now? The overall assessment of the obtained classification is built up from the answers to these questions.
In general, the following can be said. When the structure of similarities and differences among the objects under study reflects several distinct levels of grouping into mutually nested groups, cluster analysis proves capable of elegantly displaying such a structure. For example, if we are comparing organisms of different, well-differentiated species belonging, say, to different families and different classes, cluster analysis will perform excellently. Where the levels of species, generic, and family-level differences are not strongly differentiated from one another, or where the data are altogether unstructured, cluster analysis will still produce some tree. Alas, that tree will be unstable and useless.

Some Conceptual Consequences
Let us begin by asking whether the transition from the "rectangular" matrix to the "square" matrix is unambiguous (given a chosen metric). Of course it is. Can the "rectangular" matrix be recovered from the "square" one? No. One and the same set of distances among the objects under study corresponds to many possible distributions of their character states. At the fourth stage of cluster analysis a significant loss of information about the properties of the objects occurs, and the same loss occurs at each step of the fifth stage.
So, the resulting tree reflects only a portion of the information about the comparison results. Yes, yes — it is the most characteristic portion... but only a portion. The rest is discarded. And if the clustering had proceeded along a different path, a different portion of the information would have been discarded.
What and who determines which portion of the empirical knowledge about the similarities and differences of objects will be reflected in the classification? What — the algorithm, and the contingencies of its application to a given set of objects and characters. Who — the researcher, who selected the set to be classified, the characters for describing it, forced the actual diversity of the objects into the accepted set of character states, and also applied one or another metric and one or another linkage method. Does the typical pheneticist-systematist recognize this? Usually not — after all, they are using an "objective" method!
We have established that cluster analysis reliably reflects only that hierarchy of groups that is well reflected in the structure of similarities and differences of the objects being classified. But such a structure will also be reflected by a system built by an evolutionary, "eclectic" systematist! They will strive to arrange the objects such that their position in the system reflects as fully as possible the properties they consider important, as well as concordantly varying characters. In constructing such a system, they may draw on their ideas about the kinship of the objects under study, but they will regard these ideas as merely one source of information. Alas, some of the information about the objects in question will have to be sacrificed, not being reflected in the classification. No great harm done — for such a choice will be made consciously, on the basis of experience in studying the group in question!

Searching for the Ancestral Cover
Having classified the covers of "KT" using phenetic methods, I felt that without recourse to cladistics my exposition would be incomplete. With some trepidation (cladistics has no shortage of fervent adherents) I step onto this treacherous ground...
The most important idea of Willi Hennig was the decision to reconstruct kinship not from all similar characters of the objects, but only from those that distinguish groups of relatives from their common ancestors {in cladistics such similarity is called similarity by synapomorphies, and it is fundamentally distinguished from similarity based on characters inherited from a common ancestor (symplesiomorphies) or from independently acquired features (false synapomorphies, i.e. homoplasies). Unique features of the classified object (autapomorphies) are also of no interest to cladist-systematists}. For this, one must establish which state of each character is ancestral (plesiomorphic) for the set of objects being classified, and which is derived (apomorphic). This problem can be solved in different ways. The best option is to choose some related object from another ("sister") group, and treat the character states characteristic of it as ancestral. Following this logic, I spent a long time examining the covers of "Domashny Kompyuter" ("Home Computer") (a "relative" of KT), trying to find among them one that would reflect the ancestral state of the characters I had used. Alas, I was unable to make an "objective" choice and was further filled with admiration for cladists, who are capable of doing so. I decided to use a different, "ontogenetic" criterion.
When an artist begins to draw a cover, no metaphor has yet been employed in it, there are no people, no hints. And what color of inscription should be considered ancestral? Black: R 0; G 0; B 0! Characters that can take several values are broken down into simple ones expressed as either zero (ancestral state) or one (derived state). The intensity of each of the color channels is assessed in three {and is the criterion used for distinguishing exactly three gradations of these characters an "objective" one? It is objective! The point is that 255 is divisible by 3 (85 per step), and this division corresponds to the logic of "little—medium—much." Moreover, many cladistics adherents believe that the use of an "objective" method grants them an indulgence for almost any arbitrary decision} steps, from absent or weak to maximum saturation. In none of the covers is the blue channel more saturated than one third (85). Since the objects do not differ on this character, we exclude it from analysis. Is the chosen decision arbitrary? Yes. But understand — any decision will be arbitrary, so ours is no worse than others!
Description of classified objects for cladistic analysis

Metaphor

Number of
people, at least

Heading
at top

Obscene
hint

Color coordination of "…terra" and background

Color of "…terra"

Print media

Table

Wall

Partial

Full

<85

>85

>170

<85

>85

>170

#753

#754

#755

#756

#757

#758

#759

In establishing the presumed course of evolution, cladistics accepts several axioms, of which we will mention two. First, when one species gives rise to another, the ancestral species must cease to exist in order for two co-equal descendant species to appear. Second, evolution must proceed by the most parsimonious path: of two phylogenetic trees (cladograms), the one that requires the minimum number of forward and reverse changes should be preferred. Testing of these axioms, where it proves possible (for example, when a good paleontological record exists), shows that evolution typically proceeds otherwise. Ancestral species live happily alongside their descendants, and parallelisms and convergences (at least in morphological evolution) are the norm rather than the exception. Why then do cladists insist on these axioms? Because without accepting them, the "objective" method does not work! A paradox? Cladistics has no shortage of such axioms — there is nothing surprising here.
So, to the bouquet of arbitrary decisions required for constructing a phenetic classification, cladists add yet a further series of assumptions vulnerable to criticism.
Very well, I take up cladistic programs. One is used to input the description of the objects. Another is for constructing the cladograms. A third is for viewing these cladograms (cladistics adherents do not shy away from difficulty). One would have to make sense of the command line in the user-hostile interface... I will skip the details and describe the result.
The clever program itself discarded several redundant characters from consideration, constructed 945 cladograms, and selected from them 39 minimal ones — each requiring 29 "evolutionary steps." I will present two of them as examples. Our initial data did not reflect the distribution of similarities and differences characteristic of objects linked by common descent, and therefore the cladograms turned out to be numerous and undistinctive. Thus, the simultaneous three-way split reflected in both cladograms shown in the figure indicates that the algorithm could not resolve the early stages of the "evolution" of this group (and it should not be blamed for this — the data are, after all, artificial).
[IMG_16]
Yes, the result is undistinctive. Cladistic analysis works well in those cases where the sought-for structure (the trace of history) is well reflected in the material. Need it be said that traditional methods in such a situation would also detect this trace? That said, cladistics does allow one to offload the sorting-through of large datasets onto software, which is of course no bad thing. Incidentally, cladists themselves will consider a satisfactory result one that conforms to traditional views on the evolution of the group under study. And in what will the novelty of their work then consist? Of course, in obtaining the traditional system by "objective" methods!

The Triumph of Democracy in Science
Has the lineage of Aristotle–Linnaeus–Haeckel–Mayr been broken off in biological systematics? One of my colleagues publicly asserts that it has. "Today there is not a single scientist who works within the compromise paradigm!" As paradoxical as it may seem, such a position is entirely unassailable. When one presents this colleague with examples of living and authoritative figures working in this tradition, he replies: "Those are not scientists, those are senile dotards! Today only that which conforms to the objective methodology of cladistics can be considered science."
...A young female instructor submits for the department's consideration the program of a special course in which the views of several Moscow paleontologists {I refer to the highly esteemed A.P. Rasnitsyn and A.S. Rautian, as well as the late V.V. Zherikhin} on the evolution of biological communities are mentioned. A cladist colleague insists on removing from the program any mention of the views of these scholars. The explanation is as follows. Within evolutionary systematics, the opinion of an authority who has deeply studied the diversity of some group is typically accorded more attention than the opinion of a beginning researcher. This can lead to the "suppression" of young scientists' opinions. Therefore, traditional systematics is authoritarian, while "objective" cladistics is democratic. The cited scholars do not approve of cladistics. Therefore they are potentially guilty of infringing upon the scientific opinions of young researchers. Although these scholars are mentioned in connection with an entirely different problem, my esteemed colleague demands that the young colleague be forbidden not merely to expound upon the views of the opponents of democracy, but even to mention their names. Do you think this is a joke? If only it were...
To some extent, the stronghold of evolutionary systematics is Russian science, while cladistics has triumphed among Western scholars. There are several possible explanations for this phenomenon. One (the cladists' favorite) is the "backwardness" of Russian science and the progressiveness of Western science. A second explanation connects the difference in approaches with the specifics of research funding.
When obtaining a grant, a Western scholar must set out a schedule of the work to be performed. At the first stage we do such-and-such, at the second — such-and-such, and such-and-such a result is to be presented by the end of the reporting period. The formalized cladistic approach allows one to easily plan such activities. Collect such-and-such categories of characters, feed them into the program, and obtain a result. Planning the activities of an "eclectic" systematist is far more difficult. Where a scholar lives from grant to grant, they will employ whatever methodology is convenient. In Russia (and throughout the Soviet Union and the post-Soviet space), scholars were until recently on salary. The majority of them did nothing, while some (probably out of idleness) began to reflect on how to carry out their task in the best possible way. Such rule-breakers often rejected the rigid constraints of cladistics. And indeed, Willi Hennig himself invented cladistics not within the framework of a project assignment, but rather in the context of reflecting on matters not directly related to his tasks...
Nevertheless, I believe that traditional systematics, in its third millennium of history, is in full flourishing — regardless of how many heads {the search for scientific truth by counting the number of adherents of a given viewpoint reminds me of the procedure of counting livestock by heads} its adherents number, and regardless of how much contemporary science funding impedes its development. Why? I will explain presently.

A Few Words in Favor of "Eclecticism" and Compromises
As you have already understood, several "objective" methods of systematics are currently popular. These methods allow large arrays of facts to be processed. Some of these methods yield a clear picture of the distribution of similarity and difference relationships; others — demonstrate relationships of kinship. Using these data will allow an optimal compromise to be found that will make possible the construction of the best system available. Paradoxically, the "objective" schools strengthen by their successes the school of their opponents, who gratefully accept any valuable information about the objects under study. The fuller our picture of knowledge about the classified objects, the better the compromise we will find.
And the claims of "objective" methods to exclusivity? I think they will eventually wither away. There is no need to take too much offense at those fervent adherents of young scientific schools who, having seized upon some one aspect of reality, deny all the others. They will mature, will recover from their childhood ailments... It is precisely for this reason that, being an adherent of compromise systematics, I consider it necessary to study both the methods of phenetic systematics and the methods of cladistics. Everything will be put to use!
And by the way, how should one correctly classify the covers of "Computerra"? There is no single, true system. That system should be considered natural which most fully reflects the essential properties of the objects from the point of view of the application for which the system is being created. The possible purposes of biological classification are a topic for a separate conversation. As for the readers and staff of "KT," the most convenient method of classifying covers is probably by the order of their issue numbers — the very one with which we began.
So, were our verbose reflections meaningless?! I hope not — they provided an occasion to reflect on general questions. All the best!

This article was prepared for publication in "Computerra," but was not published due to its excessive length by the journal's standards. The editor-in-chief of the journal advised me to start a blog and post this article there. A little time passed, and I followed his advice...