TreeJuxtaposer was created for the visual comparison of large trees, especially phylogenies, although it could be applied to other domains (Munzner et al. 2003). TreeJuxtaposer's main goal is the automatic detection and visualization of structural differences among hierarchies. It uses a similarity measure that computes the best corresponding node for each internal node and each leaf node. The hierarchical structure is represented by a rectilinear layout similar to a dendrogram and classifications are placed side by side, fitting the size of the screen without the use of scrolling. It relies on color to visualize similarities and differences. Fig. 2 shows an example of the representation used by TreeJuxtaposer. It shows the structural differences between two taxonomies, emphasized through colored-coded edges. TreeJuxtaposer implements an accordion-type distortion technique as the focus+context mechanism that supports the concept of guaranteed visibility of selected areas. Brushing and linking facilitate visual exploration: the selected node and the best corresponding nodes in all other trees are temporarily highlighted. Users can also apply the linked navigation option in order to compare analogous areas in each tree, so that subtrees underneath the best corresponding nodes in other trees are resized and synchronized with the selected subtree.

TaxVis (Graham and Kennedy 2007) is a tool designed to explore relationships between multiple taxonomic trees through concept relationships. A concept is defined as a unique combination of (name, author, and date). TaxVis inputs are a set of hierarchies and a set of concept relations given by expert taxonomists (Graham et al. 2008). The main visualization requirements are: to track siblings and parents of a genus across hierarchies, to track children of a particular higher-level node across the hierarchies, and to compare the amount of levels across hierarchies (Graham et al. 2000). Fig. 3 illustrates the multiple tree view that holds the taxonomies being compared; this illustrates the main panel where users interact and identify relationships among taxa. A hierarchy is represented in a set-based visualization rather that in a node-link metaphor. Multiple taxonomies are placed from top to bottom within the panel and each taxonomy is also displayed in a top-down fashion with lower rank taxa drawn underneath their parent taxa, in an icicle plot style. Each taxon is represented by a rectangular box. When users select a taxon, the corresponding concepts in the alternative classifications are highlighted with color and edges; for instance, by selecting a genus, users can visualize the corresponding species of that genus as well as the location of those species in the other classifications. When users select a taxon, it is moved to the top of the displayed fragment of the classification, at the same time that the descendant taxa are displayed underneath it, and they are highlighted within the other classifications. Concept relationships such as congruent (=), non-congruent (≠), contains (⊃), is contained in (⊂), and overlaps (∩) (Graham and Kennedy 2007) are represented by color coded lines. The tool includes other views, such as a list of all taxa names within the data sets, attributes of a selected taxon, a control panel to regulate various properties of the display and interaction, and a history of selected comparisons. Complementary to TaxVis is the Concept Relationship Editor―CRE (Graham et al. 2008), a separate tool that allows the edition of concept relationships between two classifications.

The Taxonomic Tree Tool―TTT (Lin and Wang 2013) is a web-based application designed to compare and edit classifications. In TTT there are two types of users: general users and registered users. General users can access public information on the web site such as public classifications, public tree comparison cases, news, and statistics: registered users can additionally upload and edit their own classifications and compute tree comparisons. Tree comparisons can be visualized either through indented trees or through a node-link diagram in D3. In the indented tree layout, the comparison tree (CT) and the reference tree (RT) are placed side by side. Each tree has its own vertical scrollbar and scrolling is not synchronized between trees. At first, the trees appear compressed, that is, only the root node of each tree is displayed and a plus sign indicates that the node can be expanded to the next tree level. For each node the visualization includes rank, taxa, and a set of glyphs that depict relationships between nodes in the two compared classifications. Fig. 4 illustrates the case of comparing two fictitious taxonomies T1 (CT) and T2 (RT). The legend in the figure explains the meaning of the glyphs. The ancestor relation indicates whether the two compared taxa have the same ancestor route: 1 means same route and 0 stands for different routes. Nominal relations indicate how names are related. A green circle indicates that the name refers to exactly the same name and corresponds to the same taxon in both trees. A blue circle indicates that the corresponding taxa is a synonym, and a red circle shows no relation between both nodes. Similar to the ancestor’s case, the descendant relations evidence how related descendants are; for instance, it reveals whether the descendants´ branches are congruent (exactly the same), one branch of a tree is included in, or excluded from the other tree. Finally, the multiple links glyph indicates that a taxon in the CT has more than one relation to taxa in the RT. For example, family "Felidae" in both taxonomies indicate same ancestror, nominal and descendant relations in both hierarchies,

In the node-link layout, differences are color-coded. Users can filter by view type and visualize either overlaps, differences or both.

ProvenanceMatrix (Dang et al. 2015) is a visualization tool for exploring and analyzing the outcomes of taxonomic alignments (merges) generated with the reasoning toolkit EULER/X (Chen et al. 2014). EULER/X takes as input two taxonomies and a set of expert assertions that relate concepts between taxonomies at the leaf level. Experts express assertions through region connection calculus (RCC-5) relations such as equals, includes, is included in, overlaps, or disjoint (Franz et al. 2008). The reasoning toolkit produces alternative outcomes that can be visualized with ProvenanceMatrix. As shown in Fig. 5, ProvenanceMatrix uses a matrix to represent the relationships that may exist between elements from two taxonomies. Taxonomies are displayed along the axes of the matrix. The taxonomy on the horizontal axis can be displayed by using different orderings of the taxonomic concepts. Taxonomies can be displayed either in depth-first search, breadth-first or similarity ordering. In a similarity-based order the concepts are ordered by similarity of their articulation sets. Hierarchical parent-child relationships within each tree are drawn through lines that are painted over the matrix cells, and identation also helps to show hierarchical relations. The relationship between two taxonomic concepts are represented through glyphs within the cells of the matrix. Glyphs are color-coded circles that consist of five-piece slices that resemble a pie chart. Each slice is an indication of the relationship between concepts; for instance, a green slice indicates that concepts are equal and a blue slice indicates that a concept is included into the other one. Two concepts might hold several relationships, therefore a cell can contain several colored slices. ProvenanceMatrix is highly interactive: users can select different orderings of the hierarchies, perform brushing and linking to visualize relationships, collapse/expand sub-hierarchies, and filter the matrix by articulation type.

The expert-provided tasks for the visual comparison of taxonomic changes are organized in three categories, namely, pattern identification, query, and edit. Pattern identification tasks provide a means to recognize specific differences and similarities between alternative classifications; query tasks allow users to inspect more detailed information; and edit tasks let users make modifications to the classifications. We describe each task as follows.

Pattern identification tasks:

1. Identify congruence. Let T₁ and T₂ be alternative taxonomies. At the species level, congruence refers to equivalence of taxonomic concepts, and a concept is defined as the ordered triplet (scientific name, author, year). That is, species u in T₁ is said to be congruent to species v in T₂ when both species are identified by the same concept. At other higher level taxonomic ranks such as genus or family, a taxon p in T₁ is said to be congruent to taxon q in T₂ when both taxa have the same name.

2. Identify corrections. Differences between alternative taxonomies are due to revisions or to authors using different classification criteria. We consider the following four types of corrections: splits, merges, moves, and typos.

a) Identify splits: A split occurs when taxonomists divide one concept into two or more concepts. It is more likely that experts propose splits at lower level ranks, such as at family, genus, or species level. For instance, concept u in taxonomy T₁ can be split into concepts v₁, v₂, ... v_n in taxonomyT₂ , where either u = v₁ or u is a synonym of v₁, and v₂, ... v_n are new concepts. An inference algorithm for the identification of splits would need the taxonomic history of the involved species.

b) Identify merges: A merge occurs when taxonomists combine two or more concepts into one. It is more likely that experts propose merging concepts at lower level ranks, such as at the family, genus, or species level. For instance, concepts u₁, u₂, ... u_n in taxonomy T₁ can correspond to concept v in taxonomy T₂. At the species level, concepts u₁, u₂, ... u_n are registered as synonyms for concept v.

c) Identify moves: A concept can be identified as moved when it appears re-classified in another position of an alternative taxonomy; that is, a concept u in T₁ is re-classified as concept v in T₂ when parent(u) ≠ parent(v). An inference algorithm for the identification of moves would need the taxonomic history of the involved concepts.

d) Identify typos: A typo is a misspelling of a name.

3. Identify additions. Additions occur when new concepts are added to a taxonomy. In other words, concept v has been added to T₂ if v ∈ T₂ and v ∉ T₁.

4. Overview changes. This task presents an overview of corrections and additions as stated in items 2 and 3 above.

5. Summarize. This task refers to obtaining a numerical understanding of change between taxonomies T₁ and T₂, for example, with respect to number of species, number of split cases, number of merge cases, amount and percentage of species added.

Query tasks:

6. Find inconsistencies. Inconsistencies are due to circumstances that go beyond the different types of corrections described above and that refer to violations of rules (e.g., repeated names within one taxonomy or missing names in a newer version of a taxonomy).

7. Filter. This task refers to finding cases that satisfy certain conditions. Through filter criteria users can visualize selected pieces of information. For example, filtering by author or by date.

8. Retrieve details. The goal of his task is to retrieve the attributes of a particular concept. For example, retrieve the details of a concept with name "Passiflora coriacea" will display data such as, author, year, and its list of synonyms.

9. Focus. Navigate to an area of interest, in order to see the information in greater detail.

Edit task:

10. Edit. The goal of this task is to allow users to make changes in the classifications after analyzing the results of the comparison.

1. Identify congruence. As summarized in Fig. 7, most reviewed tools provide functionality to carry out this task. Brushing and linking, coloring, and the use of glyphs are common strategies for the identification of congruent relationships between alternative classifications. The identification of congruence is considered explicit in TaxVis, TTT, and ProvenanceMatrix. TaxVis relies on expert-provided information for the identification of congruence and users can visualize congruent relationships through exploration as well as brushing and linking. TTT calculates nominal relations that provide users with the information to infer congruence; nominal relations are represented through color-coded glyphs. In ProvenanceMatrix relations are given by experts and congruence is also identified by color-coded glyphs. TreeJuxtaposer does not handle attributes, so it does not consider concepts and does not have the mechanisms for the identification of congruence. Nonetheless, users can identify equal scientific names by browsing and doing visual exploration on the marked areas.

2. Identify corrections. Possible types of corrections are: splits, merges, moves, and typos.

a) Identify splits

Let us recall that according to the definition of split in the list of requirements, a split occurs in a classification by decision of human experts. In order to identify that a taxon has been split, it is required to know its taxonomic history; that is, whether in the past it belonged to a more general concept. From our literature review we observed that the identification of splits was not a distinct requirement for any of the assessed tools, therefore none of the tools address this task in an explicit way (see Fig. 8). However, since TaxVis includes concept relations given by experts, this tool can allow the identification of splits in an implicit way because taxonomists provide the relations that correspond to splits that occurred in the past. When selecting a taxon, concept relations are listed and are also drawn between the alternative classifications; thus, users can recognize splits by exploring the taxonomies and inspecting the provided taxon history. ProvenanceMatrix displays concept relations equals, includes, is included in, and overlaps, hence users might discover splits through a laborious exploration of these relations, although the tool does not address splits as such. In brief, we consider splits are implicit in TaxVis and not addressed in the other reviewed tools because the taxonomic history of taxa is not included in the data.

b) Identify merges

Analogous to the split subtask, the identification of merges was not a distinct requirement for the reviewed tools. A merged taxon can be determined by tracing its synonyms or when the concept relations are directly indicated. Since concept relations are given by experts in TaxVis, merges can be identified implicitly through visual exploration of correspondences between taxa. Merges do not occur in evolutionary trees, thus they are not considered in TreeJuxtaposer. TTT distinguishes synonyms by means of nominal relations but this is not enough to be able to visually find merges. Merges in ProvenanceMatrix might be discovered by a cumbersome exploration of the concept relations equals, includes, is included in, and overlaps, but the tool does not address splits as such. A summary of the ratings is presented in Fig. 9.

c) Identify moves

A taxon can be moved by a taxonomist to another position within the classification when, according to expert criteria, the taxon should be re-classified. Fig. 10 summarizes the assessment of the tools for this task. TreeJuxtaposer visualises structural changes among taxonomies via color-coded edges, therefore a re-classified taxon can be implicitly recognized by visual exploration of the classifications. TaxVis also supports this task in an implicit way by means of the visual exploration of expert-introduced concept relations. Other tools do not address this task. TTT does not provide functionality for the identification of cases where species within the Reference Tree (RT) have been classified under a different taxon in the Comparison Tree (CT). ProvenanceMatrix displays concept relations: equals, includes, is included in, overlaps, and disjoint. Through laborious exploration of the data users might discover the re-classification of species, however, the tool does not address taxon moves as such.

d) Identify typos

None of the tools contains functionality to support this task.

3. Identify additions

In one way or another all tools support this task. In TreeJuxtaposer, added nodes are marked in red color, the task is supported in an explicit way. In TaxVis, when selecting a non-leaf node, its corresponding descendants are highlighted within the alternative taxonomies; so, users can identify additions, in an implicit way, by performing visual identification of differences. In TTT, the red glyphs that represent no ancestor relation and the red glyps that indicate no nominal relation imply that new nodes have been added; thus we consider that TTT supports this task in an explicit way. In ProvenanceMatrix, this task is also approached in an explicit way: non-congruent nodes indicated in red color correspond to new nodes added to the taxonomy.

4. Overview changes

All tools visualize changes, but most of them do not exactly comply with the definition of task overview changes in the list of requirements (see Fig. 11). TaxVis' visualization of change is closer to this definition because users can identify splits, merges, and additions with little effort in an implicit way. TreeJuxtaposer's main focus is the visualization of topological changes, and for this it relies on color-coding, TTT visualizes several change conditions through color-coded and shape-coded glyphs, and ProvenanceMatrix uses color-coded glyphs; thus, our assessment is not addressed but with the observation that they provide similar functionality.

5. Summarize.

All tools present at least some basic numerical information in an explicit way. TreeJuxtaposer approaches this task through the find function and displays the amount of named nodes within a selected taxonomy. TaxVis displays the amount of subtaxa of a selected taxa and its percentage relationship with respect to a compared classification. TTT provides a wide range of statistics on taxons and on structure. Statistics on taxa are amount of orders, classes, families, genera, and species. Statistics on structure also indicate amount of equal taxa, amount of overlapped taxa, and amount of unmatched taxa for each taxonomic rank. ProvenanceMatrix provides a bubble chart that visualizes the proportion of each type of articulation.

6. Find inconsistencies

None of the tools supports this task.

7. Filter

TaxVis and ProvenanceMatrix support this task in an explicit way, whereas TreeJuxtaposer and TTT do not support it at all. TaxVis has a filter relations menu that allows the visualization of congruent, contains, included, overlaps, and is not congruent with relations. TreeJuxtaposer does not handle attributes, therefore it cannot filter attributes. ProvenanceMatrix provides filtering through different types of articulations.

8. Retrieve details

All tools support this task in an explicit way. TreeJuxtaposer includes a search by name mechanism. TaxVis retrieves the information of a node after selecting it. TTT provides a search-by scientific name feature. ProvenanceMatrix retrieves the data through a mouse-over operation and also pictures are retrieved from Wikipedia.

9. Focus

All tools except TTT feature focus mechanisms explicitly. TreeJuxtaposer achieves this task through features such as guaranteed visibility, accordion, lens and mouse pointer. In TaxVis this tasks is performed through increasing the size of selection. ProvenanceMatrix approaches this task by collapse, expand, resize, and order by features.

10. Edit

The comparison through visualization provides users with an understanding of differences and similarities between taxonomies, and afterwards editing becomes necessary. Only TTT provides functionality for this task, and it is explicit. Although TaxVis and ProvenanceMatrix do not approach this task, they both have complementary tools for this functionality. TaxVis edition complementary tool is the Concept Relationship Editor (CRE) and ProvenanceMatrix´s is EULER/X ‒ a multi-taxonomy alignment tool. The assessment is presented in Fig. 12.