Matrices

Build sophisticated matrices in TW. Use them to create keys, score images and observations, manage projects, and streamline creation of custom outputs and sharing of descriptors.

Nomenclature Quick Start

TW provides a suite of common tasks help you to manage your matrices, descriptors, observations and related data. This section assumes you have some familiarity with the TW user interface conventions and introduces new user interface features where needed. Next, please familiarize yourself with the kinds of tasks TW offers related to Matrices.

Understanding the data model

Extensive information exists about the TaxonWorks data model. Information is presented in text and searchable graphic formats. A few comments:

Each matrix consistas of defined Rows and Columns.
Rows are OTUs, Collection Objects and other objects in the database which could be added to a matrix. The same OTU or any other object could be added to as many matrices as needed.
Columns are Descriptors (characters) which could be added to a matrix. The same descriptor could be added to sevaral matrices.
TaxonWorks does not store the entire matrix, instead we store individual observations for each cell in the table.

Important

In practice this means that if you have two matrices which include the same OTU and the same descriptor, the observations in that OTU/descriptor cell will be identical in both matrices, regardless of which matrix the original observation was made in.

This means you can clone and combine matrices as needed: observations will be replicated for each matrix having the same combination of Row object and Column object.

A portion of a TaxonWorks Observation Matrix

Curating data

Basic information

In most cases, work with matrices starts in the Observation Matrix Hub task.
Click the New button to create a new matrix.
- After clicking New you can give your matrix a name and optionally an OTU which the matrix describes. Check the Is public checkbox if you'd like the interactive key based on your matrix to appear in Taxon Pages, or if you'd like your matrix to be accessible from the API. Now click Create: you're now in Edit mode, which we'll cover below.
Alternatively, in the Observation Matrix Hub, find an existing matrix from the list and choose Edit from its radial navigator.
You can also pin a matrix in the Hub, so that it can be used as a default matrix in various interfaces.

Once in Edit mode, rows and columns need to be defined.

Adding a Descriptor (column) to the Matrix

To add a new Descriptor, click the New Descriptor link at the top of the interface. Select the descriptor Type. The available types are:
- Qualitative - traditional phylogenetic descriptor with multiple character states
- Presence/absence - similar to the previous one, but only two predefined states are possible
- Quantitative - a single measurement character (with an optional measurement unit)
- Sample - a numeric character, for which a min/max range and other statistical data can be scored (with optional measurement units per observation)
- Gene - ???
- Free text
- Media - a specialized descriptor designed for attachment of Depictions, which serve as a visual observation in image matrices (see below for details)
Show more mode allows adding some additional details about the descriptor: Short name, Description name - used for autogenerated descriptions, Key name - used in the interactive key of the matrix, Weight - larger values bring the descriptor closer to the top in an interactive key.
If Qualitative Descriptor is selected, two or more character states need to be defined. Each Character state has two fields: Label ('0', '1', '2', ...) and Name. Some additional information can be provided when Show more is selected.
- An example of a Qualitative descriptor: 'Body color' with two states: '0' - red, '1' - black.

Example of creating a Qualitative Descriptor; the `red` state has already been added

Once the descriptor information is entered you can click one of the Create buttons. In all cases, the newly created descriptor is added to the matrix as a Column.
- Create will save the record, but you can continue editing the descriptor
- Create and return to matrix will return you back to the Edit matrix interface
- Create and new can be used if multiple descriptors need to be defined
Alternatively, if descriptors already exist in the database, they can be added to an existing matrix in the Edit observation matrix interface by using the search interface on the lower left side. Change the Row\Column switch to Column, the Dynamic\Fixed switch to Fixed, and use the Search option. Alternatively, multiple descriptors from another matrix can be added using the From Another Matrix option. As explained above, neither of these options creates a new descriptor, it simply associates an existing descriptor with this matrix as a column.

A matrix to which 4 descriptors have been added (the right column)

Adding an OTU or Collection Object (row) to the Matrix

To add Rows to the matrix, change the Row/Column switch to Row and change the Dynamic/Fixed switch to the Fixed position. OTU and Collection Object are the mostly frequently used types as Rows, but other types can be added as well; select the Search tab to see your options.

Search options for observation matrix `Rows` in the lower left; multiple rows have already been added to this matrix

Alternatively you can add rows from your pinboard, by selecting from tagged objects, or from another matrix.

Other options for adding Rows:

You can add a new Row to the matrix directly from the Browse OTU or Browse Nomenclature task. In either task, select the OTU radial button, then the Observation matrices sector; from there you can select a matrix to automatically add this OTU to it and proceed directly to scoring mode (discussed below).
You can add many Rows at a time by selecting from filter results in any filter that has the Radial matrix button , including Filter OTU, Filter Collection Object, etc.
An Advanced option to add multiple OTUs to a matrix is the use of Dynamic rows. This option allows you to select either all OTUs marked with a specific Tag or select a parent taxon, for example a genus, to add all children taxa, for example species. Such rows are dynamic in the sense that if a new OTU is tagged or untagged with your chosen tag, or if new species are added to or deleted from your selected genus in the future, they will automatically be added to or removed from the matrix.

Reordering And Editing Rows and Columns in the Matrix

With the Sortable columns/rows checkbox at the top checked, you can resort columns and rows by dragging each up or down the list.
To delete a Row or Column from the matrix, use the Trash can button. Keep in mind, deleting a Row or Column does not delete the corresponding object from the database, it only deletes its association with a particular Matrix. Deleting Rows and Columns likewise does not affect associated observations (to do so you must (un)score a row/column pair before it's deleted from a matrix).
To edit a Row or Column, for example to add a new character state to a descriptor, select Edit in its radial navigator.

Adding scores to a Row

In the Edit Matrix interface, select the scoring button from the Row you want to score.
There you'll see the list of descriptors and for each descriptor its possible states. You can simply check applicable states or add values for numeric descriptors - all changes are saved automatically.

Scoring a row of a matrix by selecting character states for the descriptors of the matrix

Scores can also be cloned from another object in the matrix using the Clone and copy button. After cloning, only a few changes may be required to separate one OTU or Collection Object from another. It is recommended to select the most similar object for cloning.
The Destroy all observations in row button can be use to clear all scores from that row.
The Matrix Row Coder interface can also be accessed from the Browse OTU or Browse Nomenclature tasks. In either of those tasks, select the Observation matrices sector of the OTU radial button; from there you can select a matrix to automatically add this OTU to (if not added previously), and proceed directly to scoring mode.

Adding scores to a Column

In the Edit Matrix interface, select the scoring button from the Column you want to score.
You will see the list of OTUs and Collection Object Rows from your matrix. Check applicable states or add values for numeric descriptors - all changes are saved automatically.
On the top of the form, there are options to Destroy all observations, Populate column (which assigns one particular character state to every row) and additional aids to scoring your column.

Scoring a column of a matrix by selecting character states for the rows of the matrix

Using a Matrix

From the Observation Matrix Hub task, select View observation matrix from the radial navigator of the matrix. In view mode, you can preview the entire matrix or download the matrix in TNT, nexus, NeXML, or CSV formats, to be used, for example, in phylogenetic reconstruction or to be uploaded into a Lucid builder. See the top of this page for an image of an observation matrix being viewed.
Once the matrix is completed, it can be used for species determination as an Interactive Key; that task is accessible from any radial navigator for the matrix, see below for details.
Before using the matrix for identification, it is recommended to validate the matrix. To validate a matrix, switch to Edit mode and then select the checkbox Validation. Each row of the matrix will be evaluated to see if it has enough resolution to separate this particular row from all other rows in the same matrix. If a row does not pass validation, the matrix can still be used for interactive identification, but adding some additional descriptors may improve the resolution.

Interactive key interface

The Interactive key interface is divided into 4 frames: the top one with the title and options; the middle one with the list of descriptors; and two lower frames: the list of rows matching the selected character states, and eliminated rows which do not match the selected character states, to be populated once the identification is started.

By default the descriptors are sorted by their weight and separating power. Other options are available in the Descriptor sorting box in the top panel.
To start the identification, select a character state from any descriptor. It is possible to select more than one state in case of uncertanty: to do so, click on the descriptor name (not the dropdown list) and make multiple selections.
By default the rows which do not have scores for the selected descriptors are not eliminated from the list: it is assumed that any character state is possible. This behavior can be changed by checking the Eliminate unknowns checkbox in the settings on the top of the page, if it is needed, for example, for the matrix evaluation.
If a particular row in the matrix does not have a score matching a selected character state, this OTU or collection object is moved to the list of Eliminated rows. That behavior can be slightly changed by adjusting the Error tolerance in the top panel. A tolerance equal to '1' means that a single error in identification is permitted, the row will be eliminated only when 2 or more errors exist, etc.
To clear the interactive key, and start the identification from the beginning, use the Reset button.
If the matrix is large, it may be more convenient not to eliminate rows after selection of each character, since the wait time could be several seconds before you can select a next character. In that case you may want to check the Refresh only taxa option. When checked, the list of descriptors will not be updated until the Refresh button is clicked.
- After the refresh, the taxa not matching the search criteria are moved to the right, and the list of descriptors is also updated to bring more useful ones closer to the top of the list.
Continue the determination process in this way until only one or a few rows are still present in the list of Remaining.
It is also possible to use the Select button in the Remaining section to reduce the list of rows to be identified from the very beginning. If the list of entities is reduced, the list of descriptors will also be optimized to identify the remaining entries.
To make the interactive key more functional, you can add depictions to each character state of each descriptor. To do this, go back to the Edit Matrix interface and choose Edit from the radial navigator of a descriptor. In Edit mode, you can use the radial annotator next to each character state to add a depiction. Depictions of character states are displayed when you click on the name of a descriptor in an Interactive key.
If your project is set up on Taxon Pages, your interactive keys will automatically show up on their OTU page. For example here is a taxon pages interactive key to species of Erythroneura:

The interactive key interface on Taxon Pages

The Distinguish app

An observation matrix in TaxonWorks can be used as a multi-entry key within or outside of TaxonWorks using the app distinguish. Internally this integration extends the role of a multi-entry key engine far beyond its use in diagnosing taxa. For example the interface can be used to:

Evaluate progress in the construction of large complicated datasets
Create a single point of entry for adding annotations of all types that reflect workflows, scientific addenda, etc.
Navigate to and from other linked resources within TaxonWorks
Provide multi-language context for the underlying coding of data

Features include:

Ability to translate any Descriptor (= Character traditionally) into any number of languages
Qualitative weighting of descriptors
Prioritization of choices based on descriptor order, weighting, or next-best-algorithm
Toggling between ranks of terminals (only species, only genera, etc.)
Trivial re-use and reformulation of the key scope by adding or removing new descriptors or terminals
Rows from OTUs (taxa), CollectionObjects, or Extracts

Image matrices

An Image matrix is a specialized matrix which has defined descriptors and rows, but does not have any scores. Instead of scores, images are added as a depiction to each cell of the table. An image matrix can be used as a pictorial key for taxon identification; the images from the matrix are also displayed in the Browse OTU page. So instead of adding images directly to each OTU, they could be added to the image matrix instead to get additional functionality.

To work with an image matrix, create a new matrix as described above.
Add rows to the matrix, OTUs for example.
Add new descriptors to the matrix - all should be of type Media. When creating media descriptors, only give the descriptor a name (no states); we'll describe how to add images later below.
- Usually image matrices have fewer descriptors than a typical morphological matrix, and each descriptor will represent a standard view of the organism or a particular structure useful for determination of taxa. For example, 'dorsal view', 'lateral view', 'head', 'wing venation'.

Once the rows and columns are defined, the most convenient way to work with an image matrix is to use the Observation Matrix Dashbord task. Think of the dashboard as a filter task specialized for working with matrices, both image and otherwise.

In the Observation Matrix Dashbord task, first select a Taxon name as the parent taxon and then select the rank of the children taxa which you would like to see in your matrix using the Select ranks facet. For example, select a genus name, and then select species rank to see all the species-level taxa from that particular genus. Once selected, click the Search button.
You will see a table of OTUs with a summary including how many observations each has and how many depictions are added to a particular OTU in an image matrix (otu observation_depictions). Unselectable rows with green buttons are those taxon names without an OTU; check Show taxon names with OTUs only in the Otu facet and then Search again to not show those, and/or select Linked to valid only to only display valid names.

The `Observation Matrix Dashboard` showing data on the species of a genus

Use the Open in a matrix button next to one of the OTUs to add that OTU to a matrix - in this case select your image matrix from the list. Once the matrix is selected, you get redirected to an editing interface that looks like a table, where your media descriptors are the headers of the columns.
To add an image to a particular cell of the table, simly drag and drop it from a folder on your computer into that cell. Images can also be dragged from one cell to another.
Once images have been added, you can provide some metadata. Click on an image to add a caption. Click the Radial Annotator button to add a citation and/or attribution.
Adding images to multiple OTUs: in the Observation matrices dashboard, once you filter the list of OTUs, select a few of them and click the Edit image matrix button. The table for editing multiple OTUs will be opened. WARNING: do not select all OTUs for editing, since ALL of them will be added to the matrix as separate Rows.

You can select OTUs from the dashboard task in the same way you did above and use the Add to matrix button to add rows to any matrix, whether an image matrix or not.
The image matrix for an OTU can be accessed by selecting it from the Observation matrix slice of any OTU Radial button.
If your project is set up on Taxon Pages, your image matrices will automatically show up on their OTU page. For example here is an image matrix for 'American Stenacronus'':

The image matrix interface on Taxon Pages

Viewing image matrix images for a set of species

In the Observation Matrix Dashbord task, select a parent taxon and the rank of the children taxa which you would like to see, for example, select a genus name, and select species rank to see all the species-level taxa from this particular genus. Once selected, click the Search button.
In the filtered results, Select All or a few OTUs using checkboxes before the names, and click the View image matrix button. A table will open to display all illustrations from all media descriptors on any of the OTUs selected, so that OTUs can be compared to each other.
In View mode, you can hide some rows by clicking the checkbox before each row, or you can quickly switch to Edit mode.

Each descriptor and character state has additional fields, which are accessible using the Show more option in the Edit descriptor interface.
In the field Description name, provide alternative verbiage for a descriptor and each character state. For example, you have a descriptor "Head, length to width ratio" with a state "2.2". That is how the character is displayed in the matrix. For Description name, alternative values could be provided, for example "Head" for descriptor and "2.2x as long as wide" for the state, so that the autogenerated description would have a sentence "Head 2.2x as long as wide".
Two consecutive descriptors with the same Description name will be combined into a single sentence, for example "Head 2.2x as long as wide; brown".

The procedure is the same as for the automatically generated description, the alternative values can be provided in the field Key name.

By default, the descriptors are organized by their separating power, so that the descriptors that break the group more evenly are positioned closer to the top.
That order can be altered by adding a weight to each descriptor, using the Show more option in the Edit descriptor interface. Descriptors with higher weights are positioned closer to the top of the key.