Multi Criteria Analysis

Introduction¶

A MCA provides a mechanism to an end-user (often called stakeholder) to compare, via performance criteria, two or more simulations and to define, via the definition of criteria weights, his/her own preferred solution(s).

This section covers the steps required to perform a Multi-Criteria Analysis. This consists of first creating a MCA Setup and then, associated with this setup, one or more MCA Sessions. Two or more MCA sessions can then be compared via a MCA Comparison.

MCA Setup¶

A new MCA Setup node is created in the Analysis explorer by selecting the database-, or group node, and selecting “New -> Multi-criteria analysis” from the context menu. The MCA Setup node can be renamed, copied, cut and pasted between groups similar to how files are handled in the Windows explorer.

An MCA Setup is opened by double-clicking the MCA Setup node in the explorer, or selecting “Open” from the node context menu. The MCA Setup spreadsheet view includes four worksheets: Indicators, Criteria, Normalized Criteria, and Pre-analysis. Only the Indicators worksheet is initially enabled.

Scenarios and indicators¶

The indicators that will later be used to calculate the criteria on which the MCA is based are imported either as model indicators (i.e. through models that have been registered in the Scenario manager), or by importing indicators from a text file.

Scenarios, along with model indicators, are added by clickin the “Add scenarios” button in the toolstrip , and selecting a model in the import pop-up dialog that appears. All scenarios and corresponding model indicators will be imported to the MCA Setup when clicking OK. In the example below, five scenarios and two indicators were imported.

Each indicator is represented by a column, and for those scenarios for which the indicator is calculated the indicator value will be added in the table. An empty cell indicates that the indicator has not been calculated for a given scenario.

The indicator name appears in the header line, and in line 2 the indicator is assigned an id. Finally, a unit may be specified in the third header line. A unit is assigned to an indicator by selecting the cell containing the indicator title, and setting the unit in the property control. Notice that units and indicator id can only be edited until the first MCA Session is created.

A given scenario can be removed again by selecting the cell that contains the scenario name, and clicking the “Remove scenario” button .

Updating scenarios with new simulations

Once a scenario has been added to a MCA Setup, a few properties can be set by selecting the cell containing the scenario name. The most important property is the “Simulation” property. By default this is set to the latest simulation of the scenario, but in case there are multiple simulations available for the selected scenario, the user can change the simulation to be used for the indicator calculation here.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML102a69.PNG$

To update all scenarios such that the latest simulation is automatically selected, click the update button in the MCA Setup toolstrip .

Adding indicators from the Indicator manager

To associate an indicator which is defined independently in the Indicator manager (i.e. not associated with the model through the model indicator functionality in the scenario manager), select a scenario in the table, and click the “Add other indicator” button in the toolstrip . Use the indicator selection dialog that appears to select an indicator and click OK. The indicator is added to the list of indicators in the table. Repeat for the remaining scenarios.

To remove an indicator, select the cell containing the indicator name and click the “Remove other indicator” button .

File based indicators

In most cases indicators will be imported through the scenarios that form the basis of a MCA Setup. In some cases it will however be feasible to import a set of prescribed indicator values that are not tied to a scenario in the scenario manager. This can be obtained by clicking the “Import indicators” button in the MCA Setup toolstrip .

A file selection dialog will open, allowing the user for selecting a file with the .scenario extension. The .scenario file is a text file with two rows, the first being a semi-colon (';') separated list of indicator names, and the other a semi-colon separated list of the corresponding indicator values.

An example containing four indicators is inserted below. The decimal separator must match the Windows settings.

The name of the imported .scenario file will be used as scenario name, and the name and value of the file based indicators will appear the same way as regular scenario-based indicators.

If a scenario with the same name already appears, the indicators will be imported to the existing scenario.
If the existing scenario already contains indicators of the same as those being imported, the imported values will replace the existing.
If the existing scenario does not contain indicators with the same name as those being imported, the new indicators will be appended to the existing scenario.

MCA Setup properties

Initially, a few basic MCA settings can be specified in the property control, after clicking in an empty cell in the Indicators sheet. The properties are:

Name – The name of the MCA Setup as specified in the Explorer.
Description – A description of the MCA Setup (optional)
Weighting method – This is where the weighting method to be used in all MCA Sessions is specified. Three methods are currently available:
- Ordinal ranking (default)
- Rating
- Fixed point scoring
Normalization – The method used to transform the calculated criteria to a common scale [0;1]. The following methods are availabe:
- Percentage of maximum (default)
- Percentage of range
- Percentage of total
- Unit vector

Notice that the MCA Setup properties can not be changed after the first MCA Session has been created.

MCA criteria¶

When all scenarios and indicators have been added to the MCA Setup, the next step is to combine the indicators into criteria using a mathematical expression. This is done under the Criteria tab.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML3cee26.PNG$

To add a new criteria, click the “Add criteria” button in the toolstrip. The new criteria gets added to the criteria table. The new criterion is provided with an autogenerated name. To get to the properties of the new criterion, select the cell containing the criterion name, and the propeties appear in the property control.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML503a24.PNG$

The properties are:

By formula (editable) – ‘True’ indicates that the criterion is calculated using defined indicators and a formula.
Direction (editable) – ‘Maximize’ indicates that larger values are ‘better’; ‘Minimize’ indicates that smaller values are ‘better’.
Formula (editable) – This is where the formula is defined. Select the value field and click

to get to the formula editor. In the editor the criteria can be expressed as a function of the indicators defined previously.
Name (editable) – The name of the criterion
Group (editable) – The group to which the criterion belongs
Significant number of digits (editable) - The number of digits displayed in the table.
Unit (editable) – The criterion unit.

Pre-analysis¶

The purpose of the pre-analysis is to identify dominant and redundant scenarios. A dominant scenario is one that ranks above all other scenarios on all criteria, and a redundant scenario is one that ranks the same for all criteria as one or more other scenarios.

The pre-analysis functionality can be found under the Pre-analysis tab in the MCA Setup sheet. Notice that this tab is not enabled until at least one criteria has been defined.

The pre-analysis table shows the rank for all scenarios and criteria, and the dominant and redundant scenarios are marked in color codes.

To remove a scenario or a criterion from the analysis, select the cell containing the name of the scenario / criterion and set the “Include in analysis” property to False. Notice that criteria and scenarios can only be removed and added until the first session is created.

MCA Sessions¶

When the MCA Setup is complete, one or more MCA Sessions can be added simply by selecting “New session” in the MCA Setup context menu in the Analysis explorer.

The new Sessions will be added as nodes under the parent MCA Setup

Notice that the MCA Setup is automatically saved when a new MCA Session is created. The Session nodes can be renamed, deleted and cloned from the node context menu.

The MCA Session configuration sheet is opened either by double clicking the session node, or selecting “Open” in the context menu.

This opens the MCA Session spreadsheet view, which includes four worksheets: General, Criteria, Normalized Criteria, Weights, Decision Matrix, Sensitivity, and Sensitivity Chart. Of these the first four are active.

Notice that the weighting method reflects the selected Criteria weighting option in the MCA Setup.

The first step in the MCA Session is for the stakeholders to specify lower and upper values for what they consider the acceptable range for a given criteria. The cells can optionally be left blank, indicating no upper and/or lower limit.

When the acceptable ranges have been specified, the scenarios that violates the specified ranges can be identified under the Criteria tab.

Criteria that falls outside the specified range are marked with red. The same information expressed with normalised criteria values are available under the “Normalised criteria” tab.

Next, the stakeholder will have to assign a weight to each criteria. This is done under the weights tab. The weights table format varies according the MCA Setup’s MCA Setup’s Criteria Weighting property, and the options for the different weighting methods are as follow:

Ordinal ranking – Criteria are ranked using integers from 1 – nCriteria, where nCriteria is the number of criteria. The most important criteria must be assigned rank 1.
Rating weighting – Criteria are assigned a qualitative measure ranging from Least important to Most important.
Fixed point scoring – All criteria are assigned a weight between 1 and
1. The assigned weights must sum up to 100. The higher the weight, the more important the criteria.

Once the weights have been assigned, the remaining tabs becomes enabled (Matrix, Sensitivity, and Sensitivity Chart).

To review the scenario scores and ranks calculated based on the specified weights, navigate to the Decision matrix tab.

In this sheet, the following information is summarised:

Scenario/Criteria combinations that violate the Acceptable Limits are highlighted red.
Scenarios that include violating criteria are not ranked.
The highest ranking scenario is highlighted green

To assess the sensitivity of a given criteria with regards to the specified weights, move on to the Sensitivity tab.

Here, the stakeholder are allowed to provide alternative weights, and immediately visualise the impact in the comparison tables above. Scores can be plotted simply by selecting the range of interest in the spreadsheet, and use the built-in spreadsheet plotting functionality .

Finally, the sensitivity of the scenario score as a function of a change in the weight of a given criteria can be inspected under the Sensitivity tab.

MCA Comparison¶

To compare the outcome of two or more MCA Sessions, select the sessions of interest in the Analysis explorer, and run the Compare sessions tool.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML8a0c80.PNG$

A new Session comparison node is created.

Open the Comparison view by double-clicking the node, or click “Open” in the context menu. The Comparison view shows how the different stakeholders ranked the analysed scenarios.

Scenario/Criteria combinations that violate a stakeholder’s Acceptable Limits are highlighted red.
Scenarios that include violating criteria are not ranked.
The highest ranking scenarios are highlighted green

The actual scores can be compared under the “2D Column” sheet.

Trade-off analysis¶

As part of a larger project for the Nile Basin Initiative, the MCA tool in the Workbench Analysis manager has been enhanced with an explicit trade-off analysis toolset.

The primary purpose of the trade-off analysis toolset is to help decision makers understand the consequences of selecting a scenario in lieu of another one, i.e. understand what is being gained and lost by selecting one scenario over another.

To enable this, the trade-off analysis toolset provides both graphical and tabular presentation of trade-offs between selected scenarios.

Accessing the Trade-off Functionality¶

The trade-off functionality is represented by this node under the MCA Setup . In order to perform a meaningful trade-off analysis at least two scenarios and two criteria must have been included in the MCA Setup. Assuming that this has been fulfilled, the trade-off functionality can be accedessed by selecting “Open” from the trade-off node context menu, or simply by double clicking the node. This will open the trade-off functionality view, which consists of a spreadsheet-like view with three sheets:

Scenario selection
Trade-off analysis charts
Trade-off analysis tables

The configuration of the trade-off analysis is done in the Scenario selection charts, and the result of the analysis is displayed under the Trade-off analysis charts and -tables sheets.

The Scenario Selection View¶

The scenario selection view is where the scenarios that are relevant for the analysis are selected and where configuration of the analysis is performed.

By default the view contains a table with one row for each scenario included in the MCA Setup. A scenario is identified by its name, and in the following columns the criteria values for the given scenario are displayed.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5c92836.PNG$

Show and hide criteria in the selection table

In some cases a large number of criteria values may have been defined for a given MCA Setup, and it may be that only a subset of these are relevant for the Trade-off analysis. In sush cases the user may choose to exclude those criteria that are no relevant. This is done in the Edit table configuration dialog which is opened by clicking the the “Edit table” link button above the selection table.

Criteria are excluded by first setting the “Number of criteria” in the Edit table dialog to the desired number, and then use the criteria selectors in the table below to specify which criteria to show in which column.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5d27d0a.PNG$

When clicking Apply or OK, the selection table will show only the criteria specified in the Edit table dialog.

Filtering the table

In those cases where a MCA Setup contains a large number of scenarios and/or criteria, it may be feasible to exclude the irrelevant scenarios from the table by setting up a filter. The filter functionality is accessed by clicking the “Filter table” link button above the selection table.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5ec8a43.PNG$

The Filter table dialog allows for specification of maximum and/or minimum for each criteria, and only those criteria that falls within the specified range will appear in the table after clicking OK or Apply.

A filter can be cleared by clicking the “Clear filter” link button.

Including and excluding scenarios in the Trade-off analysis

The first column in the selection table contains a check-box. By default, all criteria are included in the trade-off analysis, but one or more scenarios may be excluded by removing the check-marks.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5e7c85c.PNG$

All scenarios can be included/excluded by clicking the check-box in the header line. Notice that this will only affect the scenarios that are currently visble in the table (i.e. it will not affect those that have been excluded by a filter).

Include session rank and session scores in the selection table

To guide the user in selecting the relevant scenario for inclusion in the Trade-off analysis, it is possible to enable the display the session scores and session ranks when MCA Sessions have been performed.

This is done in the Edit table dialog, simply by checking the Show session ranks / scores check-boxes. The ranks and scores for each session is displayed at the end of the table for each scenario.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5d5eebf.PNG$

Calculating trade-off based on a deviation from a reference

By default the trade-offs are calculated simply by comparing the raw criteria values, but in some cases it may be more feasible to compare criteria by their deviation from a common reference. This common reference can be either the criteria from a specified reference scenario, or criteria that has been typed in manually.

A reference scenario can be specified in the Edit table dialog by selecting the “Deviation from reference” radio-button whereby the “Reference scenario” drop-down list becomes enabled, and the user may choose the scenario that should constitute the common reference.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML5dfb45f.PNG$

When a reference scenario is selected, the “Reference” column in the criteria table gets populated with the criteria values of the selected scenario. If the user chooses to edit one or more of the values, the selected scenario changes to “Custom” to indicate that it no longer represents one of the predefined scenarios.

To type in the references directly in empty fields, simply select the “Custom” option directly in the selection box, and type in the numbers.

After clicking Apply or OK, the selsection table will no longer contain the raw criteria values, but instead the deviations from the specified reference. The actual reference values are displayed in the row below the table.

Trade-off Analysis Charts¶

When the relevant scenarios and criteria have been selected in the selection table, the result of the trade-off analysis can be seen under the Trade-off analysis charts tab. This tab contains a number of charts depending on the number of included criteria.

Each chart contains a marker per scenario, and compares the scenario in terms of two criteria values. By default, all charts has the criteria of the first column in the selection table on the x-axis, and one chart for each of the following criteria.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML625b04b.PNG$

The number of charts, as well as the criteria on the axes can further configured in the dialog that opens when the “Configuration” link button is clicked.

Trade-off Analysis Tables¶

This sheet contains the raw criteria values for the selected scenarios, as well as the deviations in case a reference scenario has been selected.

$C:\Users\jov\AppData\Local\Temp\SNAGHTML629fab3.PNG$