Requirement Analysis

The Requirement Analysis section of dashboard provides a functional perspective to test execution results. This section allows analysing results with reference to the requirements (which includes Epics, Stories, and Acceptance Criteria) synchronised from Qualitia Design Studio.

Data in this section is available only if there are any Qualitia projects integrated with Qualitia Design Studio.

Important Notes

Dashboard data is available for duration mentioned above the date selector. You can select dates from that duration to view expected results in the dashboard.
Use the clear () button to remove all the selections for that filter. Clicking this option selects all the available options for the corresponding filter.
To select multiple options for any filter:
- If you are using Microsoft Excel 2013, press and hold the Ctrl key and select required options from the list.
- If you are using Microsoft Excel 2016,
  - either press and hold the Ctrl key and select required options from the list or
  - use the multi-select () icon and then select required options from the list.
If there is no data present for the selected filter criteria, dashboard are displayed empty.
To view the detailed information about any bar/line from the chart, hover your mouse pointer over that bar/line.
If a test case is executed multiple times in a selected time frame, multiple occurrences are counted while loading the metrics and charts.
Based on the selection of items from lower filters (for example, Suite/Browser/Operating System, if applicable), Qualitia alters the project's list depending on the data availability. This condition is not applicable if any particular chart or matrix works only on the project selection.
By default, the dashboard Excel file opens in a full screen mode due to which you may not see the menu options. However, you can continue using standard Excel shortcuts as usual, such as CTRL + P to print, CTRL + S to save and so forth.
For charts with radio buttons, after changing any filters, reselect the radio buttons to view the expected data. Due to limited event handling procedures in Excel, you may not see refreshed data immediately after changing the filters.

Filters

Date:

The date selector allows you to filter results based on the date of the test case execution. You can select either a single date or a date range using this date selector. You can select tenure from the available list. Available options are Days, Months, Quarters, and Years.
Dashboard data is available for duration mentioned above. You can select dates from this duration to view expected results in the dashboard.

Project:

You can filter the results displayed on dashboard using this project selector if you want to view results for a specific project or a group of projects from all the available list of projects.

Epic:

You can use this option to filter dashboard results based on epics from Qualitia Design Studio. You can select one or multiple epics from the available number of epics to view the expected results.

Story:

You can use this option to filter dashboard results based on stories synchronised from Qualitia Design Studio. You can select one or multiple stories from the available number of stories to view the expected results.

Acceptance Criteria (AC):

You can use this option to filter dashboard results based on Acceptance Criteria synchronised from Qualitia Design Studio. You can select one or multiple acceptance criteria to view the expected results. To add or remove unmapped instances of test cases while rendering the chart, use Unmapped AC.

The background colors of the filter options indicate different states of these options:

Blue background indicates that this option is selected.
Grey background indicates that this option is not selected but the data exists for that option.
White background indicates that this option is not selected and data does not exist for that option.

Metrics

The boxes present at the top display following information for selected criteria. You can click any number displayed in the box to view the list of Acceptance Criteria and other related information.

These metrics consider data of the last selected date only.
AC status is considered as Pass/Fail/Defect/Not executed only if all test cases under these are executed on the last selected date. Else, AC status is considered as Never Executed.
If test cases from that AC are executed multiple times, only the last execution status will be considered.
Deleted AC are also considered when refreshing the record.
When generating dashboard for the first time, the system does not consider Acceptance Criteria which do not have any test cases mapped with them. These AC might have test cases mapped and then deleted for one reason or another. But the system does not consider such AC if there are no test cases when loading data for the first time. In the subsequent synchronizations of the dashboard, the system considers AC without test cases only if they have the history of implementation earlier. Else, unimplemented AC will not be considered.

AC Passed - Displays number of Acceptance Criteria for which test cases implemented and passed after test case execution.
AC Failed - Displays the number of Acceptance Criteria for which test cases are implemented and failed after test case execution.
AC with Defects - Displays the number of Acceptance Criteria with defects.
AC Not Executed - Displays number of Acceptance Criteria that are implemented but test cases are not executed due to one reason or another.
AC Never Executed - Displays number of Acceptance Criteria that are implemented but test cases are not yet executed.

The Acceptance Criteria filter is not applicable for these Metrics.

Acceptance criteria count may affect in case there are instances for any test case executions when they are mapped and unmapped. You can use the Unmapped AC option from the Acceptance Criteria filter to add or remove the unmapped instances while rendering the chart.

A single acceptance criteria can have multiple test cases. Different test cases can have different statuses. In that case, acceptance criteria status is represented in the following permutations and combinations:

If status of any Test case is	then the status of AC will be represented as
Not Executed	Not Executed
Failed	Failed
Defect	Defect

The highest precedence is given to Not Executed status and then to Fail, Defect, and Pass respectively. The AC status is reported as Pass only if all the test cases from that AC are passed throughout all the test suites executed for the selected projects in the specified timeline.

For example, if any one test case is Not Executed from an acceptance criterion, then the status of that acceptance criterion is represented as Not Executed, without considering the status of other test cases. In similar way, other permutations and combinations are also possible.

AC Status: Distribution Across Latest Executions

This chart displays various daily statues of an AC (mentioned on the x-axis) during the selected dates. You can use all the filters to narrow-down the search results. Colors of the stacks represent Qualitia test case execution results as follows:

green refers to the passed test cases,
red refers to the test cases with defects,
amber refers to the failed test cases,
grey refers to the test cases that are Not Executed, and
blue refers to the test cases that are Never Executed.

If test cases are marked for the execution and are not executed due to ongoing error flags or any such reasons, they are reported as Not Executed.
If an AC contains any test cases which are never added to suite or never marked for execution, it is counted as Never Executed.

You can click any bar to view date-wise status of that AC in the ACStatusTrend section. The line chart in ACStatusTrend plots the daily AC status for the date range selected on the Requirement Analysis screen. This chart depicts the consistency of the AC based on the daily statuses.

Test Case Execution Trend

This chart displays the trend of test case execution results. In this stacked column chart, each bar represents a single date (mentioned on the x-axis) and test execution results are stacked on each other. You can filter this chart for specific Date and Project based on your requirements by selecting corresponding options from the filters. Colors of these stacks represent Qualitia test case execution results as follows:

green refers to the passed test cases,
red refers to the test cases with defects,
amber refers to the failed test cases, and
grey refers to the test cases that are Not Executed.

If test cases are marked for the execution and are not executed due to ongoing error flags or any such reasons, they are reported as Not Executed.

Increase the zoom level of the Microsoft Excel sheet in case value for any status is too low.

Coverage across Browsers

The horizontal bar from this chart aggregates and plots tests run against each browser and thereby through the relationship between test cases and acceptance criteria, it determines the coverage for automated tests against the browser.

You can use all the available filters to generate expected results in the chart. Additionally, you can click the bar to view the complete list of test cases and other related information.

Coverage across OS

The horizontal bar from this chart plots the number of tests that have been run on every operating systems, and portrays a view on the coverage of testing for selected requirements against an OS.

You can use all the available filters to generate expected results in the chart. Additionally, you can click the bar to view the complete list of test cases and other related information.

Requirements comparison by Defects

This chart allows comparison of Stories and Epics by the number of defects found while executing corresponding test cases. You can view this chart at two levels – Story Level and Epic Level. You can use Story or Epic filters to view the expected results. You can use buttons on top of chart to select the required chart level.

You can choose between top 5/10 or bottom 5/10 Stories/Epics to be considered by selecting an appropriate option.

Additionally, you can click the bar to view the list of test cases and other related information.

In the case of multiple suites having the same data, the information will be displayed only for top alphabetically matching suites in ascending order.

Test Case Execution Status

This chart displays the percentage of test cases with Pass, Fail, Defect and Not Run statuses against the total number of test cases from the selected Design Studio integrated projects, date range, and other selected filters.

Also, the segment colors represent various test execution statuses. Green represents pass, red represents defect, amber represents fail, and the grey color represents the Not Run/Not Executed status.

Qualitia Automation Studio for Web and Mobile 5.6