Document toolboxDocument toolbox

Header

Work with Old Qualitia Reports

All the test suites executed through versions before Qualtia v8.0.0 Beta, can still be viewed from the Qualitia Client.

Accessing Old Qualitia Reports

After every test execution, Qualitia generates the test execution report. You can view the test execution report for a specific Release Number, Date, or Executions of the application under test.

  • Software Release: A software release is the private or public distribution of an initial or upgraded version of a software product.
  • Software Build: Builds are enhancements and fixes made on the application under test within a release, which are delivered to the test team to perform tests on these builds.

Donut Chart:

Each suite execution report has a donut chart in the top right corner of the home screen of the Summary Report. This donut chart displays the overall execution status of the suite. Based on the requirements, you can then filter to view the test cases with specific statuses.

Navigation Controls:

Additionally, in the Detail Report, you can route through the tasks or steps that are failed or have defects in them using controls given at the top right corner of the screen. Shortcut keys have also been enabled for this navigation and these can be viewed from the tool tips available for the controls. You can show/hide the log section using the control () provided for the purpose.

Note:

These features will be available for the suites executed after migrating to Qualitia automation studio version 5.7. These will not be available to the reports from older version of Qualitia.

To view a test execution report:

  1. From the File menu, click Old Qualitia Report.
    The Reports screen appears. This is a Report List. The status of the latest twenty suites within the current project is listed in a tabular format.
    These Suites are sorted on the Generation Date in descending order.
    You can apply filters based on Generated on Date, Release, and/or the Suite to view a specific report.
  2. Click the Show Calendar drop-down arrow and select Date, if required.
  3. Click the Select Release drop-down arrow and select Release Number.
  4. Click the Select Suite drop-down arrow and select Suite.
  5. Click Go.
    The list of Test Execution Reports related to the selected values appears on the screen.
  6. To view a detailed Test Execution Report of an execution, double-click a specific execution in the report list.
    A detailed test execution report and logs for a specific execution appear on the screen.

Understanding Report Legends

In Qualitia, color coding used in various reports is as follows:

Legends used in Develop functionality

Number of test cases automated.
Number of test cases not automated.
Legends used in Execute functionality

Number of test cases passed.

Number of test cases not executed.
Number of test cases failed.
Number of test cases with defects.
Legends used in Execute functionality

Status of test cases changed from failed to passed.
Status of test cases changed from passed to failed.

Understanding Test Execution Report

Qualitia displays the report on test execution results after the test cases are executed. Qualitia generates the test execution report even if the test case is not concluded successfully.

Details of Report List

The report list appears showing the status of the latest twenty executions within the current project in a tabular form. These executions are sorted on the Generation Date in descending order.

The report list describes Suite Name, Schedule Name, Build, Release, Iteration Number, User Name, Generated On, and Status of each Suite of the current project.

Details of a Test Execution Report

The test execution report displays a detailed summary of the test execution performed of a specific suite. It includes Project Name, Suite Name, Build Number, Release Number, Iteration, Start Time of the Execution, End Time of the Execution, Passed Test Cases, Failed Test Cases, Not Executed Test Cases, and Defected Test Cases. The percentage of test cases for each status is shown in the form of a donut chart in the header section of the report. The grid view of the Summary Report can be filtered to show test cases for a specific status by either selecting the particular donut slice for that status from the donut chart or by the Filter available on the Status column header in the grid report.

The details log gets generated during execution process which is integrated within the execution report to quickly analyse the results and identify application defects and resolve automation failures. It also displays a tabulated tree list of test cases associated with the executed suite. This list includes Name of the Test Case, Execution Status, Scenario, Task and Step details (Object and Action), and so forth.

Data used to Perform the Test Case

The tree can be expanded/collapsed at every test case level to view/hide the task and steps within a test case. Navigation controls have been provided in the top right section of this Detailed Report that will allow the user to traverse through the tasks and steps corresponding to Defect or Fail status in the grid section of the report. Clicking on any particular step in the report will navigate to the logs information specific to that step to analyse the results.

It displays Info Log, Error Log, and Debug Log on the bottom pane of the screen. The log sections can be shown/hidden from the button provided for the same at the top right section of the screen.

The following table describes the details of columns appear in report.

Heading

Description

Test Case Name

The status of the selected test case and the sequence and status of tasks associated with the selected test case.

Status

The status of the selected scenario and the sequence and status of test cases associated with the selected scenario.

Scenario

Indicates the name of the scenario.

Object

Indicates the object name.

Action

Indicates the action associated with an object.

Data

Indicates the data associated with an action.

Export To Excel

The Export to Excel feature enables you to export the summary report in a spreadsheet to a desired location. This feature exists in the Report tab of Qualitia Automation Tool.

You need to enable macros in Microsoft Excel before using this feature of Qualitia Automation Tool.

To export the summary report to Microsoft Excel:

  1. Go to the Report tab in Qualitia and open any report which needs to be exported in a spreadsheet.
  2. Click the link "Export To Excel", which is available on the right most upper corner in the Report tab.
    A spreadsheet opens up showing the summary report.

To save this spreadsheet, use Save As option available in Microsoft Excel. Give the desired name and select the file type .xlsx only.

(warning) We recommend to use only Microsoft Excel to save the summary report as the feature has not been tested with other spreadsheet formats.

Generating Canned Reports

Qualitia has provision to generate canned reports that provide visual analysis of the requirements synchronized from Qualitia Design Studio.

These reports help in evaluating the overall progress of test automaton coverage over the requirements created in the Qualitia Design Studio. 

You can store these reports in the CSV format for further use using any spreadsheet application.

The following types of reports are available: 

Report TypeDescription 
Stories without Test Cases

This report lists the following types of stories:

  • stories that do not have any business rules, or,
  • stories that have one or more business rules but not a single test case is created.
Business Rules without Test Cases

This report lists out all the business rules that are created in Design Studio but no test case is created for that business rule in Qualitia Automation Studio.

Business Rules with Partial Coverage

This report lists out all the business rules that have test cases with unimplemented activities. 

Unimplemented activities means;

  • activities that are disabled in all of the test cases created under that business rule or
  • activities of the business rules that are not having any test cases created. 
Requirements to Execution Results traceability

This report displays some pivot values related to the test case execution results. It displays:

  • total number of test cases present under different business rules,
  • number of test cases passed,
  • number of test cases failed,
  • number of test cases with defects during execution,
  • number of test cases non-executed (Suite execution started but test case not executed due to reasons like error flags and so on),
  • number of no run test cases. No run means;
    • test cases that are not yet added to any suite, 
    • test cases are added in the suite but that suite is not executed, or
    • test cases are added in the suite but not marked for execution.

(warning) Only Suite Execution will be considered when generating this report. Dry run execution will not be considered.

(warning) Qualitia considers stories that are marked for synchronization from the Design Studio when generating this report.

Import Qualitia Bot Reports

Qualitia allows test case execution in different environments. In order to do so, you can create a Qualitia bot (erstwhile Qualitia offline package) and execute test cases on-premise or in cloud environments based on your requirements. After test case execution, Qualitia gives you provision to add test execution results in the Qualitia. You can import Qualitia bot results from the Reports tab. 

Qualitia does not support importing reports of one project into another. 

To import offline package results:

  1. From the Reports tab, click Import Reports.
    The Browse The Folder screen appears.
  2. Navigate to the folder where you have stored offline package execution results. 
  3. Select the Iteration Level folder.
  4. Click OK.
    A notification appears about successful operation. 

Footer