| Table of Contents |
|---|
Introduction
The Real Time Reporting Portal allows you to monitor the test suite executions that are in progress or completed. It updates the results of a test suite execution in real-time after each step in the test case iteration is executed.
Also, QAS updates the local copy of the report after each task iteration is completed. If a test case does not have tasks, then it updates after each test case iteration execution is completed.
The Real Time Reporting Portal opens automatically when executing a test suite in the following ways:
- Through a Qualitia Client: It directly displays the Suite of Suites Screen.
- Through an Offline Suite: It displays the login screen if the offline suite used is present on your machine and configured with the Qualitia Server. Ensure that you have access to the associated project to track the test suite execution progress.
| Info |
|---|
Real Time Reporting shows the reports of test cases executed through a TFS/Azure DevOps pipeline only after the pipeline execution is completed. Real Time Reporting Portal does not support test cases executed through HP ALM. |
Accessing Real Time Reporting Portal
...
Enable the Real Time Reporting checkbox in the mapped execution profile or Systems Execution Profile Settings.
- The Real Time Reporting Server has enough drive space to displays test suite results. For more information, refer to Server Space Notification.
- For an offline suite execution, your computer is connected to Qualitia Server. Also, you need to specify the Qualitia Server URL in the Settings tab of Qualitia Offline Configuration Manager.
...
| Settings | Description |
|---|---|
| Enable Real-Time Reporting | Displays the test execution results in the Real Time Reporting Portal. Also, if Web Dashboards feature is enabled, you can view these records in the Dashboard portal. |
Maximum Size Per Log File | Lets you specify the maximum size for every log file of a test suite execution. The default value is 1MB. |
Application Step Log Size | Lets you specify the maximum size for logs of a step. The default value is 1MB and the maximum value you can define is 2MB. |
| Application Log Level | Generates Info Logs, Error Logs, or All Logs for the test suite executions. You can select None, if you do not want any. |
...
The Suite of Suites screen shows the list of the test suite executions associated with the projects to which you have access. You can choose to view the test suite executions that are in progress, completed, aborted or all of them. It also updates the in progress test suite executions in terms of percentage.
For every test suite, it displays the following details:
Test Suite Environment Details
Shows the test suite name, machine name, IP address, and project name.
Qualitia denotes the test suite execution is through a Qualitia Client and Offline denotes the offline suite execution. If the offline suite is executed through a Continuous Integration (CI) tool, then the CI tool name is specified beside the suite execution result.
Mouseover on the Environment Icon to view the operating system and browsers used for test case execution.
- The total execution duration
Test Suite Execution Details
The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. It also shows a progress chart with the completion percentage for the executions in the process.
| Info |
|---|
|
Server Space Notifications
...
In both cases, you need to contact the Qualitia Admin to purge reports. For detailed instructions on how to purge reports, refer to Purging Reports.
Filtering a Test Suite List
You can filter the list of test suites based on associated projects, test suite name, users, and date range. You can also search for a project, test suite, and user.
...
You can upload a locally saved test suite execution report to the Real Time Reporting when:
- During the execution, the results are not pushed to the Real Time Reporting Portal due to network failure or the services of Qualitia Server restarted
- An offline suite execution was not connected with Qualitia Server
- A test suite execution was not configured to display its results on the Real Time Reporting portal
- A test suite executed through parallel execution using Power Shell Utility
...
The Test Suite Execution Summary Report contains the execution status for each test case and its iterations. You can view the execution duration for each test case along with start and end time.
You can view the total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed.
The following details of the test suite are displayed.
| Report Element | Description |
|---|---|
| Status Icon | Indicates the number of test case iterations that have failed and have defects along with the total number of test case iterations. Also, a refresh icon denotes the relevant test case iteration execution that is in progress. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. Upon hovering on the bug icon, all the associated defect IDs along with their links are displayed. Also, it shows whether the defect is logged automatically. |
| Test Case Element | Indicates the test case name or a test case iteration number. |
| Module | Indicates the associated module name. |
| Epic | Indicates the associated epic name. |
| Story | Indicates the associated story name. |
| Scenario | Indicates the associated Test Scenario name. |
| Manual TC Id | Indicates the associated Manual TC Id. |
| Info |
|---|
The execution time for the suites executed on desktop applications does not count the time spent on launching UFT. The Module, Epic, and Story columns appear only if the associated project of the test suite is integrated with a Qualitia Design Studio project. |
If the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the following details of the test case:
The test case name
The iteration number in progress and the total number of iterations
- A timer shows the amount of time lapsed since execution
...
You can export the Test Suite Execution Summary Report as an excel file.
To export the Test Case Execution Summary Report:
- Click three dots, and select Export to Excel.
...
To customize the Test Suite Execution Summary Report:
Click three dots, and select Table Config to show or hide the columns.
Saving the Test Suite Execution Summary Report
...
You can log defects manually for a test case iteration whose execution status is Pass, Fail, or Defect. You need to configure the project with a Defect management System: Jira or TFS/Azure DevOps. For more information, refer to Integrating a Qualitia Project with Defects Management.
Test Case Iteration Detailed Report
...
- The execution duration along with the start and end time.
- The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. Also, a pie-chart shows the test case count per status for the test suite.
The following details of the test case iteration are displayed.
| Report Element | Description |
|---|---|
| No | Indicates the step number. |
| Status Icon | Indicates the execution status of the step. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. You can click the defect id links to view the defects in the associated defect management system. Also, you can know the defects that are auto-created. |
| Test Case Element | Indicates one of the following test case elements:
|
| Data | Indicates the data associated with the action. |
| Object | Indicates the object name. |
| Execution Time | Indicates the execution time as per the associated hierarchy type. |
Also, if the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the details of the test case execution.
Filtering the Test Case Iteration Detailed Report
You can filter the report based on execution statuses.
...
To navigate to a specific line, enter the line number in the Go to Line text box and press Enter.
You can navigate through the failed steps and tasks in the report using the navigation controls shown in the following figure:
Note: Click the Failure Summary button to know failure categories for the test case iteration. For more information, refer to Viewing Failure Summary Report.
...
Viewing Screenshot and Logs for a Step
You can view the screenshots and logs of the application under test for each step execution.
To view screenshot and logs for a step:
...
- If the user terminates the test suite execution forcefully
- If the Qualitia Client or offline execution crashes
- If the Qualitia Server services restart
- If the Qualitia Server loses its connection with the Qualitia Client or Offline Package due to network failure
However, for the last two scenarios, the test suite execution continues, but its results are not pushed to Qualitia Server. In these scenarios, after the completion of the test suite execution, you can import the locally saved results from the Suite of Suites screen.
...