| Table of Contents |
|---|
...
Web-based dashboard is a great way of understanding overall progress of the project. This provides the capability to analyze results of the test cases executed in Qualitia Automation Studio for Web, Mobile, Or Desktop applications.
You can either enable this dashboard functionality while installing/upgrading Qualitia Server or you can do it after you complete the installation or upgrade process.
Qualitia consumes an additional database (except master and project databases) for managing data related to the dashboard functionality.
To enable Reports and Dashboards Portal:
- Log on to the machine where you have installed Qualitia Server.
- Double-click the Qualitia Automation Studio Server icon.
The Welcome to Qualitia Setup Wizard screen appears. - Click Next.
The Database Server Details screen appears.
- Click Next.
- Select Enable Dashboard option and enter the desired name for the dashboard database.
- Click Next and then click Install.
Now you can access dashboards from the Dashboard section of Qualitia Automation Studio clients or using the dashboard URL on using any browser on any machine in the network.
Reports and Dashboard Portal URL contains the Hostname/IP:Port from the machine where Qualitia server is installed followed by the login page URL, that is, qualitia/sign-in. By default, system uses port number 8887 which users can change while installing Qualitia server.
The sample URL may look like:
You need to login and the landing screen contains the options to navigate to Web Dashboards, Real-time reporting or Server Administration portals, depending upon your user role in Qualitia.
Getting Started with Reports and Dashboards Portal
...
- green refers to the passed test cases,
- red refers to the test cases with defects,
- amber refers to the failed test cases,
- grey refers to the test cases that are Not Executed.
If test cases are marked for the execution and are not executed due to on-error flags or any such reasons, they are reported as Not Executed.
...
Execution Trend on Web
This chart shows the total count of executions run on the Web, with a split by status for each platform.
...
- green refers to the passed test cases,
- red refers to the test cases with defects,
- amber refers to the failed test cases,
- grey refers to the test cases that are Not Executed.
...
Trends in Defects Logged by Automation Tool
This simple line chart plots the total number of defects that have been logged by Qualitia Automation Studio at the end of executions with the client’s defect management tool over a selected period.
The X-axis shows the dates in which test case executions have occurred whereas the Y-axis shows the number of defects that have been logged during test case executions.
You can filter this chart based on the Date, Project, and Suites. The default view of the chart is at a day level. Higher levels such as Week, Month, and so forth can be selected from the provided Group by filter.
Hover on a dot to view the exact number of defects and click the data point to view the complete list of defects and other relevant details about these defects.
...
Drill View - Grid showing test case level details
...
The Real Time Reporting Portal allows you to monitor the test suite executions that are in progress or completed. It updates the results of a test suite execution in real-time after each step in the test case iteration is executed.
Also, QAS updates the local copy of the report after each task iteration is completed. If a test case does not have tasks, then it updates after each test case iteration execution is completed.
The Real Time Reporting Portal opens automatically when executing a test suite in the following ways:
- Through a Qualitia Client: It directly displays the Suite of Suites Screen.
- Through an Offline Suite: It displays the login screen if the offline suite used is present on your machine and configured with the Qualitia Server. Ensure that you have access to the associated project to track the test suite execution progress.
Real Time Reporting shows the reports of test cases executed through a TFS/Azure DevOps pipeline only after the pipeline execution is completed.
Real Time Reporting Portal does not support test cases executed through HP ALM.
Accessing Real Time Reporting Portal
...
Settings | Description |
|---|---|
| Enable Real-Time Reporting | Displays the test execution results in the Real Time Reporting Portal. Also, if Web Dashboards feature is enabled, you can view these records in the Dashboard portal. |
Maximum Size Per Log File | Lets you specify the maximum size for every log file of a test suite execution. The default value is 1MB. |
Application Step Log Size | Lets you specify the maximum size for logs of a step. The default value is 1MB and the maximum value you can define is 2MB. |
| Application Log Level | Generates Info Logs, Error Logs, or All Logs for the test suite executions. You can select None, if you do not want any. |
...
The Suite of Suites screen shows the list of the test suite executions associated with the projects to which you have access. You can choose to view the test suite executions that are in progress, completed, aborted or all of them. It also updates the in progress test suite executions in terms of percentage.
For every test suite, it displays the following details:
Test Suite Environment Details
Shows the test suite name, machine name, IP address, and project name.
Qualitia denotes the test suite execution is through a Qualitia Client and Offline denotes the offline suite execution. If the offline suite is executed through a Continuous Integration (CI) tool, then the CI tool name is specified beside the suite execution result.
Mouseover on the Environment Icon to view the operating system and browsers used for test case execution.
- The total execution duration
Test Suite Execution Details
The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. It also shows a progress chart with the completion percentage for the executions in the process.
- Realtime Reporting Portal displays the execution results as per the local time of your computer.
- If the execution progress for a suite execution shows a red dot, it indicates that the suite execution results are not updated from the last 5 minutes. You need to check whether the test suite execution is in progress or the connection to Qualitia server is lost.
Real Time Reporting portal marks the suite execution results as Aborted when they are not updated for more than 1 hour. For more information, refer to Aborted Test Suite Executions.
Server Space Notifications
...
In both cases, you need to contact the Qualitia Admin to purge reports. For detailed instructions on how to purge reports, refer to Purging Reports.
Filtering a Test Suite List
You can filter the list of test suites based on associated projects, test suite name, users, and date range. You can also search for a project, test suite, and user.
...
You can upload a locally saved test suite execution report to the Real Time Reporting when:
- During the execution, the results are not pushed to the Real Time Reporting Portal due to network failure or the services of Qualitia Server restarted
- An offline suite execution was not connected with Qualitia Server
- A test suite execution was not configured to display its results on the Real Time Reporting portal
- A test suite executed through parallel execution using Power Shell Utility
...
The Test Suite Execution Summary Report contains the execution status for each test case and its iterations. You can view the execution duration for each test case along with start and end time.
You can view the total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed.
The following details of the test suite are displayed.
Report Element | Description |
|---|---|
| Status Icon | Indicates the number of test case iterations that have failed and have defects along with the total number of test case iterations. Also, a refresh icon denotes the relevant test case iteration execution that is in progress. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. Upon hovering on the bug icon, all the associated defect IDs along with their links are displayed. Also, it shows whether the defect is logged automatically. |
| Test Case Element | Indicates the test case name or a test case iteration number. |
| Module | Indicates the associated module name. |
| Epic | Indicates the associated epic name. |
| Story | Indicates the associated story name. |
| Scenario | Indicates the associated Test Scenario name. |
| Manual TC Id | Indicates the associated Manual TC Id. |
The execution time for the suites executed on desktop applications does not count the time spent on launching UFT.
The Module, Epic, and Story columns appear only if the associated project of the test suite is integrated with a Qualitia Design Studio project.
If the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the following details of the test case:
The test case name
The iteration number in progress and the total number of iterations
- A timer shows the amount of time lapsed since execution
...
You can export the Test Suite Execution Summary Report as an excel file.
To export the Test Case Execution Summary Report:
- Click three dots, and select Export to Excel.
...
To customize the Test Suite Execution Summary Report:
Click three dots, and select Table Config to show or hide the columns.
Saving the Test Suite Execution Summary Report
...
You can log defects manually for a test case iteration whose execution status is Pass, Fail, or Defect. You need to configure the project with a Defect management System: Jira or TFS/Azure DevOps. For more information, refer to Integrating a Qualitia Project with Defects Management.
Test Case Iteration Detailed Report
...
- The execution duration along with the start and end time.
- The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. Also, a pie-chart shows the test case count per status for the test suite.
The following details of the test case iteration are displayed.
Report Element | Description |
|---|---|
| No | Indicates the step number. |
| Status Icon | Indicates the execution status of the step. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. You can click the defect id links to view the defects in the associated defect management system. Also, you can know the defects that are auto-created. |
| Test Case Element | Indicates one of the following test case elements:
|
| Data | Indicates the data associated with the action. |
| Object | Indicates the object name. |
| Execution Time | Indicates the execution time as per the associated hierarchy type. |
Also, if the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the details of the test case execution.
Filtering the Test Case Iteration Detailed Report
You can filter the report based on execution statuses.
...
To navigate to a specific line, enter the line number in the Go to Line text box and press Enter.
You can navigate through the failed steps and tasks in the report using the navigation controls shown in the following figure:
Note: Click the Failure Summary button to know failure categories for the test case iteration. For more information, refer to Viewing Failure Summary Report.
Viewing Screenshot and Logs for a Step
You can view the screenshots and logs of the application under test for each step execution.
To view screenshot and logs for a step:
...
- If the user terminates the test suite execution forcefully
- If the Qualitia Client or offline execution crashes
- If the Qualitia Server services restart
- If the Qualitia Server loses its connection with the Qualitia Client or Offline Package due to network failure
However, for the last two scenarios, the test suite execution continues, but its results are not pushed to Qualitia Server. In these scenarios, after the completion of the test suite execution, you can import the locally saved results from the Suite of Suites screen.
...