Installation Process for Reports and Dashboards Portal
You can either enable this dashboard functionality while installing/upgrading Qualitia Server or you can do it after you complete the installation or upgrade process.
Qualitia consumes an additional database (except master and project databases) for managing data related to the dashboard functionality.
To enable Reports and Dashboards Portal:
- Log on to the machine where you have installed Qualitia Server.
- Double-click the Qualitia Automation Studio Server icon.
The Welcome to Qualitia Setup Wizard screen appears. - Click Next.
The Database Server Details screen appears.
- Click Next.
- Select Enable Dashboard option and enter the desired name for the dashboard database.
- Click Next and then click Install.
Now you can access dashboards from the Dashboard section of Qualitia Automation Studio clients or using the dashboard URL on using any browser on any machine in the network.
Reports and Dashboard Portal URL contains the Hostname/IP:Port from the machine where Qualitia server is installed followed by the login page URL, that is, qualitia/sign-in. By default, system uses port number 8887 which users can change while installing Qualitia server.
The sample URL may look like:
You need to login and the landing screen contains the options to navigate to Web Dashboards, Real-time reporting or Server Administration portals, depending upon your user role in Qualitia.
...
Real Time Reporting Portal
Introduction
- Through a Qualitia Client: It directly displays the Res.
- Through an Offline Suite: It displays the login screen if the offline suite used is present on your machine and configured with the Qualitia Server. Ensure that you have access to the associated project to track the test suite execution progress.
| Info |
|---|
Real Time Reporting shows the reports of test cases executed through a TFS/Azure DevOps pipeline only after the pipeline execution is completed. Real Time Reporting Portal does not support test cases executed through HP ALM. |
Accessing Real Time Reporting Portal
...
| Settings | Description |
|---|---|
| Enable Real-Time Reporting | Displays the test execution results in the Real Time Reporting Portal. Also, if Web Dashboards feature is enabled, you can view these records in the Dashboard portal. |
Maximum Size Per Log File | Lets you specify the maximum size for every log file of a test suite execution. The default value is 1MB. |
Application Step Log Size | Lets you specify the maximum size for logs of a step. The default value is 1MB and the maximum value you can define is 2MB. |
| Application Log Level | Generates Info Logs, Error Logs, or All Logs for the test suite executions. You can select None, if you do not want any. |
...
Test Suite Environment Details
Shows the test suite name, machine name, IP address, and project name.
Qualitia denotes the test suite execution is through a Qualitia Client and Offline denotes the offline suite execution either as a standalone or using a Continuous Integration tool.
Mouseover on the Environment Icon to view the operating system and browsers used for test case execution.
- The total execution duration
Test Suite Execution Details
The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. It also shows a progress chart with the completion percentage for the executions in the process.
| Info |
|---|
|
Server Space Notifications
10% to 30%: A warning message in amber appears at the top of the Portal page, which indicates the amount of drive space available along with the percentage. You must monitor the drive space and ensure that it does not reduce to less than 10%.
Less than 10%: A warning message in red appears at the top, which indicates that Real Time Reporting cannot display information on new suite executions. Also, you cannot upload or save a report.
Filtering a Test Suite List
Select a date range or click the Filter button in the top-right corner.
Select the appropriate options from the list of filters.
Uploading a Test Suite Execution Summary Report
- During the execution, the results are not pushed to the Real Time Reporting Portal due to network failure or the services of Qualitia Server restarted
- An offline suite execution was not connected with Qualitia Server
- A test suite execution was not configured to display its results on the Real Time Reporting portal
- A test suite executed through parallel execution using Power Shell Utility
...
| Report Element | Description |
|---|---|
| Status Icon | Indicates the number of test case iterations that have failed and have defects along with the total number of test case iterations. Also, a refresh icon denotes the relevant test case iteration execution that is in progress. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. Upon hovering on the bug icon, all the associated defect IDs along with their links are displayed. Also, it shows whether the defect is logged automatically. |
| Test Case Element | Indicates the test case name or a test case iteration number. |
| Module | Indicates the associated module name. |
| Epic | Indicates the associated epic name. |
| Story | Indicates the associated story name. |
| Scenario | Indicates the associated Test Scenario name. |
| Manual TC Id | Indicates the associated Manual TC Id. |
| Info |
|---|
The execution time for the suites executed on desktop applications does not count the time spent on launching UFT. The Module, Epic, and Story columns appear only if the associated project of the test suite is integrated with a Qualitia Design Studio project. |
If the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the following details of the test case:
The test case name
The iteration number in progress and the total number of iterations
- A timer shows the amount of time lapsed since execution
...
Customizing the Test Suite Execution Summary Report
Click three dots, and select Table Config to show or hide the columns.
Saving the Test Suite Execution Summary Report
...
- The execution duration along with the start and end time.
- The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. Also, a pie-chart shows the test case count per status for the test suite.
| Report Element | Description |
|---|---|
| No | Indicates the step number. |
| Status Icon | Indicates the execution status of the step. If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. You can click the defect id links to view the defects in the associated defect management system. Also, you can know the defects that are auto-created. |
| Test Case Element | Indicates one of the following test case elements:
|
| Data | Indicates the data associated with the action. |
| Object | Indicates the object name. |
| Execution Time | Indicates the execution time as per the associated hierarchy type. |
Filtering the Test Case Iteration Detailed Report
Click the Filter button.
Select the appropriate options from the list of filters.
...
- If the user terminates the test suite execution forcefully
- If the Qualitia Client or offline execution crashes
- If the Qualitia Server services restart
- If the Qualitia Server loses its connection with the Qualitia Client or Offline Package due to network failure
| Info |
|---|
| In case of network failure scenarios, if the connectivity resumes before the test suite execution completes, then its results are displayed. As per the execution status, the test suite execution is displayed on the Completed or Aborted tab. |
...
| Failure Category | Description |
|---|---|
| Action Failure | Indicates that the defined action for the step did not execute correctly. |
| Action Is Not Supported | Indicates that the specified action in the step is not defined in Qualitia or the specified action cannot be used in the selected browser or environment. |
| Cannot Select From Dropdown | Indicates that the step did not execute as the specified drop-down list does not exist or it does not contain the option that is defined as a parameter. |
| Class File Is Missing In The Project | Indicates that the custom action did not execute because its source file is missing. |
| Failed To Store Data | Indicates that the action related to storing data failed. |
| File Does Not Exist | Indicates that either the specified file is not found in the specified location or the file extension is incorrect, or the step failed to create a file in the specified location. |
| Incorrect Data Format | Indicates that the parameters contain data in an invalid format, such as invalid date format, invalid file format, or invalid data encryption format. |
| Data Is Not Supported | Indicates that the parameter contains data that the object or the action does not support. |
| Invalid Expression | Indicates that the conditional expression syntax or the expression provided as input parameter for an action, is invalid. |
| Invalid Input Parameters | Indicates that the test data provided for the action is invalid. |
| Object Does Not Exist | Indicates that the object is not found or disabled on the selected browser or environment. |
Testdata Provided Empty Or NULL | Indicates that the test data is not provided for the action. |
| Timeout Encountered | Indicates that the associated object is not found within the specified Wait Time. |
Uniqueness Violation | Indicates that the unique values are not provided for this category. For example, if a value is already used in Qualitia and you try to use the same value in custom action it is reported as a uniqueness violation. |
| TestCase_Cannot_Execute | Indicates that the whole test case was aborted due to reasons such as incorrect application URL, incorrect environment selected, or invalid Javascript expression. |
| Expected Result Not Met | Indicates the result of a step is a defect when actual results and expected results are different. |
| Lack Of Permissions | Indicates that you do not have access to the environment or on application under test feature you are testing. |
You can add custom failure categories to custom actions as per your requirement, refer to to Adding Custom Categories in Custom Actions for Web and Mobile Projectsa Failure Category for a Custom Action.
- From the Suite of Suites screen, click a test suite execution for which you want to view the failure summary.
- In the Test Suite Execution Summary Report, click the Failure Summary button.
A list of associated failure categories is displayed along with the number of failed steps. - Do the following in the order listed:
- Click a failure category associated with the test suite.
The failure category expands to show the list of associated test case iterations where failure has occurred. - Click a test case iteration.
A list of applicable failure categories in that test case iteration is displayed. - Click a failure category associated with the selected test case iteration.
The failure category expands to show the list of associated failed steps of the selected test case iteration.
If the step is part of a task, then the task name and task iteration are also shown. - Click a failed step.
The associated test case iteration detailed report opens and it highlights the selected step. - You can view the screenshot and info log for the failed step.
- Click a failure category associated with the test suite.
...