Installation Process for Reports and Dashboards Portal

You can either enable this dashboard functionality while installing/upgrading Qualitia Server or you can do it after you complete the installation or upgrade process. 

Qualitia consumes an additional database (except master and project databases) for managing data related to the dashboard functionality.

To enable Reports and Dashboards Portal:

...

Reports and Dashboard Portal URL contains the Hostname/IP:Port from the machine where Qualitia server is installed followed by the login page URL, that is, qualitia/sign-in. By default, system uses port number 8887 which users can change while installing Qualitia server.

The sample URL may look like:

• http://192.168.25.xx:8887/qualitia/sign-in
• http://localhost:8887/qualitia/sign-in

...

Dashboards provide the capability to analyze the results of the test suites executed using Qualitia Automation Studio. They help product owners, managers, and testers  to get a holistic and comprehensive view of execution results across projects and test environments.

You can access the Web-based dashboards from the Automation Insights section.

Installation Process for Reports and Dashboards Portal

Getting Started with Reports and Dashboards Portal

Prerequisites

...

10 and 11

• Access to Qualitia Server machine and port 8887:

Accessing Web-Based Dashboards

• http://192.168.25.xx:8887/qualitia-reports/dashboard
• http://localhost:8887/qualitia-reports/dashboard

Last Updated

Dashboard Refresh Interval

Dashboards

Automation Test Progress Report

...

Dashboards

Real Time Reporting Portal

Introduction

• Through a Qualitia Client: It directly displays Suite of Suites Screen .
• Through an Offline Suite: It displays the login screen if the offline suite used is present on your machine and configured with the Qualitia Server. Ensure that you have access to the associated project to track the test suite execution progress. 

Accessing Real Time Reporting Portal

Pre-Requisites

• Enable the Real Time Reporting checkbox in the mapped execution profile or Systems Execution Profile Settings.

• The Real Time Reporting Server has enough drive space to displays test suite results. For more information, refer to Server Space Notification.

• For an offline suite execution, your computer is connected to Qualitia Server. Also, you need to specify the Qualitia Server URL in the Settings tab of Qualitia Offline Configuration Manager. 

Additional Configurations for Log Files

...

Maximum Size Per Log File

...

Lets you specify the maximum size for every log file of a test suite execution. The default value is 1MB.

...

...

Lets you specify the maximum size for logs of a step. The default value is 1MB and the maximum value you can define is 2MB.

...

Suite of Suites Screen

...

Test Suite Environment Details

• Shows the test suite name, machine name, IP address, and project name.

• Qualitia denotes the test suite execution is through a Qualitia Client and Offline denotes the offline suite execution either as a standalone or using a Continuous Integration tool.

• Mouseover on the Environment Icon to view the operating system and browsers used for test case execution.

...

Test Suite Execution Details

...

Server Space Notifications

• 10% to 30%: A warning message in amber appears at the top of the Portal page, which indicates the amount of drive space available along with the percentage. You must monitor the drive space and ensure that it does not reduce to less than 10%.

• Less than 10%: A warning message in red appears at the top, which indicates that Real Time Reporting cannot display information on new suite executions. Also, you cannot upload or save a report.

Filtering a Test Suite List

• Select a date range or click the Filter button in the top-right corner.

• Select the appropriate options from the list of filters.

Uploading a Test Suite Execution Summary Report

• During the execution, the results are not pushed to the Real Time Reporting Portal due to network failure or the services of Qualitia Server restarted
• An offline suite execution was not connected with Qualitia Server
• A test suite execution was not configured to display its results on the Real Time Reporting portal
• A test suite executed through parallel execution using Power Shell Utility

1. Create a ZIP file of the report folder that you want to upload.
2. Open the Suite of Suites screen.
3. Click the Upload button placed at the top-right corner.
4. Drag the ZIP file of the report or browse to upload the ZIP file of the locally saved test suite execution report.

Test Suite Execution Summary Report 

...

Indicates the number of test case iterations that have failed and have defects along with the total number of test case iterations.

Also, a refresh icon denotes the relevant test case iteration execution that is in progress. 

If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. Upon hovering on the bug icon, all the associated defect IDs along with their links are displayed. Also, it shows whether the defect is logged automatically.

...

• If the test suite execution is in progress, a blue-sticky status bar appears at the bottom of the screen, which shows the following details of the test case:

  • The test case name

  • The iteration number in progress and the total number of iterations

  • A timer shows the amount of time lapsed since execution

Filtering the Test Suite Execution Summary Report

1. Click the Filter button.

2. Select the appropriate filters.

Exporting the Test Suite Execution Summary Report 

• Click three dots, and select Export to Excel. 

Sharing the Test Suite Execution Summary Report

1. Click three dots, and select Share Link.
2. Click Copy to copy the link.

Wrapping up the Test Suite Execution Summary Report

Customizing the Test Suite Execution Summary Report

• Click three dots, and select Table Config to show or hide the columns.

Saving the Test Suite Execution Summary Report

1. Click three dots, and select Save As Report.
   The report is saved in the Download folder, which is specified in the browser's settings.

Logging a Defect Manually in the Integrated Defect Management System

Test Case Iteration Detailed Report 

• The execution duration along with the start and end time

• The number of steps that have failed or have defects along with the total number of steps 

• The number of tasks that have failed or have defects along with the total number of tasks  

• The execution duration along with the start and end time.
• The total number of test cases, with details of the number of test cases that passed, failed, have defects, and are not executed. Also, a pie-chart shows the test case count per status for the test suite.

...

Indicates the execution status of the step.

If the current project is integrated with a defect management system, then for defects logged automatically or manually, it displays a bug icon. You can click the defect id links to view the defects in the associated defect management system. Also, you can know the defects that are auto-created.

...

Indicates one of the following test case elements:

• Iteration number
• Task name
• Condition type
• Step name, which indicates the action performed.
  If a step contains results for an API response or generic actions such as CompareTabularResults, you can view those results in the associated info logs.

...

Indicates the execution time as per the associated hierarchy type.

Filtering the Test Case Iteration Detailed Report 

• Click the Filter button.

• Select the appropriate options from the list of filters.

Wrapping up the Test Case Iteration Detailed Report

Navigating through the Steps and Tasks

1. Select an appropriate step and click the Screenshot and Log icon. The screenshot and the log appears.

2. You can enlarge a screenshot and navigate across the screenshots of the next or previous steps.

3. The Info Log section shows the following details:
   1. Step Itinerary : Indicates the task, test case, scenario, and suite name.
   2. Message: Describes how the step was executed. If the step has failed, it shows the reason of failure. 
       If the step contains the results for an API response or generic actions, it contains a link that opens the results.
      You can view the results in the following formats:
      • json
      • xls
      • js (used for opening XML files)

   3. Additional Messages: Indicates the name of the action, its execution Start and End time, and associated message received from the engine. If it contains the additional message, click the + symbol to see the additional messages. 

   4. Execution Status: Indicates the execution status of the step as an integer.
      1. 0 denotes the step has passed.
      2. 1 denotes the step has failed.
      3. 2 denotes the step has a defect. 
4. The Error Log section shows if any exception occurred in execution.
5. The Configuration section shows the configured settings used for the execution.  

Aborted Test Suite Executions  

• If the user terminates the test suite execution forcefully
• If the Qualitia Client or offline execution crashes
• If the Qualitia Server services restart
• If the Qualitia Server loses its connection with the Qualitia Client or Offline Package due to network failure
  

Viewing an Aborted Test Suite Execution Summary Report

Viewing a Failure Summary

...

Testdata Provided Empty Or NULL

...

Uniqueness Violation

...

...

Indicates that the whole test case was aborted due to reasons such as incorrect application URL, incorrect environment selected, or invalid Javascript expression.

...

You can add custom failure categories to custom actions as per your requirement, refer to Adding a Failure Category for a Custom Action.

1. From the Suite of Suites screen, click a test suite execution for which you want to view the failure summary.
2. In the Test Suite Execution Summary Report, click the Failure Summary button.
   A list of associated failure categories is displayed along with the number of failed steps.
3. Do the following in the order listed:
   1. Click a failure category associated with the test suite. 
      The failure category expands to show the list of associated test case iterations where failure has occurred.
   2. Click a test case iteration.
      A list of applicable failure categories in that test case iteration is displayed.
   3. Click a failure category associated with the selected test case iteration.
      The failure category expands to show the list of associated failed steps of the selected test case iteration.
      If the step is part of a task, then the task name and task iteration are also shown.
   4. Click a failed step.
      The associated test case iteration detailed report opens and it highlights the selected step.
   5. You can view the screenshot and info log for the failed step.

Purging Data from the Server Administration Portal

Purge Data Tab

• Number of Test Execution Reports

• Total storage size of the suite execution report data

• Total storage size of the image data

• Total storage size of the log data

...

Column

...

Description

...

Generated On

...

Indicates the date on which the report was generated.

...

Suite Name

...

Indicates the name of the test suite

...

Project Name

...

Indicates the name of the associated project

...

Execution Type

...

Indicates the execution type:

• Qualitia denotes the test suite is executed through a QAS Client.

• Offline denotes that the test suite is executed through an offline package, either standalone or using a CI server.

...

Iteration No

...

Indicates the iteration number of the test suite execution

...

Total Size

...

Indicates the size of reports data including screenshots and logs

...

Report Size

...

Indicates the size of reports excluding screenshots and logs

...

Images Size

...

Indicates the size of associated screenshots

...

Logs Size

...

Indicates the size of associated log files

...

Test Cases

...

Indicates the number of test cases in the test suite

...

Test Case Status

...

Indicates the number of test cases that passed, failed, have defects, and are not executed.

Purging the Selected Reports

1. Click Purge Data.

2. Select any or all of the following checkboxes:

   1. All Report Files: To remove all the reports data of the selected reports to remove its entire folder structure. However, the execution summary of the suite execution report is stored in the database.

   2. Images Only: To remove only the screenshots captured for the selected reports.

   3. Logs Only: To remove only the log files for the selected reports.

3. Click Purge.
   An alert message appears that shows the number of reports and the size of files that are ready for removal.

4. Click Continue.

Filtering the Suite Execution Reports List

1. Select a date range or click the Filter button.

2. Select the appropriate options from the list of filters.

Logs Tab

...

Column

...

Description

...

Date

...

Indicates the date and time when the reports were purged.

...

Activity

...

Indicates the purging activity

...

User

...

Indicates the user who purged the reports

...

Purge Details

...

Indicates the type of reports data that was removed.

...

Data Size

...

Indicates the amount of data purged

...

Details

...

Click to view the details of test suite execution reports that were purged.

Purging Data from the Server Administration Portal

Uploading Multiple Suite Execution Reports to Real Time Reporting Portal

• From any computer
  You can directly upload the reports from your computer if its connection with Qualitia Server is steady. Also, it must have all the test suite execution reports that you want to upload.

• From the Qualitia Server
  You can use Qualitia Server to expedite the uploading of reports as there is no chance of network connection failure.

1. None of the reports you want to upload are:

   • Already uploaded to the Real Time Reporting portal, except for the ones that were aborted due to loss in network connectivity.

   • Incomplete as the suite execution is still in progress.

   • Generated using Qualitia version lower than 8.0.

2. Your system must have an offline package. If your system has Qualitia client, you can create an offline package. For detailed instructions, refer to Creating an Offline Package.
   Else, you can transfer an offline package from any other computer that contains the offline package to your system.

3. Qualitia server has enough space available. If the Real Time Reporting portal shows an error message that the available drive space is less than 10%, contact the Qualitia Admin to purge old reports to free up space.

...

Create a folder on the system from where you want to upload the reports.

...

Copy each suite execution report folder (ITR_xxx) that you want to upload.

...

Open the Command Prompt, and navigate to the folder where the Qualitia offline jar is present.

...

Enter the following code:
java -jar qualitiaoffline-8.3.0.jar UploadReport=true QualitiaServerURL="<Qualitia_Server_URL>” ReportPath="<Report_Path>" allowScreenshots="Yes"
The following parameters denote:

1. QualitiaServerURL is the URL of the Qualitia Server.

2. Report Path is the location of the folder that contains the suite execution reports

3. allowScreenshots is used to upload the screenshots of the reports. Enter “Yes” to upload screenshots or enter “No” if you do not want to upload associated screenshots.
   Note:

   1. If you are uploading reports from Qualitia server, then you do not need to specify allowScreenshots=”Yes”.

   2. If you do not enter the allowScreenshots parameter, then by default the screenshots of the reports are uploaded.

Press Enter to start uploading the reports.
For every report, the following details are displayed:

...

The upload progress in percentage.

...

The confirmation message for the reports uploaded successfully and they are removed from the reports folder.

...