Execute an Azure DevOps build or release pipeline by mapping it with an offline suite. Create an offline suite that includes the test cases relevant to the pipeline.

Pre-Requisites

Ensure that your computer:

• Contains the Azure DevOps self-hosted agent.
• Contains the Qualitia components: Offline Package and the required offline suites.
  Note: You can create these Qualitia components even if the Qualitia client is not installed on your computer. Simply create these Qualitia components from a computer where the Qualitia client is installed, and then transfer these components to your computer. 
• Satisfies all the pre-requisites for offline suite executions. For more details, refer to Prerequisites for Offline Suite Execution. 
  
  Qualitia also recommends that you must know the fundamentals of offline suite executions. For more information, refer to Offline Package.

Configuring the Qualitia and Azure DevOps Components

Enabling an offline suite execution through Azure DevOps includes the following configurations:

Configuring the Offline Package File

To configure the Offline Package file:

1. Open the extracted Qualitia Offline folder.
2. Open the config folder.
3. Open the startupSettings file in a text editor, such as Notepad or Notepad++.
4. Set the value of CI_Tool to Azure DevOps.

Mapping an Azure DevOps Pipeline with an Offline Suite

After creating a build or a release pipeline, map it with the corresponding offline suite. In the Azure DevOps pipeline, create a task to invoke the Offline Package and the mapped offline suite as per their specified locations.

• Get the total number of test cases executed and the number of test cases executed for each execution status.

To map an Azure DevOps pipeline with an offline suite:

1. Open the pipeline in Azure DevOps.
2. Add two Command Line Prompt Utilities pipeline tasks.
   To know how to add pipeline tasks, refer to https://docs.microsoft.com/en-us/azure/devops/?view=azure-devops
3. Open the first task and enter the following information, in Script:
   1. Enter the location of the 64 bit Java executable path enclosed with inverted commas.
   2. Add -jar separated by space next to Java path
   3. Add Qualitia offline suite jar path enclosed with inverted commas next to -jar.
      For example, "C:\Program Files\Java\jre1.8.0_261\bin\java.exe" -jar "D:\offlinesuites\Qualitia Offline\qualitiaoffline-8.3.1.jar"
   4. Optionally, you can enter CLI Commands to override either the execution profile or specific settings of the mapped execution profile, such as browser type. For detailed instructions on how to use CLI Commands to override the settings , refer to the Step 6 of Executing an Offline Suite using CLI Commands.
4. Expand Advanced, enter the location of offline suite enclosed with inverted commas, in Working Directory.
   For example, "D:\offlinesuites\PaymentModule\Qualitia Offline".
5. Open the second task to enter the following information, in Script:
   @echo PassedCount = %PASSEDCOUNT%
   @echo FailedCount = %FAILEDCOUNT%
   @echo DefectCount = %DEFECTCOUNT%
   @echo NotExecutionCount = %NECOUNT%
   @echo TotalCount = %TOTALTCCOUNT%
   Note: You can specify the names of tasks, such as Qualitia Offline Suite Name and Suite Name Settings to identify the pipeline with the offline suite.
   

Execution Guidelines and Best Practices

After mapping with the Azure DevOps pipeline, you can now go-ahead to execute the mapped pipeline. To know how to execute an Azure DevOps pipeline, refer to https://docs.microsoft.com/en-us/azure/devops

Qualitia recommends considering the following best practices to optimize the Azure DevOps pipeline executions that are mapped with offline suites: 

• Whenever the test cases of the corresponding test suite is updated in QAS, perform the following steps:
  • Create the offline suite again.
  • Save the new offline suite in another location of your computer.
  • Open the Azure DevOps pipeline to replace the mapped offline suite
• To allow the suite execution results to display in the Real Time Reporting portal, ensure that you specify the Qualitia Server URL and enable the Real Time Reporting checkbox in the mapped execution profile.
• Before executing a pipeline, ensure that the mapped execution profile of the corresponding offline suite is configured as per the requirements. For detailed instructions, refer to Configuring the Settings for an Offline Suite.
• To execute a pipeline on different browsers, environments and other execution settings. Create different execution profiles in the QAS client and export them into the Offline Suite folder of your computer. For detailed information on execution profile, refer to Working with Execution Profiles.
• You can edit the test data for an offline suite and execute the pipeline again to cover more test scenarios. For detailed instructions on editing test data for an offline suite, refer to Editing the Test Data for an Offline Suite.

Viewing the Suite Execution Reports

If you have configured the Qualitia Server URL in the mapped execution profile, you can track the Azure DevOps offline execution in the Real Time Reporting Portal.

The suite execution result is displayed as Offline | Azure DevOps besides to the suite execution cart.

However, in certain cases, the offline suite execution results are not pushed to Qualitia Server or are partially pushed when the connection to Qualitia Server fails during the execution. In this case, you can upload the locally saved suite execution report in the following ways:

• Uploading a Test Suite Execution Summary Report
• Uploading Multiple Suite Execution Reports to Real Time Reporting Portal

Known Issues or Limitations

• You cannot have parallel executions of test cases.
• Does not send the suite execution results to Azure DevOps.