Execute an Azure DevOps build or release pipeline by mapping it with an offline suite. Create an offline suite that includes the test cases relevant to the pipeline.

Pre-requisites:

Ensure that your computer:

Contains the Azure DevOps self-hosted agent.
Satisfies all the pre-requisites for offline suite execution. For more details, refer to Prerequisites for Offline Suite Execution.

Qualitia also recommends that you must know the fundamentals of offline suite executions except for the CLI commands. For more information, refer to Offline Package.

Known Issues or Limitations

Currently, you cannot have parallel executions of test cases.
Does not send the suite execution results to Azure DevOps.

Implementing the Components in your Computer

Enabling your computer to execute an offline suite through Azure DevOps includes the following process:

Creating a Build or Release Pipeline	Create a build or a release pipeline to map with Qualitia test cases. For detailed instructions, refer to the following links: Configuring a Build Pipeline Configuring a Release Pipeline
Creating an Offline Package	Offline package creation is a one-time activity on your computer to execute offline suites. For detailed instructions on offline suite creation, refer to Creating an Offline P ackage. Note: If you have installed the QAS client on another computer, create the offline package from that computer and transfer it to your computer.
Configuring the Offline Package File	Specifying in the Offline Package that the associated CI Tool is Azure DevOps. This is also a one-time activity. For detailed instructions on offline suite creation, refer to Creating an Offline Suite .
Creating an Offline Suite	Create a test suite that includes the test cases relevant to the pipeline. After a test suite is created, create an offline suite. For detailed instructions on offline suite creation, refer to Creating an Offline Suite .
Mapping an Azure DevOps Pipeline with an Offline Suite	Create two tasks in the Azure DevOps pipeline and configure them to map with the Offline Package and the associated offline suite. For detailed instructions, refer to Mapping an Azure DevOps Pipeline with an Offline Suite.
Executing the Pipeline	You can go ahead to execute the Azure DevOps pipeline after configuring it tasks. To know how to execute an Azure DevOps pipeline, refer to https://docs.microsoft.com/en-us/azure/devops/?view=azure-devops

Configuring the Offline Package File

You can skip this configuration if you have set the value of CI_Tool to Azure DevOps in the mapped execution profile of the offline suite that you want to execute.

To configure the Offline Package file:

Open the extracted Qualitia Offline folder.
Open the config folder.
Open the startupSettings file in a test editor, such as Notepad.
Set value of CI_Tool to Azure DevOps.

Mapping an Azure DevOps Pipeline with an Offline Suite

After creating a build or a release pipeline, map it with the corresponding offline suite. In the Azure DevOps pipeline, create two tasks that are used to:

Invoke the Offline Package and the mapped offline suite as per their specified locations.
Get the total number of test cases executed and the number of test cases executed for each execution status.

To map an Azure DevOps pipeline with an offline suite:

Open the pipeline in Azure DevOps.
Add two Command Line Prompt Utilities with names of your choice.
Azure DevOps adds these two as tasks.
To know how to add pipeline tasks, refer https://docs.microsoft.com/en-us/azure/devops/?view=azure-devops
Open the first task to enter the following information, in Script:
1. Enter the location of the Java path enclosed with inverted commas.
2. Add -jar separated by space next to Java path
3. Add Qualitia offline suite jar path enclosed with inverted commas next to -jar.
  For example, "C:\Program Files\Java\jre1.8.0_261\bin\java.exe" -jar "D:\offliesuites\Qualitia Offline\qualitiaoffline-8.3.1.jar"
4. Optionally, you can override the execution profile or any of its settings:
  - To override the mapped execution profile, enter the location of the new execution profile next to the offline suite jar path.
    For example, enter "D:\offliesuites\Qualitia Offline\Edge.json" separated by space after the Qualitia offline suite jar path.Program Files\Java\jre1.8.0_261\bin\java.exe" -jar "D:\855\863\Web\FF_Manual\Qualitia Offline\qualitiaoffline-8.3.1.jar" "D:\855\863\SFDC\ADO\FF\Qualitia Offline\Edge.json"
  - To override
Expand Advanced, enter the location of offline suite enclosed with inverted commas, in Working Directory.
For example, "D:\offliesuites\PaymentModule\Qualitia Offline".
Open the second task to enter the following information, in Script:
@echo PassedCount = %PASSEDCOUNT%
@echo FailedCount = %FAILEDCOUNT%
@echo DefectCount = %DEFECTCOUNT%
@echo NotExecutionCount = %NECOUNT%
@echo TotalCount = %TOTALTCCOUNT%
Note: You can specify the names of tasks, such as Qualitia Offline Suite Name and Suite Name Settings to identify the pipeline with the offline suite.

You can now execute the Azure DevOps pipeline.

Execution Guidelines and Best Practices

Qualitia recommends considering the following best practices to optimize the Azure DevOps pipeline executions that are mapped with offline suites:

If you want to push suite execution results to the Real Time Reporting portal, ensure that you specify the Qualitia Server URL and enable the Real Time Reporting checkbox in the mapped execution profile.
Before executing a pipeline, ensure that the mapped execution profile of the corresponding offline suite is configured as per the requirements. For detailed instructions, refer to Configuring the Settings for an Offline Suite.
You can execute a pipeline on different browsers, environments and other execution settings. Create different execution profiles in the QAS client and export them into the Offline Suite folder of your computer. For detailed information on execution profile, refer to Working with Execution Profiles.
Whenever relevant test cases are updated or newly added relevant in the corresponding test suite, perform the following steps:
- Create the offline suite again.
- Save the offline suite in another location of your computer.
- Open the Azure DevOps pipeline to replace the mapped offline suite.
You can edit the test data for an offline suite to cover more test scenarios. For detailed instructions on editing test data for an offline suite, refer to Editing the Test Data for an Offline Suite.

Viewing the Suite Execution Reports

If you have configured the Qulaitia Server URL in the mapped execution profile, you can track the Azure DevOps offline execution in the Real Time Reporting Portal.

The suite execution result is displayed as Offline | Azure DevOps besides to the suite execution cart.

However, if you did not connect with Qualitia Server or the connection with Qualitia Server was lost during the pipeline execution, you can upload the locally saved suite execution report in the following ways:

On this page:

Azure DevOps with Offline Package