Versions Compared

Old Version 6

changes.mady.by.user Shruti Agrawal

Saved on

compared with

New Version 7

changes.mady.by.user Shruti Agrawal

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

To integrate QAS with Azure DevOps, you need to first set up the environment.

Prerequisites

Prerequisites for Azure DevOps

...

Let’s set up the environment, to get started with the Azure DevOps Integration. This section depicts information related to all the prerequisites and procedures to get prepared for QAS's integration to Azure DevOps.

How Azure DevOps interacts with QAS components to execute QAS test cases successfully.

Qualitia Components:

  • Qualitia Server

  • Qualitia Client

Azure DevOps Components:

  • Azure DevOps as the Cloud Version

  • Azure DevOps Self Hosted Agent

To integrate QAS with Azure DevOps, follow these three key steps:

Association Between Qualitia Automation Studio and Azure DevOps

This step involves initiating the association from within Qualitia Automation Studio (QAS), mapping projects and test cases between QAS and Azure DevOps.

Key Activities:

  • Map QAS Project to Azure DevOps Project: Establish a link between your project in QAS and the corresponding project in Azure DevOps.

  • Map Automated Test Cases: Link QAS automated test cases with Azure DevOps manual test cases, ensuring that test cases are synchronized across both platforms.

  • Associate Test Suites: Map Qualitia Suites to Azure DevOps Test Suites to group relevant test cases for streamlined execution.

Execution Setup in Azure DevOps (One-Time Setup)

This one-time setup ensures that QAS test cases can be executed directly from Azure DevOps pipelines.

Key Activities:

  • Setup Build/Release Pipelines: Create and configure build or release pipelines in Azure DevOps to execute the associated QAS test cases.

  • Task Extension for Custom Tasks: Install the Qualitia Task Extension, which allows custom Qualitia tasks to be included in the pipeline installation. (Task Extension is used for Qualitia Custom tasks for pipeline installation)

  • Configure Self-Hosted Agents: Set up a self-hosted Azure DevOps agent for executing the QAS test cases locally.

  • Agent Pool Setup: Create and configure an agent pool in Azure DevOps to manage agents responsible for running QAS executions.

Result Publishing and Execution

This step ensures that the results of the executed test cases are published and available both in Azure DevOps and within QAS.

Key Activities:

  • Publish Results in Azure DevOps: After each pipeline execution, results of the QAS test cases are published to Azure DevOps test cases and test runs.

  • Result Sync with QAS: Test case results are synchronized with Qualitia Automation Studio to provide a holistic view of the automation results within QAS.

The following table explains how different components of Azure DevOps and QAS interact with each other.

Component

Roles Played in Different Stages

Azure DevOps

Association

  • Responds to QAS server using standard out of the box APIs offered by Azure DevOps.

Execution

  • Configures of build/release pipelines.

  • Agent pool configurations.

  • Initiates pipeline execution.

Result Publish

  • Hosts results of pipeline execution.

Azure DevOps Agent

Association

None

Execution

  • Executes the jobs in the pipeline along with the contained tasks.

  • Select Test Plan and Test Suite that contain the automated test cases and suites within the Azure DevOps projects.

Result Publish

  • Execution of the Publish Result task of QAS to publish results to QAS Server and Azure DevOps Test Runs.

  • Uses QAS Server APIs to publish result.

Qualitia Server

Association

  • Interacts with the Azure DevOps server to update automation status of Azure DevOps test cases.

Result Publish

  • Receives test case results directly from QAS - Client with the help of QAS APIs.

Qualitia Client

Association

  • Allows the authorized users to view Azure DevOps projects and associated projects and test cases.

Execution

  • Publishes the test cases to QAS project repository. The published test cases are used in the Azure DevOps pipeline execution.

Result Publish

  • Shows the status of test cases executed through Azure DevOps pipeline in Real Time Reporting Portal.

Prerequisites

Prerequisites for Azure DevOps

...

Azure DevOps Version

...

  • Azure DevOps (Cloud Version)

  • Azure DevOps Server 2019

...

 Qualitia Plugins

...

Share Azure DevOps URL to Qualitia team to get the Get Assembly task plugin.

...

 Roles and Privileges

  • Azure DevOps Administrator must install the plugins from Shared Extensions.

  • Qualitia does not recommend this option for PAT generation as it changes the access levels for all the users in the group.

Note: Please connect with your Azure DevOps Admin team to discuss the PAT generation and access level management.

Qualitia integration with Azure DevOps expects that the user has “Update Work Items” access that is used to generate the PAT (Personal Access Token).

The PAT with required access can be procured in one of the following ways:

  • Generate the PAT with a user that is part of the Project Administrators group.
    Qualitia recommends this option for generating a PAT.

  • Allow Bypass Rules on Work Items Updates
    Perform the following steps:

    • In Azure DevOps project, ensure that the user that is used to generate PAT is member Contributors group / Project Team members group.

    • Click Project Settings > Permissions.

    • Select the user group that has the user to whom you want to provide access (Contributors/Project Team members).

    • From the Bypass rules on work items updates list, select Allow.
      Refer to the following screenshot.

  • Have project access to create the following:

    • A new pipeline (Build/Release)

    • An Agent Pool

Prerequisites for Qualitia

Roles and Privileges

  • You must have administrative privileges to access Qualitia Project that you want to integrate with Azure DevOps tests.

 Network Access

Ensure that the Qualitia Server URL is reachable from the Qualitia Client, Azure DevOps Server, and Azure DevOps agent

Azure DevOps Version

  • Azure DevOps (Cloud Version)

  • Azure DevOps Server 2019

 Qualitia Plugins

Share Azure DevOps URL to Qualitia team to get the Get Assembly task plugin.

 Roles and Privileges

  • Azure DevOps Administrator must install the plugins from Shared Extensions.

  • Qualitia does not recommend this option for PAT generation as it changes the access levels for all the users in the group.

Note: Please connect with your Azure DevOps Admin team to discuss the PAT generation and access level management.

Qualitia integration with Azure DevOps expects that the user has “Update Work Items” access that is used to generate the PAT (Personal Access Token).

The PAT with required access can be procured in one of the following ways:

  • Generate the PAT with a user that is part of the Project Administrators group.
    Qualitia recommends this option for generating a PAT.

  • Allow Bypass Rules on Work Items Updates
    Perform the following steps:

    • In Azure DevOps project, ensure that the user that is used to generate PAT is member Contributors group / Project Team members group.

    • Click Project Settings > Permissions.

    • Select the user group that has the user to whom you want to provide access (Contributors/Project Team members).

    • From the Bypass rules on work items updates list, select Allow.
      Refer to the following screenshot.

  • Have project access to create the following:

    • A new pipeline (Build/Release)

    • An Agent Pool

Prerequisites for Qualitia

Roles and Privileges

  • You must have administrative privileges to access Qualitia Project that you want to integrate with Azure DevOps tests.

 Network Access

  • Ensure that the Qualitia Server URL is reachable from the Qualitia Client, Azure DevOps Server, and Azure DevOps agent.

Azure DevOps Integration

Let’s set up the environment, to get started with the Azure DevOps Integration. This section depicts information related to all the prerequisites and procedures to get prepared for QAS's integration to Azure DevOps.

How Azure DevOps interacts with QAS components to execute QAS test cases successfully.

Qualitia Components:

  • Qualitia Server

  • Qualitia Client

Azure DevOps Components:

  • Azure DevOps as the Cloud Version

  • Azure DevOps Self Hosted Agent

To integrate QAS with Azure DevOps, follow these three key steps:

Association Between Qualitia Automation Studio and Azure DevOps

This step involves initiating the association from within Qualitia Automation Studio (QAS), mapping projects and test cases between QAS and Azure DevOps.

Key Activities:

  • Map QAS Project to Azure DevOps Project: Establish a link between your project in QAS and the corresponding project in Azure DevOps.

  • Map Automated Test Cases: Link QAS automated test cases with Azure DevOps manual test cases, ensuring that test cases are synchronized across both platforms.

  • Associate Test Suites: Map Qualitia Suites to Azure DevOps Test Suites to group relevant test cases for streamlined execution.

Execution Setup in Azure DevOps (One-Time Setup)

This one-time setup ensures that QAS test cases can be executed directly from Azure DevOps pipelines.

Key Activities:

  • Setup Build/Release Pipelines: Create and configure build or release pipelines in Azure DevOps to execute the associated QAS test cases.

  • Task Extension for Custom Tasks: Install the Qualitia Task Extension, which allows custom Qualitia tasks to be included in the pipeline installation. Task Extension is used for Qualitia Custom tasks for pipeline installation.

  • Configure Self-Hosted Agents: Set up a self-hosted Azure DevOps agent for executing the QAS test cases locally.

  • Agent Pool Setup: Create and configure an agent pool in Azure DevOps to manage agents responsible for running QAS executions.

Result Publishing and Execution

This step ensures that the results of the executed test cases are published and available both in Azure DevOps and within QAS.

Key Activities:

  • Publish Results in Azure DevOps: After each pipeline execution, results of the QAS test cases are published to Azure DevOps test cases and test runs.

  • Result Sync with QAS: Test case results are synchronized with Qualitia Automation Studio to provide a holistic view of the automation results within QAS.

The following table explains how different components of Azure DevOps and QAS interact with each other.

Component

Roles Played in Different Stages

Azure DevOps

Association

  • Responds to QAS server using standard out of the box APIs offered by Azure DevOps.

Execution

  • Configures of build/release pipelines.

  • Agent pool configurations.

  • Initiates pipeline execution.

Result Publish

  • Hosts results of pipeline execution.

Azure DevOps Agent

Association

None

Execution

  • Executes the jobs in the pipeline along with the contained tasks.

  • Select Test Plan and Test Suite that contain the automated test cases and suites within the Azure DevOps projects.

Result Publish

  • Execution of the Publish Result task of QAS to publish results to QAS Server and Azure DevOps Test Runs.

  • Uses QAS Server APIs to publish result.

Qualitia Server

Association

  • Interacts with the Azure DevOps server to update automation status of Azure DevOps test cases.

Result Publish

  • Receives test case results directly from QAS - Client with the help of QAS APIs.

Qualitia Client

Association

  • Allows the authorized users to view Azure DevOps projects and associated projects and test cases.

Execution

  • Publishes the test cases to QAS project repository. The published test cases are used in the Azure DevOps pipeline execution.

Result Publish

  • Shows the status of test cases executed through Azure DevOps pipeline in Real Time Reporting Portal.

Configuring Azure DevOps for Test Executions

...

  1. Share Azure DevOps Organization Details: Before integration, provide your Azure DevOps organization details to Qualitia. Qualitia will share the necessary extensions for integration with your Azure DevOps projects.

  2. Add and Install Extensions: Once Qualitia shares the extensions, you can proceed with adding and installing them in your Azure DevOps instance.

  3. Installation Steps:

    • Sign into Azure DevOps instance.

    • Click the top-right corner to navigate to Marketplace > Manage Extensions.

    • Locate the QAS Qualitia - Test Execution extension shared by Qualitia.

    • Click the QAS Qualitia - Test Execution and install it.

  4. Verify Installation: Return to Extensions > Manage Extensions to ensure the extension was installed successfully.

Your Azure DevOps instance is now configured for QAS integration and ready for use in build and release pipelines.

Configure

...

Qualitia - Test Execution Task

Follow the steps to configure the QAS Qualitia - Test Execution task in your CI-CD pipeline within Azure DevOps:

Add Task in

...

Pipeline

  1. Log into your Azure DevOps instance.

  2. Add new CI-CD pipeline or edit an existing pipeline.

    • To create new pipeline, select required repository or use the Select classic view option.

  3. Click the + icon to add new task.

  4. Search and select QAS - Test Execution and add it.

After adding the custom task to the CI-CD the pipeline, configure it by adding the following details:

...

  1. Log into QAS - Client.

  2. Access Develop from left navigation.

  3. From the left pane, select a scenario, and open the test case.

  4. Click Link.

 To link a test case from the Ready for Execution screen:

  1. Log into QAS - Client.

  2. Access Execute from left navigation.

  3. Click the Ready for Execution tab.

  4. A list of test cases will be displayed. Select the desired test case and click Link.

Configure Test Case Linking:

  • In both methods, the same screen appears where you need to fill in:

    • ADO Project: Select the Azure DevOps project.

    • Select ADO Test Plan: Select the associated Azure DevOps test plan.

    • Select ADO Test Suite: Select the Azure DevOps test suite.

    • Optionally, use the Search box to find the Azure DevOps test by name or ID.

    • Select the test case to link and click Link.

    • The test case is now linked, and the Automation Status in Azure DevOps will be updated to Automated.

...

Viewing Status in Azure DevOps
After you link QAS test case to Azure DevOps, the Azure DevOps updates the automation status of the Azure DevOps test to Automated. 

To view the status of the test case in Azure DevOps:

  1. In Azure DevOps, select the test suite.

  2. In the Automation Status column, the status is updated to Automated.

    image-20241010-103812.pngImage Removed

  1. .

  2. From the left pane, select a scenario, and open the test case.

  3. Click Link.

 To link a test case from the Ready for Execution screen:

  1. Log into QAS - Client.

  2. Access Execute from left navigation.

  3. Click the Ready for Execution tab.

  4. A list of test cases will be displayed. Select the desired test case and click Link.

Configure Test Case Linking:

  • In both methods, the same screen appears where you need to fill in:

    • ADO Project: Select the Azure DevOps project.

    • Select ADO Test Plan: Select the associated Azure DevOps test plan.

    • Select ADO Test Suite: Select the Azure DevOps test suite.

    • Optionally, use the Search box to find the Azure DevOps test by name or ID.

    • Select the test case to link and click Link.

    • The test case is now linked, and the Automation Status in Azure DevOps will be updated to Automated.

...

Viewing Status in Azure DevOps 

Once publish any test case from Qualitia, the status of the test in Azure DevOps is marked as Automated.

...

To view the status of the test case in Azure DevOps:

...

  1. Access Execute from the left navigation.

  2. Click the Ready for Execution tab.

    image-20241010-104353.pngImage Removed

  3. A list of suites is displayed in Suite tab Suites tab

    image-20241010-104353.pngImage Added

  4. Select the suite and click link icon (image-20241014-041107.pngImage Added) under column Linked ADO Suites to open Link with ADO Suites window, as shown in the screenshot.

    image-20241010-104415.png

Working with QAS Test cases In Azure DevOps 

...