Header

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Visual Studio, a product from Microsoft, helps product teams with tools and technologies in all the stages of application development. Qualitia Automation Studio (QAS) helps replace the manual testing procedures in the application development processes by tightly integrating with Azure DevOps. This makes the application development process robust and simple. 

Qualitia's integration to Azure DevOps lets organizations leverage their existing investments as part of an integrated platform for automation testing development and allows managing automated test cases prepared by Qualitia inside Azure DevOps.


Key Benefits of QAS and Azure DevOps Integration:

  1. Seamless Synchronization: Effortlessly synchronize updated and new test cases, even when a pipeline involves several test cases.

  2. Parallel Test Execution: Supports parallel execution of test cases, reducing pipeline execution time.

  3. Result Visibility: View pipeline execution results from both Azure DevOps server and within QAS.

Features:

With Azure DevOps advanced integration, you can:

  • Map QAS Project to Azure DevOps Project.

  • Map automated QAS test case to Azure DevOps test case.

  • Execute QAS test cases locally in Azure DevOps (using self-hosted agents).

  • Run QAS test cases as part of Azure DevOps build pipelines

To integrate QAS with Azure DevOps, you need to first set up the environment.

Azure DevOps Integration

Let’s set up the environment, to get started with the Azure DevOps Integration. This section depicts information related to all the prerequisites and procedures to get prepared for QAS's integration to Azure DevOps.

This section describes how Azure DevOps interacts with QAS components to execute QAS test cases successfully.

Qualitia Components:

  • Qualitia Server

  • Qualitia Client

Azure DevOps Components:

  • Azure DevOps as the Cloud Version

  • Azure DevOps Self Hosted Agent

To integrate QAS with Azure DevOps, follow these three key steps:

Association Between Qualitia Automation Studio and Azure DevOps

This step involves initiating the association from within Qualitia Automation Studio (QAS), mapping projects and test cases between QAS and Azure DevOps.

Key Activities:

  • Map QAS Project to Azure DevOps Project: Establish a link between your project in QAS and the corresponding project in Azure DevOps.

  • Map Automated Test Cases: Link QAS automated test cases with Azure DevOps manual test cases, ensuring that test cases are synchronized across both platforms.

  • Associate Test Suites: Map Qualitia Suites to Azure DevOps Test Suites to group relevant test cases for streamlined execution.

Execution Setup in Azure DevOps (One-Time Setup)

This one-time setup ensures that QAS test cases can be executed directly from Azure DevOps pipelines.

Key Activities:

  • Setup Build/Release Pipelines: Create and configure build or release pipelines in Azure DevOps to execute the associated QAS test cases.

  • Task Extension for Custom Tasks: Install the Qualitia Task Extension, which allows custom Qualitia tasks to be included in the pipeline installation. (Task Extension is used for Qualitia Custom tasks for pipeline installation)

  • Configure Self-Hosted Agents: Set up a self-hosted Azure DevOps agent for executing the QAS test cases locally.

  • Agent Pool Setup: Create and configure an agent pool in Azure DevOps to manage agents responsible for running QAS executions.

Result Publishing and Execution

This step ensures that the results of the executed test cases are published and available both in Azure DevOps and within QAS.

Key Activities:

  • Publish Results in Azure DevOps: After each pipeline execution, results of the QAS test cases are published to Azure DevOps test cases and test runs.

  • Result Sync with QAS: Test case results are synchronized with Qualitia Automation Studio to provide a holistic view of the automation results within QAS.

The following table explains how different components of Azure DevOps and QAS interact with each other.

Component

Roles Played in Different Stages

Azure DevOps

Association

  • Responds to QAS server using standard out of the box APIs offered by Azure DevOps.

Execution

  • Configuration of build/release pipelines.

  • Agent pool configurations.

  • Initiate pipeline execution.

Result Publish

  • Host results of pipeline execution.

Azure DevOps Agent

Association

None

Execution

  • Executes the jobs in the pipeline along with the contained tasks.

  • Select Test Plan and Test Suite that contain the automated test cases and suites within the Azure DevOps projects.

Result Publish

  • Execution of the Publish Result task of QAS to publish results to QAS Server and Azure DevOps Test Runs.

  • Uses QAS Server APIs to publish result.

Qualitia Server

Association

  • Interacts with the Azure DevOps server to update automation status of Azure DevOps test cases.

Result Publish

  • Receives test case results directly from QAS - Client with the help of QAS APIs.

Qualitia Client

Association

  • Allows the authorized users to view Azure DevOps projects and associated projects and test cases.

Execution

  • Publishes the test cases to QAS project repository. The published test cases are used in the Azure DevOps pipeline execution.

Result Publish

  • Shows the status of test cases executed through Azure DevOps pipeline in Real Time Reporting Portal.

Prerequisites

Prerequisites for Azure DevOps

Azure DevOps Version

  • Azure DevOps (Cloud Version)

  • Azure DevOps Server 2019

 Qualitia Plugins

Share Azure DevOps URL to Qualitia team to get the Get Assembly task plugin.

 Roles and Privileges

  • Azure DevOps Administrator must install the plugins from Shared Extensions.

  • Qualitia does not recommend this option for PAT generation as it changes the access levels for all the users in the group.

Note: Please connect with your Azure DevOps Admin team to discuss the PAT generation and access level management.

Qualitia integration with Azure DevOps expects that the user has “Update Work Items” access that is used to generate the PAT (Personal Access Token).

The PAT with required access can be procured in one of the following ways:

  • Generate the PAT with a user that is part of the Project Administrators group.
    Qualitia recommends this option for generating a PAT.

  • Allow Bypass Rules on Work Items Updates
    Perform the following steps:

    • In Azure DevOps project, ensure that the user that is used to generate PAT is member Contributors group / Project Team members group.

    • Click Project Settings > Permissions.

    • Select the user group that has the user to whom you want to provide access (Contributors/Project Team members).

    • From the Bypass rules on work items updates list, select Allow.
      Refer to the following screenshot.

  • Have project access to create the following:

    • A new pipeline (Build/Release)

    • An Agent Pool

Prerequisites for Qualitia

Roles and Privileges

  • You must have administrative privileges to access Qualitia Project that you want to integrate with Azure DevOps tests.

 Network Access

  • Ensure that the Qualitia Server URL is reachable from the Qualitia Client, Azure DevOps Server, and Azure DevOps agent.

Configuring Azure DevOps for Test Executions

QAS provisions to execute automated test cases from QAS mapped to Azure DevOps, as a part of your build and release pipelines. In order to do so, you first need to add Qualitia - Test Execution extensions in Azure DevOps instance.

Follow the steps to add these extensions in Azure DevOps:

  • Before initiating Azure DevOps integration with Qualitia, you must share Azure DevOps organization details with Qualitia. Qualitia will then share the extensions which will help you to integrate Azure DevOps projects to Qualitia projects.

  • After Qualitia shares these extensions, you can add and install these extensions to use them in the build and release pipeline.

  • Once this task is shared by Qualitia in Azure DevOps instance, you may add, install, and use them in build and release pipelines.

 To add the Qualitia – Test Execution extensions into Azure DevOps instance:

  • Sign in to your Azure DevOps instance.

  • From the top right corner, navigate to Marketplace > Manage Extensions.
    You will find Qualitia - Test Execution extension shared by Qualitia.

  • Click the Qualitia - Test Execution and install it.

  • Navigate back to Extensions > Manage Extensions.

Your Azure DevOps instance is now ready for Qualitia integration.

🔧 Got a problem? Let’s fix!

  1. Mobile test case execution is failing on ADO: To execute mobile test case successfully on ADO, you need to provide all the mobile capabilities manually.
    Here is the list of required capabilities in ADO Pipeline.

Capability Name

Values (As per execution needs)

MobilePlatform

Android

MobileBrowser

Chrome

MobileExecutionEnvironment

Local

AppiumServerURL

http://127.0.0.1:4723/

androidAppiumServerURL

http://127.0.0.1:4723/

android.platformName

android

android.browserName

chrome

android.appium:udid

emulator-5558

android.appium:deviceName

Pixel_3a_XL_API_34

android.appium:automationName

UiAutomator2

appiumOption

Custom/Automatic

androidAppiumOption

Custom/Automatic

Follow the below steps to add capabilities:

  1. Go to ADO.

  2. Edit the pipeline.

  3. Click three dots next to Enter environment variables.

  4. Add capabilities & their values from the above table.

  5. Click OK.

  6. Save the pipeline.

    image-20240919-073926.png

  • No labels

Footer