Document toolboxDocument toolbox

Header

Supported Environments and Guidelines

Owned by Dhiraj Chandanshive (Unlicensed)
Last updated: Oct 17, 2022
22 min read

Qualitia Client and Offline Package supports test suite executions on multiple platforms and browsers for test case executions and report viewing. Along with the local machines, test suite can be executed through Qualitia Client and Offline package on remote o cloud environments.

Qualitia supports the following cloud environments:

  • Saucelabs

  • Browserstack

  • Perfecto

  • Testobject

.

Supported Environments for Test Suite Executions

Qualitia gives you the provision to execute Mobile and Web application test cases in different environments.  

Desktop Web Applications 

Mobile Applications

Remote Executions for Web Applications

 

You can execute the test cases using an offline package after configuring settings accordingly. This gives you the provision to trigger suite execution using any machine in the network. You do not need to move the offline package to different machines to execute it.


You can do it in the following two ways:

Normal Remote Execution

This option can be used when you are aware of the remote machine details where you wish to execute your test cases.  You need to choose to execute offline package on a remote machine with specific details.

Qualitia uses the selenium-standalone-server-jar for test case executions on the target remote machine.

To run selenium-stand- alone jar:

  1. Download the selenium-standalone-jar on the target machine where you want to execute test cases.

  2. Download the drivers corresponding to the offline execution browser (chromedriver, IEdriverServer, and so on) and store it in the same location where the Selenium-stand-alone-jar is saved.


  3. Open the Command Prompt window and navigate to the location where offline jar is located.

  4. Do one of the following:

    • If the execution browser is Iexpore and Selenium sever standalone version is 2.xx.x, execute following command: java -jar selenium-server-standalone-2.45.0.jar - Dwebdriver.ie.driver="D:\Data\Testing\Apps\IEDriverServer.exe”.


    • If the execution browser is Iexpore and selenium sever standalone version is 3.xx.x, execute following command: java -Dwebdriver.ie.driver="D:\Data\Testing\Apps\IEDriverServer.exe" -jar selenium-server-standalone-3.0.1.jar


Configuring the Settings for a Normal Remote Execution

Once you download Selenium-stand-alone jar and browser drivers, you need to change some settings.

Either you can create an execution profile to execute on the remote environment or edit an existing execution profile for execution on remote environment.

For detailed instructions on using execution profiles,  refer to Working with Execution Profiles.



To configure the settings for a Normal Remote Execution:

  1. Do one of the following:

    1. To execute from a Qualitia client, open the associated execution profile.

    2. To execute an offline suite, do the following in the order listed:

      1. Open the offline package folder.

      2. From the Configuration Manager folder, and double-click the Qualitia Offline Configuration Manager. 

      3. Do one of the following;

        1. Click Browse next to Select the Suite Directory, and then locate and select the offline suite.

        2. Enter the location path of the offline suite folder, and press Tab.

    3. Under the Select the profile to edit, select the execution profile created to execute on remote environment.

    4. Select the associated execution profile that is defined to execute on remote environment.

    5. From the Mapped Execution Profile list, select the execution profile you have opened.

  2. Set the ExecutionEnvironment as Remote.

  3. Set the RemoteURL value to a valid remote URL that is IP and port of remote machine.
    For example: RemoteURL=http://192.168.25.123:4445/wd/hub

  4. Click Save.

Suite Execution

Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Work with Offline Package.



  • In case you have multiple versions of java, Qualitia uses the version set against the "JAVA_HOME" system variable under System Properties settings of your computer.

  • To execute mobile test cases using Selenium grid, make sure that Appium server is up and running on the designated node.

Remote Execution through Selenium Grid

What is a Hub?

The Hub is the main command center for managing which machines your test will run on. This is where you can see list of all the available nodes currently running in your grid environment (Remote Execution Environment).

What is a Node?

A Node is a machine that you register with your Hub. By registering with Hub, the Hub now knows about the Node and any configuration information you used when you registered the node.

In order to perform remote execution through Selenium grid, you need to configure Hub and Node in the network.

Creating a Hub in the Network

A Hub can be created in the network following the steps mentioned below.

  1. Decide a machine in the network to register as hub (with IP, for example, 192.168.25.6).

  2. Download the Selenium-server-standalone jar file on Hub machine.

  3. Open Command Prompt and navigate to the location where Selenium-server-standalone-jar is saved.

  4. Execute command "java -jar selenium-server-standalone-2.xx.0.jar -role hub -port 4445".



Create Node and Register to Hub

Once a hub is created in the network, you can select multiple machines to act as its nodes. Registering nodes include the following steps.

  1. Download the Selenium-server-standalone jar file on the node machine.

  2. Launch the command prompt and navigate to the location where Selenium-server-standalone jar file is saved.

  3. Execute command "java -jar selenium-server-standalone-2.xx.x.jar -role node –hub http://hubIP:4445/grid/registerr".

You can bundle your IEServerDriver.exe, Geckodriver.exe, or ChromeDriver.exe running while registering to Hub.

Configuring the Settings for Execution through Grid

Prior to executing test cases in remote execution in the Selenium grid, you need to configure settings accordingly. Here are some important points to keep in mind when configuring settings for remote execution in the Selenium grid.

  • Selenium server jar version used in Qualitia offline package and downloaded at remote machines including Hub and Nodes must be the same all over.

  • Java needs to be installed and configured in the computer’s environment path.

  • Port number used for the hub is changeable and the default port number is 4444.

  • As hub selects a node for execution at its own at run time, you cannot provide details for specific OS / Platform.

  • To know more about how to setup Selenium grid hub and node, you recommend using the Selenium grid links given below.

Either you can create an execution profile to execute on the remote environment or edit an existing execution profile. For more information, refer to Working with Execution Profiles.

To configure the settings for executing through Selenium Grid:

  1. Do one of the following:

    • To execute the test suite from a Qualitia client, open the associated execution profile.

    • To execute the offline suite, do the following in the order listed:

      1. Open the offline package folder.

      2. From the Configuration Manager folder, and double-click the Qualitia Offline Configuration Manager. 

      3. Do one of the following;

        1. Click Browse next to Select the Suite Directory, and then locate and select the offline suite.

        2. Enter the location path of the offline suite folder, and press Tab.

      4. Under the Select the profile to edit, select the execution profile created to execute on remote environment.

      5. Select the associated execution profile that is defined to execute on remote environment.

      6. From the Mapped Execution Profile list, select the execution profile you have opened.

  2. Set the ExecutionEnvironment as Remote.

  3. Set the RemoteURL value to a valid remote URL that is IP and port of remote machine.
    For example: RemoteURL=http://192.168.25.123:4445/wd/hub

  4. Set the GridPlatform value to a valid platform where you want to execute test cases. For example: Set the GridPlatform=Win10.

  5. Click Save.

Suite Execution

Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Work with Offline Package.



  • In case you have multiple versions of java, Qualitia uses the version set against the "JAVA_HOME" system variable under System Properties settings of your computer.

  • To execute mobile test cases using Selenium grid, make sure that Appium server is up and running on the designated node.



 

Remote Executions for Mobile Applications on Selenium Grid

What is Selenium Grid?

Selenium Grid is a part of the Selenium Suite that is specialized to run multiple tests across different browsers, operating systems, and machines simultaneously.

Following are some of the salient features of Selenium Grid:

  • Execute test script on multiple machines at the same time.

  • Execute tests on different machines with different browsers

  • Execute tests on the same browser but with different versions

  • Save test execution time and effort

  • Has a Hub-node architecture, where Hub works as a central point that controls the nodes, and the nodes are the actual machines on which tests are executed.

How to work with Selenium Grid using Qualitia Offline Package?

In Selenium Grid, the hub is a computer, which is the central point where the tests from the Qualitia offline package are loaded. The Hub acts as the central server that controls the network of Test machines, also referred as Nodes. One Selenium Grid has only one Hub, which is the master of the network that controls all the nodes.
When a test with selected Desired Capabilities options is passed to the Hub, the Hub searches for the node that matches with options. For example, if you have defined the Desired Capabilities options to run a test on Windows 10 operating system using Chrome version XXX, the Hub will try to find a machine in the Grid that matches the criteria and runs the test on it. The Hub shows an error if no machine matches the criteria. A node, which is referred to as a Test Machine, opts to connect with the Hub. The Hub runs the tests on the test machines. A Grid network can have multiple nodes. Each node can have different platforms, that is operating systems and browsers. It is not necessary for the node to have the same platform as that of the Hub.

To set up a Grid network, first you must create a Hub and then connect (register) nodes to the Hub.

Building a Selenium Grid for Qualitia Mobile Testing


Mobile Test Automation has gained significant importance in recent times. Almost all the web applications are highly active these days and it is important to test how the applications work across devices. This is true for mobile native applications as well. At the same time, a huge number of devices and custom operating system versions have been introduced. This has made running tests manually over multiple devices difficult, even if the tests are automated.
This section guides on how you can use Appium along with Selenium Grid for Qualitia test case execution on multiple mobile devices.

Configuring the Hub


Perform the following steps to configure the hub:

  1. Download the Selenium Standalone Server from http://docs.seleniumhq.org/download/ Start the command prompt and navigate to the folder where you placed the Selenium server jar file.

  2. Execute the following command: java –jar selenium-server-standalone-X.XX.jar –role hub, where selenium-server-standalone-3.6.0.jar is the name of the downloaded .jar file


This starts the selenium grid server in your local ip with the default port of 4444. After you see this message, the Selenium Grid hub is up and running.
Open your browser and hit the following URL: http://<your ip>:4444/grid/console. You can now navigate to Grid Console:

Configuring the Appium Nodes

In this example, we have used two devices, Samsung S7 and Samsung S5 mini. For each of these devices, we will configure a separate grid node. To set up the grid nodes, create a separate JSON configuration file for both the devices as shown below, and upload them to the Appium server instances.

Node1.json (Android 1)

{
"capabilities":
[
{
"deviceName": "Android 1",
"platformVersion": "8.1",
"maxInstances": 1,
"platformName": "Android",
"browserName": "GoogleChrome"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"url":"http://10.81.234.7:4723/wd/hub",
"host": "10.81.234.7",
"port": 4723,
"maxSession": 2,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "192.168.25.218",
"hubProtocol": "http"
}
}

Node2.json (Android 2)

{
"capabilities":
[
{
"deviceName": "Android 2",
"platformVersion": "8.1",
"maxInstances": 1,
"platformName": "Android",
"browserName": "GoogleChrome"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"url":"http://10.81.234.7:4723/wd/hub",
"host": "10.81.234.7",
"port": 4723,
"maxSession": 2,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "192.168.25.218",
"hubProtocol": "http"
}
}

To run the test cases on the same machine with two devices and on two different Appium instances. the Appium Server port must be different for lauchned two Appium instances.

Executing a Test Case on a Specific Node

The node can be equipped with different capabilities and values. Let us consider a scenario where X machine is connected to x1 mobile device, Y machine is connected to y1 mobile device, and Z machine is connected to z1 mobile device.

To execute a test case :

(The following is an example to execute a test case on x1 device)

  1. Find out the UDID of the x1 device.

  2. While registering X machine as a node to the hub, add the following capability:

    1.  --udid UDID_of_x1_device
      As a result, the node command becomes

      appium -p 4724 --udid 5e6453d7d240 --nodeconfig=C:\Users\Desktop\capabilities.json

  3. Click Settings> System Execution Profile> Mobile, and enter the UDID of x1 device in the mapped Execution Profile.

Configuring the Mapped Execution Profile

Either you can create an execution profile to execute on the remote environment or edit an existing execution profile. For more information, refer to Working with Execution Profiles.
Before you execute a test suite, ensure that all the nodes in the network meets the minimum system requirements. For more information, refer to Requirements and Configuration Settings for Mobile Environments.

To configure the settings for executing through Selenium Grid:

  1. Do one of the following:

    • To execute the test suite from a Qualitia client, open the associated execution profile.

    • To execute the offline suite, do the following in the order listed:

      1. Open the offline package folder.

      2. From the Configuration Manager folder, and double-click the Qualitia Offline Configuration Manager. 

      3. Do one of the following;

        1. Click Browse next to Select the Suite Directory, and then locate and select the offline suite.

        2. Enter the location path of the offline suite folder, and press Tab.

      4. Under the Select the profile to edit, select the execution profile created to execute on remote environment.

      5. Select the associated execution profile that is defined to execute on remote environment.
        From the Mapped Execution Profile list, select the execution profile you have opened.

  2. Configure the settings in the associated execution profile. For more information, refer to Requirements and Configuration Settings for Mobile Environments.

  3. Set the ExecutionEnvironment as Remote.

  4. Set the RemoteURL value to a valid remote URL that is IP and port of the remote machine.

    For example: RemoteURL=http://192.168.25.123:4445

  5. Set the GridPlatform value to a valid platform where you want to execute test cases. For example: Set the GridPlatform=Win10.

  6. Click Save.

  • In case you have multiple versions of java, Qualitia uses the version set against the "JAVA_HOME" system variable under System Properties settings of your computer.

  • To execute mobile test cases using Selenium grid, make sure that Appium server is up and running on the designated node.

Suite Execution

Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Offline Package.



 

Execution on BrowserStack

Introduction

BrowserStack is a cloud environment that allows running tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment, all tests are executed on BrowserStack VM. You can view the ongoing tests execution on BrowserStack.

For more information about BrowserStack, click here.

Before you start offline execution on BrowserStack, ensure you have:

  • An active account with valid Username/Access Key and

  • Sufficient balance of execution hours



Mandatory Settings in Qualitia 

You must create execution profiles that are solely used for execution on Browserstack platform from Qualitia Client or offline package. In those execution profiles, you must define the settings that are mandatory for test suite execution.

Mandatory Settings in the Associated Execution Profile   

In the associated execution profile, do the following in the order listed:

  1. From the Environment list, select Cloud.

  2. From the Cloud Platform list, select BrowserStack.

  3. Select the Edit Desired Capabilities checkbox.

  4. Select or enter the following values:

  • Url

  • username 

  • accesskey

Additional Recommendations:

On the Execution Parameters tab, select ScreenCaptureMode as Browser.

Viewing Current Execution on BrowserStack

Once you have triggered the offline execution, you can view the status of your ongoing executions on BrowserStack. BrowserStack considers each Test case as a new project. The status is updated after each project is completed with its status against it. Also, you can filter the projects by their status (Completed, Timeout, or Errors) or you can search it by build name or session name.

You can record your test case execution configuring the browserstack.properties file accordingly. If you set the recordVideo value as true and all the test case executions will be recorded. You can download these videos once the test execution is completed.

Execution Behind Firewall 

Sometimes applications are configured to be accessed within private networks. Such applications deny access when they are being accessed from other than local networks and so from BrowserStack. Executing test cases on BrowserStack for such applications needs special configurations.

When executing test case for applications running behind firewall on Windows, you need to execute the BrowserStackLocal.exe file on the machine in the private network to allow BrowserStack to execute tests flawlessly.

To execute test cases for applications running behind firewall on BrowserStack (Windows):

  1. Make sure you have set the browserstack.local value as true in the Edit Additional capabilities.

  2. Download the BrowserStackLocal.exe on any machine in the network using the following url:
    https://www.browserstack.com/automate/python#setting-local-tunnel

  3. Unzip the folder.

  4. Open Command Prompt and navigate to the BrowserStackLocal.exe file followed by BrowserStack Access key.
    For example:
    <Path where the file is saved> BrowserStackLocal.exe --key <BrowserStack access key>

  5. Press Enter.

Once the BrowserStackLocal.exe file is executed successfully, you can execute your Offline tests on BrowserStack environment using the same method mentioned above. When executing test case for applications running behind firewall on Mac, you need to execute the BrowserStackLocal Unix Executable file on the machine in the private network to allow BrowserStack executing tests flawlessly.

To execute test cases for applications running behind firewall on BrowserStack (Mac):

  1. Make sure you have set the browserstack.local value as true in the Edit Additional Capabilities.

  2. Download the BrowserStackLocal Unix Executable file on any machine in the network using the following url:
    https://www.browserstack.com/automate/python#setting-local-tunnel

  3. Unzip the folder.

  4. Open Terminal and navigate to the BrowserStackLocal Unix Executable file followed by BrowserStack Access key.
    For example:
    ./BrowserStackLocal --key <BrowserStack access key>

  5. Press Enter.

Once the BrowserStackLocal Unix Executable file is executed successfully, you can execute your Offline tests on BrowserStack environment using the same method mentioned above.

Execution Behind Proxy

Qualitia users can execute the test cases behind a proxy.

In order to execute the test cases behind a proxy, you must configure Proxy Properties in the Edit Additional Capabilities.

Qualitia users can define these properties using the command line as well.

Execution using Browser Capabilities

Selenium allows you setting desired browser specific properties to be used when executing test cases. These properties can be set for offline execution as well. You can provide these browser specific details through browser-capability files. For each supported browser (Firefox, IE, Google Chrome, and Safari) you need to configure the settings in the Edit Additional Capabilities.

Each property file above is a set of all the browser specific properties that Selenium allows you change. If you do not change any property, Selenium uses its default values.

Qualitia gives you provision to set these values for Selenium. Selenium has its own control on what value to use and how to reflect them in Execution Browser. For more information about Selenium Capabilities, refer following links:






 

Execution on Sauce Labs

Introduction

Sauce Labs is a cloud environment which allows users to run tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment all tests are executed on Sauce Labs VM. You can view the ongoing suite execution on Sauce Lab's Dashboard.

To know more about Sauce Labs click here.

Also, before uploading an iOS native or hybrid application, ensure that you build the application by following the below Saucelabs guidelines:

Prior to starting offline execution on Sauce Labs, ensure that you have:

  • An active account with valid user name / password 

  • Sufficient balance of execution hours.



Mandatory Settings in Qualitia

You must create execution profiles that are solely used for execution on Saucelabs platform from Qualitia Client or offline package. In those execution profiles, you must define the settings that are mandatory for test suite execution.

Mandatory Settings in the Associated Execution Profile

Do the following in the order listed:

  1. From the Environment list, select as Cloud.

  2. From the Cloud Platform list, select Sauce.

  3. Select the Edit Desired Capabilities checkbox

  4. Select or enter the following values: 

  • Url

  • username 

  • accesskey

  • For testing on a desktop web application, select usew3c as True.

  • For testing on a mobile application,

    • Enter the Platform Version only and do not edit the default value for the Native Object sync time. 

    • Do not add the same capabilities that are already specified in Sauce labs Mobile section, such as deviceName.

    • For iOS mobile application, ignore value present in Appium Server URL as it is mandatory field. It is not used in the execution.

  • For testing on a mobile hybrid application, select nativeWebScreenshot as True.

  • For testing on an iOS native or hybrid application, perform the following actions:

    • From the automationName drop-down, select XCUITest.

    • Enter the latest Appium version number as the appiumVersion.

Additional Recommendations:

In the Execution Parameters tab, select ScreenCaptureMode as Browser.

Viewing Current Execution on Sauce Lab's Dashboard

Once you have triggered the offline execution, you can view the status of your ongoing executions on the Sauce Lab's Dashboard. Each test case appears on dashboard as a single job with execution status against it. All the test cases are taken one by one by the sequence they are arranged in Suite. The status is updated after each test execution is completed on Sauce Lab environment with its status (fail / pass) against it. You can view the execution of test case clicking the current job on Dashboard.

You can record your test case execution configuring the sauce.properties file accordingly.

Set the recordVideo value as true and all the test case executions will be recorded.

You can download these videos once the test execution is completed from the Metadata tab. The video contains the execution of all the test cases available in Suite as the value set against QualitiaExecMode in the associated execution profile is PS (refers to per suite).

Execution Reports

Test execution reports are launched on the machine from where the execution was triggered after the test case execution is completed. You can also download the video of test case execution from the Metadata tab.

Execution of Tests Running Behind Firewall (on Private Networks)

Few applications are configured to be accessed using private networks. Such applications deny access when they are being accessed from other than local networks and so from Sauce Lab. Executing test cases on Sauce for such applications needs special configurations.

You need to execute the sc.exe file some machine in the private network to allow Sauce executing tests flawlessly.

Executing this sc.exe will create a tunnel allowing sauce to execute test cases in the private network. 

To execute applications of private network on Sauce:

  1. Open the associated execution profile.

  2. Under Edit Additional Capabilities, set the desired value for slw.tunnelIdentifier.
    The value set here for this key will be required when executing sc.exe in step 6. For more information on configuring this property, refer Sauce Labs documentation.

  3. Download the Sauce Connect Proxy on any machine in the network using following url: 
    https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy

  4. Extract the folder at the suitable location.
    You will find the sc.exe file in the bin folder.

  5. Open Command Prompt and navigate to the folder where sc.exe file exists. 

  6. Enter the following command.

    sc -u <sauce_username> -k <accesskey> -i <tunnelID> For example: sc -u john.doe -k 8becc95c-4559-4582-a617-7d38544797a2 -i tunnel-1232



  7. Press Enter.

Once the SC.exe file is executed successfully, you can execute your offline tests on Sauce environment using the same method mentioned above.

Execution using Browser Capabilities

You can also execute test cases on remote environment using selenium supported Browser Capabilities.

For more information on this, please refer to the Execution using Browser Capabilities section.

Executing Test Cases on iOS Devices

If you are executing test cases on iOS mobile devices using Sauce Labs, you need to configure settings in the iosdc.properties file accordingly.

  • If iOS device is 9.3.x and below, automationname should be Appium or

  • If iOS device is 10.0.x and above, automationname should be XCUITest.

Execution using W3C Protocols

W3C-compliant Selenium capabilities and protocol is supported for test case executions in the Sauce Labs environment with selenium v3.11.0 and above.

This W3C protocol is supported while executing test cases using following browser versions:

  • Internet Explorer v11 and above,

  • Mozilla Firefox v53 and above, and

  • Google Chrome v 61 and above.

Here is the list of capabilities which you need to modify when you want to execute test cases using W3C protocol. 

slw.useW3C=True browser=Firefox, Chrome, or Internet Explorer) version=(for firefox, 53 and above, for chrome 61 and above, for IE 11 and above) seleniumVersion=3.11.0

For more information about executing test cases using W3C-Protocols and Capabilities, refer to the Sauce Labs documentation. 

 

Execution on Sauce Labs Real Device-TestObject

Introduction

The Sauce Labs Real Device-TestObject provides you the ability to run Qualitia test cases across Android and iOS physical devices. With its Web interface, you can run tests using different combinations of platforms, platform versions, and browsers. In this environment all Qualitia mobile application tests are executed on physical devices. Sauce Labs allows you viewing video recordings of the test case executions.

To know more about Sauce Labs click here.

Prior starting offline execution on Sauce Labs, ensure you have;

  • An active account with valid user name / password and

  • Sufficient balance of execution hours.



Mandatory Settings and Execution Guidelines 

Mandatory Execution Guidelines:
You must upload your mobile application under test in TestObject. which generates a unique app_path for your mobile app.
For more information, refer to https://wiki.saucelabs.com/display/DOCS/Uploading+and+Accessing+Applications+with+Real+Devices

To upload a mobile application in TestObject and get the unique app_path:

  1. Open https://app.saucelabs.com/live/app-testing.

  2. Click App Upload > Choose File to upload the mobile application from your computer.
    The application name appears.

  3. Mouseover the application name.

  4. Click Settings.

  5. Copy the File_id that is generated.

  6. Form an app path in the format,as storage:File_id.

  7. Enter the value of app_path in the first parameter for all the Mobile.OpenApp actions present in different test cases, which are planned for execution on TestObject.  

Settings in the Associated Execution Profile:

You must create execution profiles that are solely used for execution on the Browserstack platform from the Qualitia Client. In those execution profiles, you must define the below settings that are mandatory for test suite execution.

Do the following in the order listed:

  1. From the Environment list, select as Cloud.

  2. From the Cloud Platform list, select TestObject.

  3. Select the Edit Desired Capabilities checkbox.

  4. Select or enter the following values:

  • URL

  • Username

  • Accesskey

  • Devicename

  • testobject_api_key

  • test_app_id

  • For testing on a native app, select nativeWebScreenshot as True.

  • For testing on an iOS application, select an Appium Version.

Additional Recommendations:

  • In the Execution Parameters tab, select ScreenCaptureMode as Browser.



Creating a Real Device Project

In order to automate testing on Sauce Labs Real Device, you must create a project on TestObject. When creating a project, you provide information about the type of application to test (Web or Mobile app) and the device settings you want to use in your tests. After creating a project, you will get a testobject_api_key (for example: 1234567890123456789012345678901234567890). You will have to add this key in testobject.properties against to.testobject_api_key property.

For more information about creating the project for Sauce Labs Real Device testing, refer to the Sauce Labs documentation.







Viewing Test Case Execution and Reports 

Once you trigger the offline package execution, you can view the status of your ongoing executions in Appium Test Results section. Each Test or Suite appears as a single grid with details like device, time, and duration of the test execution against it. Click the test case name from the list to view detailed logs, video recordings, and so forth of the test case.

To view the execution report of Qualitia test cases:

  1. Log on to your Sauce Labs real device instance.

  2. From the top, click

     > Appium.

  3. In the Appium Test Results section, click the test case from the list.
    Here you can view the Test case Info, Screenshots, Video, Requests, Device Logs and Appium Logs.



 

Executions on Perfecto

Introduction

Perfecto is a cloud-based lab for Web and Mobile application testing. It allows running tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment, all desktop web application tests are executed on virtual machines (VMs) and mobile application tests are executed on physical devices. Perfecto allows you viewing video recordings of the test case executions after the suite executions are completed.

To know more about Perfecto, click here.

Prerequisite:

  • An active user account on Perfecto.



Mandatory Settings and Execution Guidelines in Qualitia

The mobile native and hybrid application file, that is the apk, ipa, and app file, is not uploaded on Perfecto automatically during the Qualitia Automation Studio executions.

Qualitia recommends that you perform the following steps to upload the native and hybrid mobile application file:

  1. You must upload the mobile native or hybrid applications on Perfecto Cloud before executing test suites.
    To upload a mobile app, login to Perfecto website, click Assets and Setup>Upload.

  2. After uploading the mobile app, copy the Perfecto lab repository path for your mobile app.



  3. Enter the Perfecto lab repository path in the first parameter of the Mobile.OpenApp action.

Mandatory Settings in Associated Execution Profile

In the associated execution profile, do the following in the order listed:

  1. From the Environment list, select as Cloud.

  2. From the Cloud Platform list, select Perfecto.

  3. Select the Edit Additional Capabilities checkbox.

  4. Select or enter the following values:

    1. Enter the following as URL
      https://testingcloud.perfectomobile.com/nexperience/perfectomobile/wd/hub 

    2. Either enter Username and Password, or SecurityToken.

  5. Perform the following steps to execute the test suite on a mobile application,

    1. Select enableAppiumBehavior as True.

    2. To execute test suite on a mobile web application:

      1. Select enableAppiumBehavior and useAppiumForWeb as Tru

      2. Select AutomationName for Android as Appium and for iOS as XCUITest.

      3. Enter SeleniumVersion to 4.0.0.

      4. Enter the chrome driver version for the target mobile device, in ChromeDriverVersion to 2.xx, where xx denotes a chrome driver version, such as 91 and 92. 

    3. To execute test suite on an Android hybrid application:

      1. Select enableAppiumBehavior and useAppiumForHybrid as True.

      2. Enter SeleniumVersion to 4.0.0.

      3. Enter the chrome driver version for the target mobile device, in ChromeDriverVersion to 2.xx, where xx denotes a chrome driver version, such as 91 and 92. 

    4. To execute test suite on an iOS hybrid application:

      • Select enableAppiumBehavior, useAppiumForHybrid,and iOSResign as True.

Additional Recommendations:

In the Execution Parameters tab, select ScreenCaptureMode as Browser.





Viewing Execution Status and Reports in Perfecto's Execution Center

Once you have triggered the offline execution, you can view the status of your ongoing executions in Perfecto's Execution Center. Each test case appears on dashboard as a single grid with execution status against it. The status is updated after each test execution is completed in Perfecto's environment with its status (FAILED / SUCCESS) against it. Select the test case from the list and click Report from top to view recording of your test case.



To view the execution report of Qualitia test cases:

  1. Log on to your Perfecto account. 

  2. From the top right corner, click More > Execution Center.

  3. In the Executions section, select the test case from the list.

  4. Click Report

Executing Test Cases Behind Firewall Applications

Few applications are configured to be accessed using private networks. Such applications deny access when they are being accessed from other than local networks and so from Perfecto. Executing test cases on Perfecto for such applications needs special configurations.

Simply download Perfecto Connect client executable that matches your Operating System. You need to execute it on the machine in the private network to allow Perfecto executing tests flawlessly.

For more information on configuration of Perfecto connect, refer to the Perfecto documentation.

To configure perfecto to run test cases for the applications behind firewall:

  1. Download Perfecto Connect executable depending on your OS.

  2. Unzip the folder and run the file using command line interface along with Cloud URL and Security Token. 
    For example: 

    perfectoconnect.exe -c <Perfecto_Cloud_URL> -s <Security_token>

  3. Executing this file will give you the tunnel ID.

  4. Under Edit Additional capabilities, enter the tunnel ID in tunnelId. 

For more information about this, refer to the Perfecto documentation.

 

Limitations of Remote/Cloud Executions

 

When using Qualitia Offline Package on the Remote and Cloud environments, following are the limitations users face when executing test cases.

Lack of Support for Java Robot Based Actions:

Cloud does not support Java robot based actions when executing test cases. Hence following actions do not get executed in Cloud and Remote environment;

  • DownloadFile

  • WindowsAuthentication

  • PressKeys

Lack of support for java robot based actions on Cloud / Remote environment fails the test case executions.


In order to overcome this problem, you need to validate the test cases that have java based robot actions as mentioned above and;

  • either execute them in local environments or

  • create their AutoIt scripts to be executed in Cloud environment.

Lack of Support for Headless Mode:

Cloud execution environment does not support the Headless mode.

For parallel execution in Cloud, you can execute tests in multiple VMs. A separate VM is allocated to execute each offline package, so you can run multiple offline packages to get multiple VMs.

User Defined Firefox Profile:

Executing test cases in Cloud environment using user defined Firefox profile is experimental.

Support for AutoIT in Cloud Execution Environment

In order to achieve execution of Windows based actions such as DownloadFile, PressKeys, WindowsAuthentication, and so on, Qualitia supports Sauce's "Pre-Run" feature. It allows uploading AutoIT.exe (or any exe) and running before starting test case executions. Running .exe performs the windows based actions flawlessly.

In order to achieve this, you need to set the following properties in the Edit Additional capabilities in the associated execution profile.

  • custom_use_prerun: This option decides whether to use Pre-Run feature or not. Set it as true to run the Pre-Run feature.

  • custom_exe_name: This contains the exe file name to be executed in Sauce environment. To run exe in Sauce, you must upload the file to Sauce network.

  • args: It is the list of command line parameters that you want to execute.

  • custom_use_upload: This option decides whether to upload file in Sauce network or not. Set it as true to use upload related properties.

  • custom_file_path: Set this to the valid path where the file is located to be uploaded in the Sauce network.

  • Same way set all other properties for pre-run and uploading script file.

  • For more information about the Pre-Run feature operation, please refer following links,




 

Footer