...
| Table of Contents | ||
|---|---|---|
|
Supported Environments for Test Suite Executions
Qualitia gives you the provision to execute Mobile and Web application test cases in different environments.
Desktop Web Applications
| Multiexcerpt include macro | ||||
|---|---|---|---|---|
|
Mobile Applications
| Multiexcerpt include macro | ||||
|---|---|---|---|---|
|
Remote Executions for Web Applications
You can execute the test cases using an offline package after configuring settings accordingly. This gives you the provision to trigger suite execution using any machine in the network. You do not need to move the offline package to different machines to execute it.
...
| Table of Contents | ||
|---|---|---|
|
Normal Remote Execution
This option can be used when you are aware of the remote machine details where you wish to execute your test cases. You need to choose to execute offline package on a remote machine with specific details.
...
- Download the selenium-standalone-jar on the target machine where you want to execute test cases.
- Download the drivers corresponding to the offline execution browser (chromedriver, IEdriverServer, and so on) and store it in the same location where the Selenium-stand-alone-jar is saved.
- Open the Command Prompt window and navigate to the location where offline jar is located.
- Do one of the following:
- If the execution browser is Iexpore and Selenium sever standalone version is 2.xx.x, execute following command: java -jar selenium-server-standalone-2.45.0.jar - Dwebdriver.ie.driver="D:\Data\Testing\Apps\IEDriverServer.exe”.
- If the execution browser is Iexpore and selenium sever standalone version is 3.xx.x, execute following command: java -Dwebdriver.ie.driver="D:\Data\Testing\Apps\IEDriverServer.exe" -jar selenium-server-standalone-3.0.1.jar
- If the execution browser is Iexpore and Selenium sever standalone version is 2.xx.x, execute following command: java -jar selenium-server-standalone-2.45.0.jar - Dwebdriver.ie.driver="D:\Data\Testing\Apps\IEDriverServer.exe”.
Configuring the Settings for a Normal Remote Execution
Once you download Selenium-stand-alone jar and browser drivers, you need to change some settings.
...
- Do one of the following:
- To execute from a Qualitia client, open the associated execution profile.
- To execute an offline suite, do the following in the order listed:
- Open the offline package folder.
- From the Configuration Manager folder, and double-click the Qualitia Offline Configuration Manager.
- Do one of the following;
Click Browse next to Select the Suite Directory, and then locate and select the offline suite.
Enter the location path of the offline suite folder, and press Tab.
- Under the Select the profile to edit, select the execution profile created to execute on remote environment.
- Select the associated execution profile that is defined to execute on remote environment.
- From the Mapped Execution Profile list, select the execution profile you have opened.
- Set the ExecutionEnvironment as Remote.
- Set the RemoteURL value to a valid remote URL that is IP and port of remote machine.
For example: RemoteURL=http://192.168.25.123:4445/wd/hub - Click Save.
Suite Execution
Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Work with Offline Package.
...
| Info |
|---|
|
Remote Execution through Selenium Grid
What is a Hub?
The Hub is the main command center for managing which machines your test will run on. This is where you can see list of all the available nodes currently running in your grid environment (Remote Execution Environment).
What is a Node?
A Node is a machine that you register with your Hub. By registering with Hub, the Hub now knows about the Node and any configuration information you used when you registered the node.
In order to perform remote execution through Selenium grid, you need to configure Hub and Node in the network.
Creating a Hub in the Network
A Hub can be created in the network following the steps mentioned below.
- Decide a machine in the network to register as hub (with IP, for example, 192.168.25.6).
- Download the Selenium-server-standalone jar file on Hub machine.
- Open Command Prompt and navigate to the location where Selenium-server-standalone-jar is saved.
- Execute command "java -jar selenium-server-standalone-2.xx.0.jar -role hub -port 4445".
Create Node and Register to Hub
Once a hub is created in the network, you can select multiple machines to act as its nodes. Registering nodes include the following steps.
...
| Note |
|---|
You can bundle your IEServerDriver.exe, Geckodriver.exe, or ChromeDriver.exe running while registering to Hub. |
Configuring the Settings for Execution through Grid
Prior to executing test cases in remote execution in the Selenium grid, you need to configure settings accordingly. Here are some important points to keep in mind when configuring settings for remote execution in the Selenium grid.
...
- Do one of the following:
- To execute the test suite from a Qualitia client, open the associated execution profile.
- To execute the offline suite, do the following in the order listed:
- Open the offline package folder.
- From the Configuration Manager folder, and double-click the Qualitia Offline Configuration Manager.
- Do one of the following;
Click Browse next to Select the Suite Directory, and then locate and select the offline suite.
Enter the location path of the offline suite folder, and press Tab.
- Under the Select the profile to edit, select the execution profile created to execute on remote environment.
- Select the associated execution profile that is defined to execute on remote environment.
- From the Mapped Execution Profile list, select the execution profile you have opened.
- Set the ExecutionEnvironment as Remote.
- Set the RemoteURL value to a valid remote URL that is IP and port of remote machine.
For example: RemoteURL=http://192.168.25.123:4445/wd/hub - Set the GridPlatform value to a valid platform where you want to execute test cases. For example: Set the GridPlatform=Win10.
- Click Save.
Suite Execution
Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Work with Offline Package.
...
| Info |
|---|
|
Remote Executions for Mobile Applications on Selenium Grid
| Table of Contents |
|---|
What is Selenium Grid?
Selenium Grid is a part of the Selenium Suite that is specialized to run multiple tests across different browsers, operating systems, and machines simultaneously.
...
Execute test script on multiple machines at the same time.
Execute tests on different machines with different browsers
Execute tests on the same browser but with different versions
Save test execution time and effort
Has a Hub-node architecture, where Hub works as a central point that controls the nodes, and the nodes are the actual machines on which tests are executed.
How to work with Selenium Grid using Qualitia Offline Package?
In Selenium Grid, the hub is a computer, which is the central point where the tests from the Qualitia offline package are loaded. The Hub acts as the central server that controls the network of Test machines, also referred as Nodes. One Selenium Grid has only one Hub, which is the master of the network that controls all the nodes.
When a test with selected Desired Capabilities options is passed to the Hub, the Hub searches for the node that matches with options. For example, if you have defined the Desired Capabilities options to run a test on Windows 10 operating system using Chrome version XXX, the Hub will try to find a machine in the Grid that matches the criteria and runs the test on it. The Hub shows an error if no machine matches the criteria. A node, which is referred to as a Test Machine, opts to connect with the Hub. The Hub runs the tests on the test machines. A Grid network can have multiple nodes. Each node can have different platforms, that is operating systems and browsers. It is not necessary for the node to have the same platform as that of the Hub.
To set up a Grid network, first you must create a Hub and then connect (register) nodes to the Hub.
Building a Selenium Grid for Qualitia Mobile Testing
Mobile Test Automation has gained significant importance in recent times. Almost all the web applications are highly active these days and it is important to test how the applications work across devices. This is true for mobile native applications as well. At the same time, a huge number of devices and custom operating system versions have been introduced. This has made running tests manually over multiple devices difficult, even if the tests are automated.
This section guides on how you can use Appium along with Selenium Grid for Qualitia test case execution on multiple mobile devices.
Configuring the Hub
Perform the following steps to configure the hub:
...
This starts the selenium grid server in your local ip with the default port of 4444. After you see this message, the Selenium Grid hub is up and running.
Open your browser and hit the following URL: http://<your ip>:4444/grid/console. You can now navigate to Grid Console:
Configuring the Appium Nodes
In this example, we have used two devices, Samsung S7 and Samsung S5 mini. For each of these devices, we will configure a separate grid node. To set up the grid nodes, create a separate JSON configuration file for both the devices as shown below, and upload them to the Appium server instances.
...
To run the test cases on the same machine with two devices and on two different Appium instances. the Appium Server port must be different for lauchned two Appium instances.
Executing a Test Case on a Specific Node
The node can be equipped with different capabilities and values. Let us consider a scenario where X machine is connected to x1 mobile device, Y machine is connected to y1 mobile device, and Z machine is connected to z1 mobile device.
...
Find out the UDID of the x1 device.
While registering X machine as a node to the hub, add the following capability:
--udid UDID_of_x1_device
As a result, the node command becomesappium -p 4724 --udid 5e6453d7d240 --nodeconfig=C:\Users\Desktop\capabilities.json
Click Settings> System Execution Profile> Mobile, and enter the UDID of x1 device in the mapped Execution Profile.
Configuring the Mapped Execution Profile
Either you can create an execution profile to execute on the remote environment or edit an existing execution profile. For more information, refer to Working with Execution Profiles.
Before you execute a test suite, ensure that all the nodes in the network meets the minimum system requirements. For more information, refer to Requirements and Configuration Settings for Mobile Environments.
...
|
Suite Execution
Execute the test suite or offline suite you have configured. To know how to execute an offline package, refer to Offline Package.
| Info |
|---|
If you want to use PowerShell Utility on Selenium Mobile Grid to perform parallel execution, refer to Parallel Execution using Power Shell Utility . |
Execution on BrowserStack
Introduction
BrowserStack is a cloud environment that allows running tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment, all tests are executed on BrowserStack VM. You can view the ongoing tests execution on BrowserStack.
...
- An active account with valid Username/Access Key and
- Sufficient balance of execution hours
...
| Info |
|---|
BrowserStack does not support all Selenium versions. Qualitia users can use the browserstack.selenium_version capability to specify the version they want to use when executing test cases. |
Mandatory Settings in Qualitia
You must create execution profiles that are solely used for execution on Browserstack platform from Qualitia Client or offline package. In those execution profiles, you must define the settings that are mandatory for test suite execution.
...
On the Execution Parameters tab, select ScreenCaptureMode as Browser.
Viewing Current Execution on BrowserStack
Once you have triggered the offline execution, you can view the status of your ongoing executions on BrowserStack. BrowserStack considers each Test case as a new project. The status is updated after each project is completed with its status against it. Also, you can filter the projects by their status (Completed, Timeout, or Errors) or you can search it by build name or session name.
You can record your test case execution configuring the browserstack.properties file accordingly. If you set the recordVideo value as true and all the test case executions will be recorded. You can download these videos once the test execution is completed.
Execution Behind Firewall
Sometimes applications are configured to be accessed within private networks. Such applications deny access when they are being accessed from other than local networks and so from BrowserStack. Executing test cases on BrowserStack for such applications needs special configurations.
...
Once the BrowserStackLocal Unix Executable file is executed successfully, you can execute your Offline tests on BrowserStack environment using the same method mentioned above.
Execution Behind Proxy
Qualitia users can execute the test cases behind a proxy.
...
Qualitia users can define these properties using the command line as well.
Execution using Browser Capabilities
Selenium allows you setting desired browser specific properties to be used when executing test cases. These properties can be set for offline execution as well. You can provide these browser specific details through browser-capability files. For each supported browser (Firefox, IE, Google Chrome, and Safari) you need to configure the settings in the Edit Additional Capabilities.
...
Qualitia gives you provision to set these values for Selenium. Selenium has its own control on what value to use and how to reflect them in Execution Browser. For more information about Selenium Capabilities, refer following links:
- https://code.google.com/p/selenium/wiki/DesiredCapabilities
- https://sites.google.com/a/chromium.org/chromedriver/capabilities
Execution on Sauce Labs
Introduction
Sauce Labs is a cloud environment which allows users to run tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment all tests are executed on Sauce Labs VM. You can view the ongoing suite execution on Sauce Lab's Dashboard.
...
- An active account with valid user name / password
- Sufficient balance of execution hours.
| Info |
|---|
After logging to Sauce Labs, click Data Center located at the top-right corner, and select the Time Zone to USE West 1. |
...
Mandatory Settings in Qualitia
You must create execution profiles that are solely used for execution on Saucelabs platform from Qualitia Client or offline package. In those execution profiles, you must define the settings that are mandatory for test suite execution.
...
In the Execution Parameters tab, select ScreenCaptureMode as Browser.
Viewing Current Execution on Sauce Lab's Dashboard
Once you have triggered the offline execution, you can view the status of your ongoing executions on the Sauce Lab's Dashboard. Each test case appears on dashboard as a single job with execution status against it. All the test cases are taken one by one by the sequence they are arranged in Suite. The status is updated after each test execution is completed on Sauce Lab environment with its status (fail / pass) against it. You can view the execution of test case clicking the current job on Dashboard.
...
You can download these videos once the test execution is completed from the Metadata tab. The video contains the execution of all the test cases available in Suite as the value set against QualitiaExecMode in the associated execution profile is PS (refers to per suite).
Execution Reports
Test execution reports are launched on the machine from where the execution was triggered after the test case execution is completed. You can also download the video of test case execution from the Metadata tab.
Execution of Tests Running Behind Firewall (on Private Networks)
Few applications are configured to be accessed using private networks. Such applications deny access when they are being accessed from other than local networks and so from Sauce Lab. Executing test cases on Sauce for such applications needs special configurations.
...
- Open the associated execution profile.
- Under Edit Additional Capabilities, set the desired value for slw.tunnelIdentifier.
The value set here for this key will be required when executing sc.exe in step 6. For more information on configuring this property, refer Sauce Labs documentation. - Download the Sauce Connect Proxy on any machine in the network using following url:
https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy - Extract the folder at the suitable location.
You will find the sc.exe file in the bin folder. - Open Command Prompt and navigate to the folder where sc.exe file exists.
Enter the following command.
Code Block theme Midnight sc -u <sauce_username> -k <accesskey> -i <tunnelID> For example: sc -u john.doe -k 8becc95c-4559-4582-a617-7d38544797a2 -i tunnel-1232
- Press Enter.
Once the SC.exe file is executed successfully, you can execute your offline tests on Sauce environment using the same method mentioned above.
Execution using Browser Capabilities
You can also execute test cases on remote environment using selenium supported Browser Capabilities.
For more information on this, please refer to the /wiki/spaces/T2/pages/1381634806 section.
Executing Test Cases on iOS Devices
If you are executing test cases on iOS mobile devices using Sauce Labs, you need to configure settings in the iosdc.properties file accordingly.
...
Execution using W3C Protocols
W3C-compliant Selenium capabilities and protocol is supported for test case executions in the Sauce Labs environment with selenium v3.11.0 and above.
...
Execution on Sauce Labs Real Device-TestObject
Introduction
The Sauce Labs Real Device-TestObject provides you the ability to run Qualitia test cases across Android and iOS physical devices. With its Web interface, you can run tests using different combinations of platforms, platform versions, and browsers. In this environment all Qualitia mobile application tests are executed on physical devices. Sauce Labs allows you viewing video recordings of the test case executions.
To know more about Sauce Labs click here.
...
- An active account with valid user name / password and
- Sufficient balance of execution hours.
...
Mandatory Settings and Execution Guidelines
Mandatory Execution Guidelines:
You must upload your mobile application under test in TestObject. which generates a unique app_path for your mobile app.
For more information, refer to https://wiki.saucelabs.com/display/DOCS/Uploading+and+Accessing+Applications+with+Real+Devices
...
Settings in the Associated Execution Profile:
You must create execution profiles that are solely used for execution on the Browserstack platform from the Qualitia Client. In those execution profiles, you must define the below settings that are mandatory for test suite execution.
...
- In the Execution Parameters tab, select ScreenCaptureMode as Browser.
Creating a Real Device Project
In order to automate testing on Sauce Labs Real Device, you must create a project on TestObject. When creating a project, you provide information about the type of application to test (Web or Mobile app) and the device settings you want to use in your tests. After creating a project, you will get a testobject_api_key (for example: 1234567890123456789012345678901234567890). You will have to add this key in testobject.properties against to.testobject_api_key property.
...
Viewing Test Case Execution and Reports
Once you trigger the offline package execution, you can view the status of your ongoing executions in Appium Test Results section. Each Test or Suite appears as a single grid with details like device, time, and duration of the test execution against it. Click the test case name from the list to view detailed logs, video recordings, and so forth of the test case.
...
Executions on Perfecto
Introduction
Perfecto is a cloud-based lab for Web and Mobile application testing. It allows running tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment, all desktop web application tests are executed on virtual machines (VMs) and mobile application tests are executed on physical devices. Perfecto allows you viewing video recordings of the test case executions after the suite executions are completed.
To know more about Perfecto, click here.
Prerequisite:
- An active user account on Perfecto.
...
Mandatory Settings and Execution Guidelines in Qualitia
The mobile native and hybrid application file, that is the apk, ipa, and app file, is not uploaded on Perfecto automatically during the Qualitia Automation Studio executions.
...
Viewing Execution Status and Reports in Perfecto's Execution Center
Once you have triggered the offline execution, you can view the status of your ongoing executions in Perfecto's Execution Center. Each test case appears on dashboard as a single grid with execution status against it. The status is updated after each test execution is completed in Perfecto's environment with its status (FAILED / SUCCESS) against it. Select the test case from the list and click Report from top to view recording of your test case.
| Info |
|---|
Qualitia and Perfecto's reports may mismatch if Qualitia test case contains any situation which ends the WebDriver session. This situation may arise due to the use of actions like Mobile.CloseApp, opening multiple WebDriver sessions in a single test case and so forth. |
To view the execution report of Qualitia test cases:
...
Executing Test Cases Behind Firewall Applications
Few applications are configured to be accessed using private networks. Such applications deny access when they are being accessed from other than local networks and so from Perfecto. Executing test cases on Perfecto for such applications needs special configurations.
...
Limitations of Remote/Cloud Executions
When using Qualitia Offline Package on the Remote and Cloud environments, following are the limitations users face when executing test cases.
...
Executing test cases in Cloud environment using user defined Firefox profile is experimental.
Support for AutoIT in Cloud Execution Environment
In order to achieve execution of Windows based actions such as DownloadFile, PressKeys, WindowsAuthentication, and so on, Qualitia supports Sauce's "Pre-Run" feature. It allows uploading AutoIT.exe (or any exe) and running before starting test case executions. Running .exe performs the windows based actions flawlessly.
...