Introduction

BrowserStack is a cloud environment that allows running tests in the cloud using different combinations of operating systems, browsers, and browser versions. In this environment, all tests are executed on BrowserStack VM. You can view the ongoing tests execution on BrowserStack.

For more information about BrowserStack, click here.

Before you start offline execution on BrowserStack, ensure you have:

An active account with valid Username/Access Key and
Sufficient balance of execution hours

Configuring qualitia.properties

In order to execute test cases on BrowserStack, you need to configure the qualitia.properties file accordingly.

To execute the offline package on the BrowserStack environment:

Open the qualitia.properties file from the config folder of offline package.
Set xmlPath to the location where your test case execution XMLs (including Suite.xml and Map.xml) are stored.
Set LogPath to the folder where you want to store your test cases results.
Set ExecutionEnvironment as browserstack.

In this section

Configuring browserstack.properties

Once the qualitia.properties file is updated, you must configure the browserstack.properties file in order to execute offline package in the BrowserStack environment.

To update the browserstack.properties file:

Open the browserstack.properties file.
Set custom_browserstack_url to valid URL.
Replace the custom_username and custom_accesskey to valid values from the given example to form custom_browserstack_url.
Set properties "custom_username", "custom_accesskey" to valid values with reference to BrowserStack's active user account.
Set "OS", "OS_Version", "browser", and "version" to the corresponding OS, browser, and browser version that you want your offline package to execute on.

For example, if you want your offline package to execute on Windows 10, Internet Explorer 11 then set keys as follows:

OS=Windows
OS_Version=10
browser=iexplore
version=11

If you are providing these values through command line, each value should be provided within double quotes if values are containing spaces; otherwise, the spaces in values will result in inconsistencies. For example, java -jar qualitiaoffline-xxx.jar platform="Windows 8.1".

The OS/OS_Version properties override the values set for Platform property. For example, if you have set the OS/OS_Version properties as Windows/10 and the property for Platform is set as Windows 8, then offline Package is executed on Windows/10.

For more information on platforms, browsers, browser versions supported on BrowserStack, see the following: https://www.browserstack.com/list-of-browsers-and-platforms?product=live.

Once all the configuration settings are done successfully, you can now execute test cases on BrowserStack.

BrowserStack does not support all Selenium versions. Qualitia users can use the browserstack.selenium_version capability to specify the version they want to use when executing test cases.
For more information about supported versions of Selenium, please refer to the BrowserStack documentation.

Qualitia Bot - Execution

After you configure all the files, you can trigger the test case execution by opening the command prompt and navigating to the folder where you saved the Qualitia bot. After navigating to the designated folder: execute the following command;

java -jar qualitiaoffline-xxx.jar

This command executes tests available at the xmlPath mentioned in the qualitia.properties file.

In case you have multiple versions of java, Qualitia uses the version set against the "JAVA_HOME" system variable under System Properties settings of your computer.

Different Ways of Setting Properties

The values you provide in default property files (qualitia.properties, browserstack.properties, and so forth) can also be provided using user-defined properties file and command line interface as follows:

1	Default Property Files (from the config folder of offline package)	It is a set of default BrowserStack properties file. You can always use this file to define your set of properties.
2	User-Defined BrowserStack Properties	You can create your own file with the set of properties and use when executing test cases in the BrowserStack environment. These are called user defined property files. This gives you provision to override the property values given in the default property files (for example: browserstack.properties). For example: java -jar qualitiaoffline-x.x.x.jar browserstackfilename="D:\\BrowserStack\\BrowserStack_version01.properties"
3	Command Line Arguments	Along with the two options mentioned above, you can also provide the properties using command line interface. Values provided using the command line interface override all the values provided in the default property files (such as browserstack.properties) and user defined properties file. In order to provide arguments from the command line interface, you must enter values in following syntax. For example: java -jar qualitiaoffline-xxxx.jar custom_username=<<BrowserStack user name>> platform=MAC recordVideo=false The command mentioned above overrides all the values provided in default property files (browserstack.properties) and user defined property file (if any).

Default Property Files (from the config folder of offline package)

It is a set of default BrowserStack properties file. You can always use this file to define your set of properties.

User-Defined BrowserStack Properties

You can create your own file with the set of properties and use when executing test cases in the BrowserStack environment. These are called user defined property files. This gives you provision to override the property values given in the default property files (for example: browserstack.properties).

For example:

java -jar qualitiaoffline-x.x.x.jar browserstackfilename="D:\\BrowserStack\\BrowserStack_version01.properties"

Command Line Arguments

Along with the two options mentioned above, you can also provide the properties using command line interface. Values provided using the command line interface override all the values provided in the default property files (such as browserstack.properties) and user defined properties file. In order to provide arguments from the command line interface, you must enter values in following syntax.

For example: java -jar qualitiaoffline-xxxx.jar custom_username=<<BrowserStack user name>> platform=MAC recordVideo=false

The command mentioned above overrides all the values provided in default property files (browserstack.properties) and user defined property file (if any).

Viewing Current Execution on BrowserStack

Once you have triggered the offline execution, you can view the status of your ongoing executions on BrowserStack. BrowserStack considers each Test case as a new project. The status is updated after each project is completed with its status against it. Also, you can filter the projects by their status (Completed, Timeout, or Errors) or you can search it by build name or session name.

You can record your test case execution configuring the browserstack.properties file accordingly. If you set the recordVideo value as true and all the test case executions will be recorded. You can download these videos once the test execution is completed.

Execution Behind Firewall

Sometimes applications are configured to be accessed within private networks. Such applications deny access when they are being accessed from other than local networks and so from BrowserStack. Executing test cases on BrowserStack for such applications needs special configurations.

When executing test case for applications running behind firewall on Windows, you need to execute the BrowserStackLocal.exe file on the machine in the private network to allow BrowserStack to execute tests flawlessly.

To execute test cases for applications running behind firewall on BrowserStack (Windows):

Make sure you have set the browserstack.local value as true in the browserstack.properties file.
Download the BrowserStackLocal.exe on any machine in the network using the following url:
https://www.browserstack.com/automate/python#setting-local-tunnel
Unzip the folder.
Open Command Prompt and navigate to the BrowserStackLocal.exe file followed by BrowserStack Access key.
For example:
<Path where the file is saved> BrowserStackLocal.exe --key <BrowserStack access key>
Press Enter.

Once the BrowserStackLocal.exe file is executed successfully, you can execute your Offline tests on BrowserStack environment using the same method mentioned above. When executing test case for applications running behind firewall on Mac, you need to execute the BrowserStackLocal Unix Executable file on the machine in the private network to allow BrowserStack executing tests flawlessly.

To execute test cases for applications running behind firewall on BrowserStack (Mac):

Make sure you have set the browserstack.local value as true in the browserstack.properties file.
Download the BrowserStackLocal Unix Executable file on any machine in the network using the following url:
https://www.browserstack.com/automate/python#setting-local-tunnel
Unzip the folder.
Open Terminal and navigate to the BrowserStackLocal Unix Executable file followed by BrowserStack Access key.
For example:
./BrowserStackLocal --key <BrowserStack access key>
Press Enter.

Once the BrowserStackLocal Unix Executable file is executed successfully, you can execute your Offline tests on BrowserStack environment using the same method mentioned above.

Execution Behind Proxy

Qualitia users can execute the test cases behind a proxy.

In order to execute the test cases behind a proxy, you must configure Proxy Properties in the browserstack.properties file as shown in the screenshot below.

Qualitia users can define these properties using the command line as well.

Execution using Browser Capabilities

Selenium allows you setting desired browser specific properties to be used when executing test cases. These properties can be set for offline execution as well. You can provide these browser specific details through browser-capability files. For each supported browser (Firefox, IE, Google Chrome, and Safari) there is a separate property file present in the config folder of Qualitia's offline package. Following is the list of files;

firefoxdc.properties
googlechromedc.properties
iexploredc.properties
safaridc.properties

Each property file above is a set of all the browser specific properties that Selenium allows you change. If you do not change any property, Selenium uses its default values.

Changing Browser Capabilities

Selenium uses its own settings for each key that has been commented in the file, except unexpectedAlertBehaviour. Changing properties here overrides the values provided in qualitia.properties / browserstackdc.properties, or selenium's default values. When executing test cases using offline package, you can change some property values providing command line arguments accordingly. But all capability keys do not accept values from command line, hence it is recommended providing values through property files.

To change browser capabilities:

Open the browserdc.properties file from the config folder of offline package.
Locate the value you want to change in the file.
Remove the # sign located prior to the capability.
Enter the value you want to set for the capability.

For example; if you want to change the version of the browser to be used for test execution, remove the # sign located prior to the version and enter the version number against it.

Qualitia gives you provision to set these values for Selenium. Selenium has its own control on what value to use and how to reflect them in Execution Browser. For more information about Selenium Capabilities, refer following links:

Execution Reports

Test execution reports are launched on the machine from where the execution was triggered after the test case execution is completed. You can view the browser type and browser version on the report displayed.

Execution reports display OS and Browser with their corresponding versions on its summary page along with the execution environment used. For example, if you have chosen execution as BROWSERSTACK and execution browser as Firefox 55, report appears as follows;

For Execution through remote environment, OS version is not displayed.
For execution through any environment on Linux, OS version is not displayed.
For execution on local environment on Windows 8.1, the OS details are shown as 'Windows NT (UNKNOWN)'.

Browser not supported