my thoughts on test automation

Internal or external test execution environment?

For many organizations, outsourcing Selenium WebDriver test execution environment can be a sound option. Before you decide whether to maintain your test execution environment in-house or use external provider, you should answer yourself a couple of questions. Some of them are not trivial, however do it carefully if you want to avoid further challenges.

  • Can you provide public URL for your test environment? If no, does your external supplier provide proxy which can be used to access application running within your company’s intranet? Does this tool have any known limitations?
  • Heavy client-side processing with JavaScript is becoming more and more common practice in web development. If your application makes extensive use of JavaScript, compare specifications of in-house and externally supplied VMs. How do they refer to recommended hardware specification for OS that they run? Comparing JavaScript benchmark results can be a good option.
  • Do you need to execute your tests on desktop editions of Windows OS? See this post.
  • Do you plan to access your application over HTTPS? If so, can your external provider support mechanisms to handle possible issues with SSL certificates – e.g. installation of custom certificates on execution VMs or disabling warnings using proxy? Currently WebDriver is completely unable to handle SSL alert pop ups in Safari. Workaround for IE exists, however this is still a workaround.
  • Do you need your test execution VMs to run in particular time zone? If so, can you manage that with your external provider?
  • Do you need run your tests from specific IP address, e.g. due to IP whitelisting? If so, can your external provider supply you with such static address? Is it just a single address, a couple of them, or entire range?
  • Do you need to modify default registry settings on execution VMs? WebDriver support for IE11 is based on registry setting. Can this be done by your external provider?
  • Check if external provider can offer support during your working hours. Time zone difference can be challenging.

All in all, switching to external provider can be a good choice, if planned carefully.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: