my thoughts on test automation
Internal or external test execution environment?
For many organizations, outsourcing Selenium WebDriver test execution environment can be a sound option. Before you decide whether to maintain your test execution environment in-house or use external provider, you should answer yourself a couple of questions. Some of them are not trivial, however do it carefully if you want to avoid further challenges.
- Can you provide public URL for your test environment? If no, does your external supplier provide proxy which can be used to access application running within your company’s intranet? Does this tool have any known limitations?
- Do you need to execute your tests on desktop editions of Windows OS? See this post.
- Do you plan to access your application over HTTPS? If so, can your external provider support mechanisms to handle possible issues with SSL certificates – e.g. installation of custom certificates on execution VMs or disabling warnings using proxy? Currently WebDriver is completely unable to handle SSL alert pop ups in Safari. Workaround for IE exists, however this is still a workaround.
- Do you need your test execution VMs to run in particular time zone? If so, can you manage that with your external provider?
- Do you need run your tests from specific IP address, e.g. due to IP whitelisting? If so, can your external provider supply you with such static address? Is it just a single address, a couple of them, or entire range?
- Do you need to modify default registry settings on execution VMs? WebDriver support for IE11 is based on registry setting. Can this be done by your external provider?
- Check if external provider can offer support during your working hours. Time zone difference can be challenging.
All in all, switching to external provider can be a good choice, if planned carefully.