Setting up Jenkins CI, Selenium grid and PHPUnit Selenium to perform integration testing
When developing bigger TYPO3 websites with a lot of own extensions and complex functionality, it is nescessary to make sure, that the integration of new features or changes to existing code does not break the functionality of the website.To meet those requirements, it is recommended to use a continuous integration platform (like Jenkins CI) to automate the testing of the software. The platform can be used to automatically run the unit tests of your TYPO3 ExtBase extensions. If you use the testing framework included in the TYPO3 extension "phpunit", the tests can also help you testing your code with real data from the database.
Another thing you can (and should) test is the website and it's functionality itself. To do so, you can use a browser automation tool to run automated tests against real browsers. One tool available is the Selenium Server.
In this article I will show you how to build up a continuous integration solution consiting of Jenkins CI, Selenium Grid and 3 nodes with different webbrowsers and operating systems. Finally I will show you how to create some simple selenium tests with PHPUnit selenium and show, how to automatically run them against different environment configurations. Please note, that I will not go into deep details installing and configuring each software.
Setting up Jenkins CI and Selenium Grid
First of all, you need to install Jenkins CI on a server. For this article, a fresh Debian Server has been installed with all dependencies for Jenkins CI and finally, the latest release of Jenkins CI has been downloaded from the Jenkins CI homepage. Installation and configuration is really simple as described here. After installation and configuration, Jenkins CI is ready for use. To run PHPUnit tests, make sure you have installed PHPUnit, PHPUnit Selenium and all necessary extensions you need to run your tests.The next step is to install and configure Selenium on the Jenkins CI server. This instance of Selenium will run Selenium Grid, which will act as the master Selenium grid hub, where all other Selenium nodes connect to. Open the Jenkins Plugin manager and install the Selenium plugin. Installation is really easy and you just have to install the plugin and restart your Jenkins CI. In the Jenkins CI web interface you now have the new menu "Selenium grid" as shown on the screenshot below.
Selenium Grid is ready to handle nodes. Now it is time to configure the Selenium nodes.
Configuring the Selenium nodes
For this article, 3 virtual machines with different browser configurations have been created and configured.- Windows XP 32 bit with the following webbrowsers
- Internet Explorer 8
- Firefox 19
- Opera 12
- Chrome 25
- Windows 7 32 bit with the following webbrowsers
- Internet Explorer 9
- Firefox 19
- Opera 12
- Chrome 25
- Ubuntu 12.10 64 bit with the following webbrowsers
- Firefox 19
- Chrome 25
For all virtual machines, JAVA and the browsers listed above has been installed. Chrome, Opera and Internet Explorer requires drivers, so the locally installed Selenium Server can remote control the local webbrowser. Those drivers can be downloaded here or directly via the Selenium homepage. Finally, the latest release of Selenium has been downloaded to the machines.
For all nodes, the following command has been used to start Selenium.
java -jar \path\to\selenium-server-standalone-2.29.0.jar -role node -hub http://jenkins-server:4444/grid/register -browser "browserName=internet explorer,version=8,maxInstances=1" -browser "browserName=firefox,version=19,maxInstances=5" -browser "browserName=opera,version=12,maxInstances=5" -browser "browserName=chrome,version=25,maxInstances=5"
The node connects to the master Selenium hub and registers the local browsers to the hub. Note, that the parameter "-browser" does not accept spaces in between the individual arguments.
After all nodes have been configured and the local Selenium Server has been started, the Selenium Grid plugin in Jenkins CI should show all registered nodes as shown on the screenshot below.
The Selenium Grid and all nodes are now ready for usage.
The first Selenium tests
Now you are ready to write the first test. Create a new PHP file (selenium-tests.php) and include the following contents.
<?php
class DefaultTest extends PHPUnit_Extensions_Selenium2TestCase {
/*
* Setup
*/
protected function setUp() {
$this->setHost('your-jenkins');
$this->setBrowser('internet explorer');
$this->setBrowserUrl('http://www.google.com');
}
/**
* @test
*/
public function testTitle() {
$this->url('http://www.google.com');
$this->assertEquals('Google', $this->title());
}
}
?>
Use the setHost() method to set the hostname of your jenkins. This can be an IP-Address or a hostname. Do not include any ports - PHPUnit Seleniums does this for you.
Before setting up a job in jenkins (which I do not show in this article), you can run the test locally by using the following command.
phpunit selenium-tests.php
Now the test is sent to the Selenium Grid server and executed an a node matching the desired browser.
The result should look like shown below
PHPUnit 3.7.13 by Sebastian Bergmann.
.
Time: 6 seconds, Memory: 6.00Mb
OK (1 test, 1 assertion)
<?php
class DefaultTest extends PHPUnit_Extensions_Selenium2TestCase {
/*
* Setup
*/
protected function setUp() {
$this->setHost('your-jenkins');
$this->setDesiredCapabilities( array('version' => '8') );
$this->setBrowser('internet explorer');
$this->setBrowserUrl('http://www.google.com');
}
/**
* @test
*/
public function testTitle() {
$this->url('http://www.google.com');
$this->assertEquals('Google', $this->title());
}
}
?>
Now run the test again and you should see on the Windows XP machine, that Internet Explorer 8 is launched and the URL www.google.com is opened.
That's all. You can modify the desired capabilities to suit the needs of your tests. Since the parameter accepts an array, you can also set the browser version and the target platform together.
The last step is to check in your tests to a repository (e.g. GIT) and setup a new job in Jenkins to execute the selenium tests periodically.