Git with Selenium UI Testing – Unpredictable results

A quick, and rather silly one, that has just caught me out a couple of times:

Take a situation where you’re working on a site, and want to run some Selenium tests in the background, possibly with a CI server like Jenkins.

You fix some tests in the develop branch following some recent changes to the copy on the site because you haven’t followed your TDD approach quite correctly, and commit the code or otherwise start the tests. Whilst these are running, because UI tests can take a while, you checkout a different git branch to work on that other feature request that is ongoing.

Ten minutes later, the Selenium tests have failed. So you stash your code changes, checkout develop again, check the tests and everything looks fine, so you run the tests that have failed and they pass. And you decide to run the tests again, just in case.

Whilst they’re running, you checkout the feature branch again and do some more work. Ten minutes later, a different set of tests have failed!

You repeat the same steps as before and find nothing that looks wrong at all. Then you realise, the copy changes haven’t been merged to the feature branch. So you checkout the develop branch again, start the tests and leave them well alone while you go and have a lie down for ten minutes.

And lo, all tests shall pass. And, sometimes, this is how we learn…

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>