One of the recently added features is the new testing framework. We have invented a testing framework that will allows us to test every single line of the code. We expect that it will give thirty bees the strength it needs in order to recover from a sometimes (very) buggy code base. It will be used extensively in the coming months as it will allow us to remove bugs at a staggering rate!

For 1.0.0 we have only had the chance to automate the navigation through the back office and check if pages roughly look good, but in the next four months we will be focusing a lot on adding more and more tests. It will allow our developers to be more confident about the code and it will be highly effective at preventing future regressions. Another advantage is that it will quickly check community contributions. thirty bees will focus a lot on making good software together, so what could be better than having an automated system that will automatically test and see if your contributions are suitable to be merged into the code?

Types of tests

Do you suspect that the rounding errors are back? Let us know. If you also have know where in the code the issue might be, certainly let us know! The more details you give, the more effective we can use the new testing framework. The new framework will allow us to test the code in four ways, each issue requiring its own method:

  • Unit tests – These tests can cover every single line of the code. That is also the level at which they test, just the isolated functions. Despite the fact that PHP doesn’t allow to redefine classes during runtime and hence making PHP software harder to test, this does not mean we cannot run unit tests on PHP. The PrestaShop 1.6 architecture was partially rebuilt in order to overcome this limitation. We think that there is a better solution and use AspectMock instead. What AspectMock allows us to do, is to pick a random function, mock its dependencies and only focus on the function itself. No matter if a dependency is private or protected, AspectMock will find a way to get rid of the dependency and allow us to write powerful and effective unit tests.
  • Integration tests – When it is time to test how some functionality works together, we use integration tests. Some examples are: installing modules, processing CSV files to see if the import works. These tests can be very powerful when picking the right use cases.
  • Functional tests – These tests are used to “simulate” a web browser. They focus on the output of our software and usually focus on a just few elements on a page, to see if the framework does what it needs to do. These tests are defined at a feature level. Think of the address chooser on the customer portal, the homeslider on the homepage, etc.
  • Acceptance tests – For the user these tests are the most important ones, because they define whole use cases, such as “buy a product, then go to the one page checkout, add a different invoice address, then go through checkout”. By having a long list of these tests we also define our software, because writing down these tests effectively means that we are writing down the expected functionality of our software. We expect to use these tests very often.

Contribute

We are currently still in the process of developing the testing framework and therefore haven’t released the documentation on our documentation site, yet, but you can already check out how the testing frameworks works if you like.

The testing framework is using Codeception to run the 4 types of tests. This framework allows us to run everything on PHP and integrates well with PHPUnit. The tests themselves are run by Travis CI, the continuous integration server that keeps checking every single push and pull request we receive on our repository on GitHub. If you want to study how the tests work, fork the repository and check out the tests folder. You will see the folders for the four types of tests. Any file you add to one of these sub folders will be run by our testing framework and will immediately become part of the test suite. Don’t worry if your tests fail, you can keep trying with Travis until either the test works or gives you the right feedback about the status of the code. Every test type has at least one example already, feel free to tinker with them and learn how you can make effective tests. We would love to encourage you to share your tests with us, so that we can all make the most stable e-commerce software together. Leave a pull request or issue on our GitHub page and we will help you with the tests. Thanks for taking the time to improve the software!