Welcome

uBenchmarks are a collection of benchmarks which have been run on all Umbraco v7 derived releases.

The aim is to generate benchmarks to indicate either a performance regression or an increase in performance by benchmarking different areas of umbraco (I can't benchmark everything)

Please do not use these benchmarks as an absolute, they are intended to be used relatively to compare two versions. Just because it took me 2ms to do something doesn't mean that it will take you 2ms to do the thing too. Instead they should be used to indicate that the latest version may performing faster than the version you are currently running.

The version of umbraco we are testing is installed into an existing IIS site, and an empty local MSSQL Database. All of the tests are executed 4 times in a collated fashion, so we test all versions, then test them all again. Repeat and we take an average.

 

With love
Chief Benchmarker
Stephen


Updates

06/08/2018 

  • 7.12.1 has been added
  • 7.12.2 has been added
  • All test have been re-run now with a reboot of the server between testing each instance (10 for every release)

16/08/2018 7.12.0 has been added, but Im waiting until i re-run the test suite before any analysis is done. As usual these stats are hints only and not defacto. Here are some of the issues i'm having at the moment

  • Umbraco has changed our to .com which has broken the URL the installers call to for starter kits - which now means that the code which installs umbraco via selenium isn't working because the ID for the HTML element is different, this should be an easy fix but it gets complicated as it works in some versions of umbraco.
  • Chrome auto updates and I have had to upgrade some dependencies for it to work.

12/06/2018 I've run the test suite again, with all tests repeated 10 times. As previously thought those metrics should have been done with a pinch of salt! the v8 alpha isn't performing as bad as previously seen (hence why all tests are repeated). I have added tests for URL routing, and xpath too.

23/05/2018 I have done a first pass with the v8 alpha released at codegarden. These metrics should be taken with a grain of salt as I only had time to do a single test pass which means I don't know if v8's metrics are repeatable. - Stephen