Xdebug 2 vs Xdebug 3 Performance Comparison
Xdebug is a PHP extension that assists in debugging, tracing, profiling, and provides tooling for code coverage, and enhances PHP with a more informative debug functionality. Xdebug is the de-facto PHP extension for step-debugging PHP, and is a mature tool for PHP development.
Developed by Derick Rethans, Xdebug 2 is the current stable version. Xdebug 3 had its first RC release out just a few days ago, and this post benchmarks the performance improvements in the new version.
Current stable version of Xdebug, Xdebug 2.8, has several configuration options to selectively toggle features (such as tracing, code coverage, step debugger, etc.). However, loading Xdebug to PHP comes with a significant performance impact, to a point that a performance difference of 2x-10x is not unheard-of. Composer project, for example, makes use of composer/xdebug-handler tool re-call the Composer command with Xdebug disabled if it detects Xdebug is enabled.
Over a few sessions in various PHP conferences, Derick explained his plans on Xdebug 3, which focuses on a more modular architecture, that reduces the footprint of Xdebug by eliminating several internal processing that Xdebug might not use, due to that feature being disabled.
Xdebug 3 provides a new configuration
xdebug.mode, that accepts a mode, or simply
"off", that provides a close to zero performance impact.
Various features in Xdebug project is now split into a single mode selector, and significantly reduces the complexity in its configuration.
[xdebug] - xdebug.coverage_enable=1 - xdebug.auto_trace=0 - xdebug.gc_stats_enable=false - xdebug.profiler_enable=0 - xdebug.profiler_enable_trigger=0 - xdebug.remote_autostart=0 - xdebug.overload_var_dump=0 - xdebug.default_enable=1 - xdebug.remote_enable=0 + xdebug.mode = coverage
Xdebug modes are simple configuration options, and is the key in its performance benefits.
xdebug.mode=off: All Xdebug features are off, and adds a close to zero impact on performance.
xdebug.mode=develop: Provides developer assist features, such as enhanced
xdebug.mode=debug: Step debugger.
xdebug.mode=trace: Code tracer.
xdebug.mode=profile: Code profiler.
xdebug.mode=gcstats: Provides statistics on PHP's garbage collection.
The following benchmarks were done on PHP 7.4, PHP 8.0 RC4, with Xdebug 2.8 and Xdebug 3.0 RC1 being subjects. The benchmark script is the standard
micro_bench.php on PHP source, that runs several benchmarks on various PHP features.
Note that Xdebug 2.8 is not available, nor compile for PHP 8.0. All PHP 8.0 tests are done on Xdebug 3 only. All tests are done with Opcache is enabled, to measure close to real-life use cases.
Loading the Xdebug extension itself used cause a significant performance hit. Projects like Composer/xdebug-handler attempted to work-around, and Xdebug 3 now provides an
off mode that provides close to zero impact when Xdebug is loaded.
In Xdebug 2, there are multiple configuration options that turns off various Xdebug features. The major difference in Xdebug 3 is that, with
xdebug.mode=off, it brings a close to zero performance impact when the extension is loaded. Xdebug 2, on the other hand, processes code and PHP internals even if the features are turned off and left unused.
mode=off series indicates the elapsed time when the extension is loaded (i.e. with
xdebug.mode=off set. This effectively turns off all Xdebug features in Xdebug 3, and brings down the footprint of the extension itself to close to zero.
For the Xdebug 2 test, all features were disabled with multiple INI configurations:
xdebug.coverage_enable=0 xdebug.auto_trace=0 xdebug.gc_stats_enable=false xdebug.profiler_enable=0 xdebug.profiler_enable_trigger=0 xdebug.remote_autostart=0 xdebug.overload_var_dump=0 xdebug.default_enable=0 xdebug.remote_enable=0
Xdebug 3 footprint is nearly 99.4% less than the impact of Xdebug 2 when all features are turned off.
Xdebug's developer assists include an overloaded
var_dump function that provides more information about the caller file and position, and stack traces attached to warnings and errors.
In Xdebug 3, this mode is enabled with
xdebug.mode=develop. In Xdebug 2, it was with configuration options below, while the rest is turned off:
Xdebug 3 is nearly 25% faster with developer assists compared Xdebug 2.
When step debugger is enabled, Xdebug halts the script if it encounters a break-point, and connects to a debug client. This benchmarks tests with the step debugger enabled in Xdebug 2. This test was done with no breakpoints set in the code.
Xdebug 3 step debugger is nearly 34% faster compared Xdebug 2.
Xdebug provides code coverage feature, that helps to measure how much of the code base was executed ("covered") during a test.
In Xdebug 2, code coverage was tested with INI toggles, and
xdebug.mode=coverage in Xdebug 3.
This benchmark was also done on
league/commonmark test suite, that tests several files rather than a micro benchmark that executes the same code over and over.
Xdebug 3 code coverage is 10%-38% faster compared to Xdebug 2.
Xdebug vs PCOV
PCOV is a light-weight PHP extension that provides code coverage feature without any additional features similar to Xdebug. It performs significantly faster compared to Xdebug 2 and Xdebug 3, albeit the lack of path coverage support.
With all features compared, Xdebug provides significant performance improvements with its modular functionality.
xdebug.mode configuration can only be set in an INI file, or with
XDEBUG_MODE=develop environment variable, but it now simplifies temporarily enabling individual Xdebug features. When
xdebug.mode=off is set, there is close to zero performance impact.