Xdebug 2 vs Xdebug 3 Performance Comparison
Xdebug is a PHP extension that assists in debugging, tracing, profiling, and provides tooling for code coverage, and enhances PHP with a more informative debug functionality. Xdebug is the de-facto PHP extension for step-debugging PHP, and is a mature tool for PHP development.
Developed by Derick Rethans, Xdebug 2 is the current stable version. Xdebug 3 had its first RC release out just a few days ago, and this post benchmarks the performance improvements in the new version.
Current stable version of Xdebug, Xdebug 2.8, has several configuration options to selectively toggle features (such as tracing, code coverage, step debugger, etc.). However, loading Xdebug to PHP comes with a significant performance impact, to a point that a performance difference of 2x-10x is not unheard-of. Composer project, for example, makes use of composer/xdebug-handler tool re-call the Composer command with Xdebug disabled if it detects Xdebug is enabled.
Over a few sessions in various PHP conferences, Derick explained his plans on Xdebug 3, which focuses on a more modular architecture, that reduces the footprint of Xdebug by eliminating several internal processing that Xdebug might not use, due to that feature being disabled.
Xdebug 3 mode
Xdebug 3 provides a new configuration xdebug.mode, that accepts a mode, or simply "off", that provides a close to zero performance impact.
Various features in Xdebug project is now split into a single mode selector, and significantly reduces the complexity in its configuration.
[xdebug]
- xdebug.coverage_enable=1
- xdebug.auto_trace=0
- xdebug.gc_stats_enable=false
- xdebug.profiler_enable=0
- xdebug.profiler_enable_trigger=0
- xdebug.remote_autostart=0
- xdebug.overload_var_dump=0
- xdebug.default_enable=1
- xdebug.remote_enable=0
+ xdebug.mode = coverageXdebug modes are simple configuration options, and is the key in its performance benefits.
xdebug.mode=off: All Xdebug features are off, and adds a close to zero impact on performance.xdebug.mode=develop: Provides developer assist features, such as enhancedvar_dump.xdebug.mode=debug: Step debugger.xdebug.mode=trace: Code tracer.xdebug.mode=profile: Code profiler.xdebug.mode=gcstats: Provides statistics on PHP's garbage collection.
Benchmarks
The following benchmarks were done on PHP 7.4, PHP 8.0 RC4, with Xdebug 2.8 and Xdebug 3.0 RC1 being subjects. The benchmark script is the standard micro_bench.php on PHP source, that runs several benchmarks on various PHP features.
Note that Xdebug 2.8 is not available, nor compile for PHP 8.0. All PHP 8.0 tests are done on Xdebug 3 only. All tests are done with Opcache is enabled, to measure close to real-life use cases.
Extension Footprint
Loading the Xdebug extension itself used cause a significant performance hit. Projects like Composer/xdebug-handler attempted to work-around, and Xdebug 3 now provides an off mode that provides close to zero impact when Xdebug is loaded.
In Xdebug 2, there are multiple configuration options that turns off various Xdebug features. The major difference in Xdebug 3 is that, with xdebug.mode=off, it brings a close to zero performance impact when the extension is loaded. Xdebug 2, on the other hand, processes code and PHP internals even if the features are turned off and left unused.
The mode=off series indicates the elapsed time when the extension is loaded (i.e. with zend_extension=xdebug.so), and xdebug.mode=off set. This effectively turns off all Xdebug features in Xdebug 3, and brings down the footprint of the extension itself to close to zero.
For the Xdebug 2 test, all features were disabled with multiple INI configurations:
xdebug.coverage_enable=0
xdebug.auto_trace=0
xdebug.gc_stats_enable=false
xdebug.profiler_enable=0
xdebug.profiler_enable_trigger=0
xdebug.remote_autostart=0
xdebug.overload_var_dump=0
xdebug.default_enable=0
xdebug.remote_enable=0
Xdebug 3 footprint is nearly 99.4% less than the impact of Xdebug 2 when all features are turned off.
Developer Assists
Xdebug's developer assists include an overloaded var_dump function that provides more information about the caller file and position, and stack traces attached to warnings and errors.
In Xdebug 3, this mode is enabled with xdebug.mode=develop. In Xdebug 2, it was with configuration options below, while the rest is turned off:
xdebug.default_enable=1
xdebug.overload_var_dump=2
Xdebug 3 is nearly 25% faster with developer assists compared Xdebug 2.
Step Debugger
When step debugger is enabled, Xdebug halts the script if it encounters a break-point, and connects to a debug client. This benchmarks tests with the step debugger enabled in Xdebug 2. This test was done with no breakpoints set in the code.
xdebug.remote_autostart=1
xdebug.remote_enable=1
Xdebug 3 step debugger is nearly 34% faster compared Xdebug 2.
Code Coverage
Xdebug provides code coverage feature, that helps to measure how much of the code base was executed ("covered") during a test.
In addition to performance benefits, Xdebug provides path coverage that helps drill down the code coverage to individual branches in ternary and compound if statements.
In Xdebug 2, code coverage was tested with INI toggles, and xdebug.mode=coverage in Xdebug 3.
xdebug.coverage_enable=1
This benchmark was also done on league/commonmark test suite, that tests several files rather than a micro benchmark that executes the same code over and over.
Xdebug 3 code coverage is 10%-38% faster compared to Xdebug 2.
Xdebug vs PCOV
PCOV is a light-weight PHP extension that provides code coverage feature without any additional features similar to Xdebug. It performs significantly faster compared to Xdebug 2 and Xdebug 3, albeit the lack of path coverage support.
PHP Code Coverage Tools: Xdebug 2, Xdebug 3, PCOV, and phpdbg
Summary
With all features compared, Xdebug provides significant performance improvements with its modular functionality.
Note that xdebug.mode configuration can only be set in an INI file, or with XDEBUG_MODE=develop environment variable, but it now simplifies temporarily enabling individual Xdebug features. When xdebug.mode=off is set, there is close to zero performance impact.
Thanks to Derick Rethans for his awesome work Xdebug project for over 18 years to date. He also reviewed this post for technical accuracy. Support this well-deserved project at at xdebug.org/support.
