Home

Awesome

SPX - A simple profiler for PHP

Build Status Supported PHP versions: 5.4 .. 8.x Supported platforms: GNU/Linux, macOS & FreeBSD Supported architectures: x86-64 or ARM64 License

<a href="https://www.buymeacoffee.com/noisebynw" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>

Click here for a live demo of the analysis screen

Showcase

Showcase

SPX, which stands for Simple Profiling eXtension, is just another profiling extension for PHP. It differentiates itself from other similar extensions as being:

Requirements

Platforms support is currently quite limited. Feel free to open an issue if your platform is not supported. Current requirements are:

Installation

Prerequisites

Install the extension

git clone https://github.com/NoiseByNorthwest/php-spx.git
cd php-spx
git checkout release/latest
phpize
./configure
make
sudo make install

Then add extension=spx.so to your php.ini, or in a dedicated spx.ini file created within the include directory. You may also want to override default SPX configuration to be able to profile a web request, with this one for example for a local development environment.

ZTS PHP (multi-thread)

ZTS PHP is supported, with these extra limitations:

Also, consider ZTS PHP support as still being in beta.

Linux, PHP-FPM & I/O stats

On GNU/Linux, SPX uses procfs (i.e. by reading files under /proc directory) to get some stats for the current process or thread. This is what is done under the hood when you select at least one of these metrics: mor, io, ior or iow.

But, on most PHP-FPM setups, you will have a permission issue preventing SPX to open a file under /proc/self directory. This is due to the fact that PHP-FPM master process runs as root when child processes run as another unprivileged user.

When this is the case, the process.dumpable = yes line must be added to the FPM pool configuration so that child processes will be able to read any file under /proc/self.

Development status

This is still experimental. API might change, features might be added or dropped, or development could be frozen.

You can still safely use it in a non-production environment.

Contributions are welcome but be aware of the experimental status of this project and please follow the contribution rules described here: CONTRIBUTING.md

Basic usage

web request

Assuming a development environment with the configuration described here and your application is accessible via http://localhost.

Just open with your browser the following URL: http://localhost/?SPX_KEY=dev&SPX_UI_URI=/ to access to the web UI control panel.

N.B.: http://localhost/ must be served by a PHP script through standard web server feature like directory index or URL rewriting. The PHP script will however not be executed, SPX will intercept and disable its execution to serve its content in place.

If you see only a blank page then make sure to set zlib.output_compression = 0 in your PHP configuration file

You will then see the following form:

Showcase

Then switch on "Enabled". At this point profiling is enabled for the current domain and your current browser session through a set of dedicated cookies.

Profiling can also be triggered with Curl as shown in this example:

curl --cookie "SPX_ENABLED=1; SPX_KEY=dev" http://localhost/

N.B.: You can also enable the profiling at INI configuration level via the spx.http_profiling_enabled setting, and therefore for all HTTP requests. However, keep in mind that using this setting on a high-traffic environment could quickly exhaust the storage device's capacity of the SPX's data directory.

Then refresh the web request you want to profile and refresh the control panel to see the generated report in the list below the control panel form.

Showcase

Then click on the report in the list and enjoy the analysis screen.

Command line script

Instant flat profile

Just prepend your command line with SPX_ENABLED=1 to trigger profiling. You will get the flat profile printed on STDERR at the end of the execution, even if you abort it by hitting Ctrl-C, as in the following example:

$ SPX_ENABLED=1 composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
^C
*** SPX Report ***

Global stats:

  Called functions    :    27.5K
  Distinct functions  :      714

  Wall time           :    7.39s
  ZE memory           :   62.6MB

Flat profile:

 Wall time           | ZE memory           |
 Inc.     | *Exc.    | Inc.     | Exc.     | Called   | Function
----------+----------+----------+----------+----------+----------
  101.6ms |  101.6ms |   41.8MB |   41.8MB |       12 | Composer\Json\JsonFile::parseJson
   53.6ms |   53.6ms |     544B |     544B |        4 | Composer\Cache::sha256
    6.91s |   41.5ms |   41.5MB |   -7.5MB |        4 | Composer\Repository\ComposerRepository::fetchFile
    6.85s |   32.3ms |   47.5MB |    5.4MB |        5 | 1@Composer\Repository\ComposerRepository::loadProviderListings
    7.8ms |    7.8ms |       0B |       0B |        4 | Composer\Cache::write
    1.1ms |    1.1ms |     -72B |     -72B |        1 | Composer\Console\Application::Composer\Console\{closure}
  828.5us |  828.5us |     976B |     976B |       12 | Composer\Util\RemoteFilesystem::findHeaderValue
  497.6us |  491.0us |  710.2KB |  710.2KB |        1 | Composer\Cache::read
    2.4ms |  332.6us |   20.9KB | -378.8KB |       34 | 3@Symfony\Component\Finder\Iterator\FilterIterator::rewind
  298.9us |  298.9us |    2.2KB |    2.2KB |       47 | Symfony\Component\Finder\Iterator\FileTypeFilterIterator::accept

N.B.: Just add SPX_FP_LIVE=1 to enable the live refresh of the flat profile during script execution.

Generate profiling report for the web UI

You just have to specify SPX_REPORT=full to generate a report available via the web UI:

SPX_ENABLED=1 SPX_REPORT=full ./bin/console cache:clear

Handle long-living / daemon processes

If your CLI script is long-living and/or daemonized (e.g. via supervisord), profiling its whole lifespan could be meaningless. This is especially true in case of a service waiting for tasks to process.
To handle this case, SPX allows to disable the automatic start of profiling and exposes 2 userland functions, spx_profiler_start(): void & spx_profiler_stop(): ?string, in order to respectively control the start and the end of the profiled spans.

Here is how you can instrument your script:

<?php

while ($task = get_next_ready_task()) {
  spx_profiler_start();
  try {
    $task->process();
  } finally {
    spx_profiler_stop();
  }
}

And of course this script must be run at least with profiling enabled and the automatic start disabled as in the following command:

SPX_ENABLED=1 SPX_REPORT=full SPX_AUTO_START=0 my_script.php

Automatic start can also be disabled for web requests via the spx.http_profiling_auto_start INI parameter or via the control panel.

Side notes:

Add custom metadata to the current full report

When profiling with full report as output, it could be handy to add custom metadata to the current report so that you will be able to easily retrieve it or differentiate it from other similar reports.

This is especially true for the long-living process use case which otherwise would not allow to differentiate a report from other ones of the same process.

To do that SPX exposes the spx_profiler_full_report_set_custom_metadata_str(string $customMetadataStr): void function.

As you may have notificed, this function accepts a string as custom metadata, for the sake of flexibility and simplicity on SPX side. It is up to you to encode any structured data to a string, for instance using JSON format.

The metadata string is limited to 4KB, which is large enough for most use cases. If you pass a string exceeding this limit it will be discarded and a notice log will be emitted.

This string will be stored among other current report's metadata and you will retrieve it in the report list on web UI side.

spx_profiler_full_report_set_custom_metadata_str() can be called at any moment as long as the profiler is already started and not finished yet, which means:

Here is an example:

<?php

while ($task = get_next_ready_task()) {
  spx_profiler_start();

  spx_profiler_full_report_set_custom_metadata_str(json_encode(
    [
      'taskId' => $task->getId(),
    ]
  ));

  try {
    $task->process();
  } finally {
    spx_profiler_stop();
  }
}

Advanced usage

Configuration

NameDefaultChangeableDescription
 spx.data_dir/tmp/spxPHP_INI_SYSTEMThe directory where profiling reports will be stored. You may change it to point to a shared file system for example in case of multi-server architecture.
 spx.http_enabled0PHP_INI_SYSTEMWhether to enable web UI and HTTP request profiling.
 spx.http_keyPHP_INI_SYSTEMThe secret key used for authentication (see security concern for more details). You can use the following command to generate a 16 bytes random key as an hex string: openssl rand -hex 16.
 spx.http_ip_varREMOTE_ADDRPHP_INI_SYSTEMThe $_SERVER key holding the client IP address used for authentication (see security concern for more details). Overriding the default value is required when your application is behind a reverse proxy.
 spx.http_trusted_proxies127.0.0.1PHP_INI_SYSTEMThe trusted proxy list as a comma separated list of IP addresses<b>*</b>. This setting is ignored when spx.http_ip_var's value is REMOTE_ADDR.
 spx.http_ip_whitelistPHP_INI_SYSTEMThe IP address white list used for authentication as a comma separated list of IP addresses<b>*</b>.
 spx.http_ui_assets_dir/usr/local/share/misc/php-spx/assets/web-uiPHP_INI_SYSTEMThe directory where the web UI files are installed. In most cases you do not have to change it.
 spx.http_profiling_enabledNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_ENABLED parameter, for HTTP requests only. See here for more details.
 spx.http_profiling_auto_startNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_AUTO_START parameter, for HTTP requests only. See here for more details.
 spx.http_profiling_builtinsNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_BUILTINS parameter, for HTTP requests only. See here for more details.
 spx.http_profiling_sampling_periodNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_SAMPLING_PERIOD parameter, for HTTP requests only. See here for more details.
 spx.http_profiling_depthNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_DEPTH parameter, for HTTP requests only. See here for more details.
 spx.http_profiling_metricsNULLPHP_INI_SYSTEMThe INI level counterpart of the SPX_METRICS parameter, for HTTP requests only. See here for more details.

*: * (match all) and subnet masks (e.g. 192.168.1.0/24) are supported.

Private environment

For your local & private development environment, since there is no need for authentication, you can use this configuration:

spx.http_enabled=1
spx.http_key="dev"
spx.http_ip_whitelist="127.0.0.1"

And then access to the web UI at http(s)://<your application host>/?SPX_KEY=dev&SPX_UI_URI=/.

Available metrics

Here is the list of available metrics to collect. By default only Wall time and Zend Engine memory usage are collected.

Key (command line)NameDescription
wtWall timeThe absolute elapsed time.
ctCPU timeThe time spent while running on CPU.
itIdle timeThe time spent off-CPU, that means waiting for CPU, I/O completion, a lock acquisition... or explicitly sleeping.
zmZend Engine memory usageEquivalent to memory_get_usage(false).
zmacZend Engine allocation countNumber of memory allocations (i.e. allocated blocks) performed.
zmabZend Engine allocated bytes<b>*</b>Number of allocated bytes.
zmfcZend Engine free countNumber of memory releases (i.e. freed blocks) performed.
zmfbZend Engine freed bytes<b>*</b>Number of freed bytes.
zgrZend Engine GC run countNumber of times the GC (cycle collector) have been triggered (either manually or automatically).
zgbZend Engine GC root buffer lengthRoot buffer length, see explanation here. It could be helpful to track pressure on garbage collector.
zgcZend Engine GC collected cycle countTotal number of collected cycles through all GC runs.
zifZend Engine included file countNumber of included files.
zilZend Engine included line countNumber of included lines.
zucZend Engine user class countNumber of userland classes.
zufZend Engine user function countNumber of userland functions (including userland class/instance methods).
zuoZend Engine user opcode countNumber of included userland opcodes (sum of all userland file/function/method opcodes).
zoZend Engine object countNumber of objects currently held by user code.
zeZend Engine error countNumber of raised PHP errors.
morProcess's own RSS<b>**</b>The part of the process's memory held in RAM. The shared (with other processes) memory blocks are not taken into account. This metric can be useful to highlight a memory leak within a PHP extension or deeper (e.g. a third-party C library).
ioI/O (reads + writes)**Bytes read or written while performing I/O.
iorI/O (reads)**Bytes read while performing I/O.
iowI/O (writes)**Bytes written while performing I/O.

*: Allocated and freed byte counts will not be collected if you use a custom allocator or if you force the libc one through the USE_ZEND_ALLOC environment variable set to 0.

**: RSS & I/O metrics are not supported on macOS and FreeBSD. On GNU/Linux you should read this if you use PHP-FPM.

Command line script

Available report types

Contrary to web request profiling which only support full report type (the one exploitable by the web UI), command line script profiling supports several types of report. Here is the list below:

KeyNameDescription
fpFlat profileThe flat profile provided by SPX. It is the default report type and is directly printed on STDERR.
fullFull reportThis is the report type for web UI. Reports will be stored in SPX data directory and thus will be available for analysis on web UI side.
traceTrace fileA custom format (human readable text) trace file.

Available parameters

NameDefaultDescription
SPX_ENABLED0Whether to enable SPX profiler (i.e. triggering profiling). When disabled there is no performance impact on your application (except for ZTS PHP where a disabled SPX still adds a little overhead).
SPX_AUTO_START1Whether to enable SPX profiler's automatic start. When automatic start is disabled, you have to start & stop profiling on your own at runtime via the spx_profiler_start() & spx_profiler_stop() functions. See here for more details.
SPX_BUILTINS0Whether to profile internal functions, script compilations, GC runs and request shutdown.
SPX_DEPTH0The stack depth at which profiling must stop (i.e. aggregate measures of deeper calls). 0 (default value) means unlimited.
SPX_SAMPLING_PERIOD0Whether to collect data for the current call stack at regular intervals according to the specified sampling period (0 means no sampling). The result will usually be less accurate but in some cases it could be far more accurate by not over-evaluating small functions called many times. It is recommended to try sampling (with different periods) if you want to accurately find a time bottleneck. When profiling a long running & CPU intensive script, this option will allow you to contain report size and thus keeping it small enough to be exploitable by the web UI. See here for more details.
SPX_METRICSwt,zmComma separated list of available metric keys to collect. All report types take advantage of multi-metric profiling.
SPX_REPORTfpSelected report key.
SPX_FP_FOCUSwtMetric key for flat profile sort.
SPX_FP_INC0Whether to sort functions by inclusive value instead of exclusive value in flat profile.
SPX_FP_REL0Whether to display metric values as relative (i.e. percentage) in flat profile.
SPX_FP_LIMIT10The flat profile size (i.e. top N shown functions).
SPX_FP_LIVE0Whether to enable flat profile live refresh. Since it plays with cursor position through ANSI escape sequences, it uses STDOUT as output, replacing script output (both STDOUT & STDERR).
SPX_FP_COLOR1Whether to enable flat profile color mode.
SPX_TRACE_SAFE0The trace file is by default written in a way to enforce accuracy, but in case of process crash (e.g. segfault) some logs could be lost. If you want to enforce durability (e.g. to find the last event before a crash) you just have to set this parameter to 1.
SPX_TRACE_FILECustom trace file name. If not specified it will be generated in /tmp and displayed on STDERR at the end of the script.

Setting parameters

Well, as you might already noticed in corresponding basic usage example, setting a SPX parameter for a command line script simply means setting an environment variable with the same name.

Web UI

Supported browsers

Since the web UI uses advanced JavaScript features, only the following browsers are known to be supported:

Control panel & report list

This is the home page of the web UI, divided into 2 parts:

Analysis screen

Click here for a live demo of the analysis screen

Showcase

Performance, report size & sampling

The analysis screen can nicely handle profile reports with up to several (5+) millions of recorded function calls with Chromium on my i5 @ 3.3GHz / 8GB desktop. In case you want to profile a long running, CPU intensive, script which tends to generate giant reports, you can enable sampling mode with the suitable sampling period. See SPX_SAMPLING_PERIOD parameter for command line script.

Metric selector

This is simply a combo box for selecting the currently analyzed metric.

Showcase

Color scheme selector

By default, function related blocks in the visualizations are colored according to their cost, with a color scale displayed at the top right of the screen.

You can also define a custom color scheme by clicking on the color scheme mode link, displayed at the top of the screen just after the metric selector. A drop-down window will then appear and allow you to switch between default and category mode and define (add/edit/delete) your categories (color, name, pattern list) for the category mode (see the screenshot below).

Showcase

Timeline overview

This visualization is the timeline overview of all called functions. You can change the selected time range by, represented by a transparent green rectangle, by simply dragging it horizontally.

Except for wall time, the current metric is also plotted (current value over time) on a foreground layer.

Supported controls:

Showcase

Timeline focus

This visualization is an interactive timeline which is able to control and keep focus on the selected time range.

Supported controls:

Except for wall time, the current metric is also plotted (current value over time) on a foreground layer.

Showcase

Flat profile

This visualization is the flat profile for the selected time range and the selected metric, displayed as a sortable table.

The Inc. and Exc. sub-columns respectively correspond to:

Showcase

Flame Graph

This visualization, designed by Brendan Gregg, allows to quickly find the hot code path for the selected time range and the selected metric. Metrics corresponding to releasable resources (memory, objects in use...) are not supported by this visualization.

Showcase

Function highlighting

You can highlight a function by clicking on one of its spans within the timeline or Flamegraph widgets or its name within the flat profile widget.

Showcase

Security concern

The lack of review / feedback about this concern is the main reason SPX cannot yet be considered as production ready.

SPX allows you to profile web request as well as command line scripts, and also to list and analyze profile reports through its embedded web UI. This is why there is a huge security risk, since an attacker could:

So, unless access to your application is already restricted at lower layer (i.e. before your application is hit, not by the application / PHP framework itself), a client triggering profiling or accessing to the web UI must be authenticated.

SPX provides two-factor authentication with these 2 mandatory locks:

Thus a client can profile your application via a web request only if its IP address is white listed and its provided key is valid.

Notes on accuracy

In tracing mode (default), SPX is subject to accuracy issues for time related metrics when the measured function execution time is:

The first issue is mitigated by using the highest resolution timer provided by the platform. On Linux, FreeBSD & recent macOS versions the timer resolution is 1ns; on macOS before 10.12/Sierra, the timer resolution is only 1us.

The second issue is mitigated by taking into account SPX's time (wall / cpu) overhead by subtracting it to measured function execution time. This is done by evaluating SPX constant per function overhead before starting profiling the script.

However, whatever the platform, if you want to maximize accuracy to find a time bottleneck, you should also:

Stubs

Stubs for SPX functions to be used with Intelephense

composer require --dev 8ctopus/php-spx-stubs

Credits

I have found lot of inspiration and hints reading:

License

SPX is open source software licensed under the GNU General Public License (GPL-3). See the LICENSE file for more information.

<!-- All external links should be here to avoid duplication and long lines with links -->