Skip to content
/ jam Public
forked from jessp01/jam

JaM is a PHP monitoring system that supports storing PHP errors (events) into different storage backends

License

Notifications You must be signed in to change notification settings

kaltura/jam

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

46 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

What is JaM?

JaM is a PHP monitoring system that supports storing PHP errors (events) into different storage backends.

The events can later be retrieved from backends that implement PHP_JAM_GET_FUNC().

CREDIT line: JaM was initially forked from https://github.com/mkoppanen/php-aware which is no longer an active project.

php-aware was developed by Mikko Koppanen.

What does JaM monitor?

- PHP errors of all levels see [Predefined contants] (http://php.net/manual/en/errorfunc.constants.php) 
- Slow requests
- Peak memory usage during request

How is it better than just using set_error_handler() from my PHP code?

  • set_error_handler() will not capture FATAL errors
  • With set_error_handler(), one needs to change each app so that it calls this method, JaM, since it is a PHP extension, will globablly catch all PHP errors at the Zend Engine level, no matter what app code generated them and no matter which SAPI the code was called from [Apache, CLI, FCGI, etc]

How does it work?

The jam extension overrides Zend's Engine zend_error_cb(), set_error_handler() and restore_error_handler() with a custom function that takes a copy of the current context, sends the error to the backends set in the jam.storage_modules directive and then calls the original error handler(s).

Each backend storage is a separate PHP extension and additional backends can therefore easily be added, see [Creating additional storage backends] (README.md#creating-additional-storage-backends).

The backend will receive a zval * containing information about the current error which it then stores based on it's own configuration.

It is possible to chain the backends to store the event in multiple backends, i.e: send an email and then log to Elasticsearch, for example.

The basic flow is:

  • JaM startup overrides zend_error_cb() with interceptor and stores a pointer to the original callback

  • PHP error happens:

    • Zend Engine calls zend_error_cb() which passes the event to our callback
    • JaM main ext loops through all configured backends and passes the following:
const char *uuid; // uniq ID 
zval *event; // struct containing info about the event 
const char *error_filename; // filename in which the error occured 
long error_lineno; // line in which the error occured 
long type; // error type, see [Predefined Constants] (http://php.net/manual/en/errorfunc.constants.php)
const char *appname; // app identifier string, configured with the jam.appname directive but can be overridden from PHP code
* the backend stores the event as defined in its PHP_JAM_STORE_FUNC()
* call Zend Engine's original error callback 

Basic setup

Debian/Ubuntu and friends

For Ubuntu 14.04 and any other deb based distro that uses php5 = 5.5.n, binary packages are available from Kaltura's CE repo:

# wget -O - http://installrepo.kaltura.org/repo/apt/debian/kaltura-deb.gpg.key|apt-key add -
# echo "deb [arch=amd64] http://installrepo.kaltura.org/repo/apt/debian kajam main" > /etc/apt/sources.list.d/kaltura.list
# aptitude update
# aptitude install php5-jam php5-jam-elasticsearch php5-jam-email php5-jam-files

Then edit /etc/php5/mod-enabled/jam*ini and set directives per your configuration.

To build debs for your deb based distribution, enter the debian dir under the root source dir and run:

$ dpkg-buildpackage -b -uc

RHEL/CentOS and friends

The JaM extensions are available from Kaltura's CE repo. To add the RHEL/CentOS 6 repo, add this to /etc/yum.repos.d/jam.list:

[Kaltura]
name = Kaltura Server
baseurl = http://installrepo.origin.kaltura.org/releases/latest/RPMS/$basearch/
gpgkey = http://installrepo.origin.kaltura.org/releases/RPM-GPG-KEY-kaltura
gpgcheck = 1
enabled = 1

To add the RHEL/CentOS 7 repo, add this to /etc/yum.repos.d/jam.list:

[Kaltura]
name = Kaltura Server
baseurl = http://installrepo.origin.kaltura.org/rhel7/latest/RPMS/$basearch/
gpgkey = http://installrepo.origin.kaltura.org/releases/RPM-GPG-KEY-kaltura
gpgcheck = 1
enabled = 1

To install the main JaM ext:

# yum install php-jam

To install the backends:

# yum install php-jam-email php-jam-elasticsearchi php-jam-snmp php-jam-files

Then set proper values in /etc/php.d/jam*.ini and reload Apache if needed.

If you use a different distro that supports RPM, you can build the packages from php-jam.spec:

$ rpmbuild -bb --with email --with files --with snmp --with elasticsearch php-jam.spec

Compiling from source

Compiling the JaM extension

JaM depends on libuuid, make sure you install the relevant headers and shared objects for it.

On deb based systems:

# aptitude install uuid-dev

On RHEL based systems:

# yum install libuuid-devel

Other Linux and Unices distros should also have it in one format or another.

$ cd /path/to/jam/root/dir
$ phpize
$ ./configure
$ make
# make install

See [Core INI settings] (README.md#core-ini-settings) for the available directives.

Compiling backend extensions

By itself JaM will do pretty much nothing for you, next, select the backends you are interested in and cd into their dir under storage, for instance, if you are interested in the elasticsearch backend:

$ cd storage/elasticsearch
$ phpize
$ ./configure
$ make
# make install

See [elasticsearch->INI settings] (README.md#ini-settings) for the relevant directives.

All available backends are under the storage dir, config and build instructions are the same for all.

Core INI settings

Name Type Description Mode
jam.enabled boolean enable JaM (Default: On) PHP_INI_SYSTEM
jam.use_cache boolean Use serialization cache (Default: On) PHP_INI_PERDIR
jam.error_reporting integer Error reporting level (which events are stored) PHP_INI_PERDIR
jam.module_error_reporting string Override error reporting on backend module basis (Format: elasticsearch=E_ALL,email=E_ERROR) PHP_INI_PERDIR
jam.depth integer How many levels to serialize PHP_INI_PERDIR
jam.log_get boolean Whether to include _GET values in the serialized event PHP_INI_PERDIR
jam.log_post boolean Whether to include _POST values in the serialized event PHP_INI_PERDIR
jam.log_session boolean Whether to include _SESSION values in the serialized event PHP_INI_PERDIR
jam.log_cookie boolean Whether to include _COOKIE values in the serialized event PHP_INI_PERDIR
jam.log_env boolean Whether to include _ENV values in the serialized event PHP_INI_PERDIR
jam.log_server boolean Whether to include _SERVER values in the serialized event PHP_INI_PERDIR
jam.log_files boolean Whether to include _FILES values in the serialized event PHP_INI_PERDIR
jam.log_backtrace boolean Whether to include backtrace in the serialized event PHP_INI_PERDIR
jam.enable_event_trigger boolean Whether to log events generated with jam_event_trigger PHP_INI_PERDIR
jam.storage_modules string Comma separated list of storage backend modules to enable (i.e jam.storage_modules="elasticsearch,email") PHP_INI_PERDIR
jam.slow_request_threshold integer Setting > 0 activates slow request monitor (milliseconds) PHP_INI_PERDIR
jam.memory_usage_threshold integer Setting > 0 activates memory usage monitor (bytes) PHP_INI_PERDIR
jam.error_page string Error page filename. This page is displayed in case of a fatal error when display_errors is off PHP_INI_PERDIR
jam.appname string report the appname in which the err was triggered PHP_INI_ALL

PHP functions:

  • jam_event_trigger(int error_level, string message)

    Trigger an event. The event gets sent into configured storage backends but the internal error handler is not invoked.

    You can use this from your PHP code to send messages to the backend storage module.

  • jam_event_get(string mod_name, string uuid)

    Get an event from storage backend module. Supported in 'files' and 'tokyo' backends.

  • jam_event_get_list(string mod_name[, int start, int limit])

    Returns a list of events from the storage backend module. Supported in 'files' and 'tokyo' backends.

  • jam_event_delete(string mod_name, string uuid)

    Deletes an event from storage backend module. Supported in 'files' and 'tokyo' backends.

  • jam_storage_module_list()

    Returns a list of currently configured storage backend modules.

Storage backends

elasticsearch

Uses JSON-C and CURL libs to send an event to an ElasticSearch server.

NOTE: Because it basically just sends a JSON with the event info using CURL, it can be used to send event to any URL, not only to an elasticsearch server. You can therefore use it to send events to any other system you may have.

Bootstrapping

JaM stores its event under the 'jam' index. See [jam_mappings.json] (jam_mappings.json)

To create initial fields mappings for it, use:

$ curl -X PUT 'jam_elasticsearch.host:9200/jam' -d @jam_mappings.json

Assuning the [default jam_mappings.json] (jam_mappings.json) is used, then the value for the jam_elasticsearch.host should be:

jam_elasticsearch.host="http://YOUR_ES_HOST:9200/jam/log"

Ini settings

Name Type Description Mode
jam_elasticsearch.host String The ElasticSearch URL to send the event to PHP_INI_SYSTEM
jam_elasticsearch.timeout Int The maximum number of seconds to allow cURL functions to execute. PHP_INI_SYSTEM

email

Sends an email containing information about the error

Ini settings

Name Type Description Mode
jam_email.to_address String Email recipient address. For example [email protected] PHP_INI_PERDIR

snmp

Sends the event as an SNMP trap

Ini settings

Name Type Description Mode
jam_snmp.trap_host String hostname:port of the snmptrapd PHP_INI_SYSTEM
jam_snmp.trap_community String snmp community for the trap PHP_INI_PERDIR
jam_snmp.trap_oid String OID for the trap PHP_INI_PERDIR
jam_snmp.name_oid String OID for holding the script name PHP_INI_PERDIR
jam_snmp.error_msg_oid String OID for holding the error message PHP_INI_PERDIR
jam_snmp.uuid_oid String OID for holding the uuid PHP_INI_PERDIR

files

Stores the information in files

Ini settings

Name Type Description Mode
jam_files.storage_path String Path to store the events to PHP_INI_PERDIR

spread

Sends the event to a spread network

This product uses software developed by Spread Concepts LLC for use in the Spread toolkit. 
For more information about Spread see http://www.spread.org

Ini settings

jam_spread.spread_name String port@hostname format of the spread daemon to connect to
jam_spread.group_name String In which group to send the message to
jam_spread.user_name String Username of the sender, must be unique to the machine

stomp

Sends the event to stomp message queue

Ini settings

jam_stomp.server_uri String Uri of the server in tcp://hostname:port format
jam_stomp.queue_name String Name of the queue to send the message to
jam_stomp.username String Username to the queue (optional)
jam_stomp.password String Password to the queue (optional)

tokyo

Stores into tokyo cabinet or tokyo tyrant

Ini settings

jam_tokyo.backend String Can be 'cabinet' or 'tyrant'

Tokyo Tyrant specific ini-settings

jam_tokyo.tyrant_host String Hostname if 'tyrant' is chosen as the backend
jam_tokyo.tyrant_port Integer Port if 'tyrant' is chosen as the backend

Tokyo Cabinet specific ini-settings

jam_tokyo.cabinet_file String Location of the cabinet file if cabinet is chosen
jam_tokyo.cabinet_block Boolean Set to 'on' to use non-blocking locks

zeromq2

Sends events to zeromq2

Ini settings

jam_zeromq2.dsn String Where to connect the publisher socket (Default: tcp://127.0.0.1:5555)
jam_zeromq2.topic String Topic to publish the messages in (Default: jam)

Creating additional storage backends

  • run:
./storage/create_backend_ext_skeleton.php <new-backend-ext-name>     

This will create a dir with all needed skeleton files for you.

  • In, yourext/jam_yourext.c, implement the needed functions for your backend, minimum is to implement PHP_JAM_STORE_FUNC(yourext), other functions are not mandatory.
  • If your extension has any directives, add them to PHP_INI_BEGIN() and init them in php_jam_yourext_init_globals()
  • If your extension depends on additional C/C libs, edit yourext/config.m4 accordingly.
  • run the standard PHP configuration and build commands, i.e:
$ phpize
$ ./configure
$ make
# make install

About

JaM is a PHP monitoring system that supports storing PHP errors (events) into different storage backends

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C 91.2%
  • M4 6.4%
  • PHP 1.8%
  • Shell 0.6%