django-statsd ========================================= Django Statsd ============= Integration between statsd and django. It allows you to use different clients, sends timings as middleware and integrates with django debug toolbar. Credits: - jbalogh and jsocol for statsd and commonware, which I just ripped parts out of and put in here. - robhudson for django-debug-toolbar Changes ------- 0.3.12: - Event better Django 1.6 support for the patches, with tests. 0.3.11: - Django 1.6 support 0.3.9: - statsd 2.0 support - improved Django debug toolbar support 0.3.8.5: - don't count some 404 as 500 and fix deprecation warnings 0.3.8.4: - gauge support 0.3.8.3: - some bug fixes 0.3.8.1: - add in a tasty pie middleware 0.3.8: - add in a nose plugin 0.3.7: - add in metlog client 0.3.6: - add in log handler 0.3.5: - fix tests to work standalone - add in waterfall view of timings 0.3.3: - fix setup.py to include loggers etc 0.3.2: - update to work with latest Django Debug Toolbar 0.3: - added in logging handler for logging error counts to stats Installation ------------ From pypi:: pip install django-statsd-mozilla Because there is already a django-statsd on pypi. Requirement, https://github.com/jsocol/pystatsd or:: pip install statsd Because there is already a pystatsd on pypi. This will be automatically added when you install django-statsd-mozilla. First off, pick your client, one of: - django_statsd.clients.null This one does nothing, good for development. No point in wasting those UDP packets. - django_statsd.clients.toolbar Use for the django debug toolbar, stores all the statsd pings on the request so they can be used in the toolbar. - django_statsd.clients.normal Use this for production, it just passes through to the real actual pystatsd. - django_statsd.clients.log Just writes the values to a log file using Python's logging module. - django_statsd.clients.moz_metlog Use this to route messages through _metlog: http://github.com/mozilla-services/metlog-py. Note that using metlog will require you to bind the metlog instance to bind the metlog client instance as settings.METLOG. - django_statsd.clients.nose Route messages through to the nose plugin. This also works with the toolbar client, so you don't need to change them on -dev. Usage ----- To send timings from your code, use just like pystatsd, but change your imports to read:: from django_statsd.clients import statsd For example:: from django_statsd.clients import statsd statsd.incr('response.200') Django statsd will choose the client as specified in your config and send the data to it. You can change your client by specifying it in the config, the default is:: STATSD_CLIENT = 'django_statsd.clients.normal' To send timings or counts with every request, add in some middleware:: MIDDLEWARE_CLASSES = ( 'django_statsd.middleware.GraphiteRequestTimingMiddleware', 'django_statsd.middleware.GraphiteMiddleware', ) + MIDDLEWARE_CLASSES If you are using tastypie, you might enjoy:: 'django_statsd.middleware.TastyPieRequestTimingMiddleware' To get timings for your database or your cache, put in some monkeypatches:: STATSD_PATCHES = [ 'django_statsd.patches.db', 'django_statsd.patches.cache', ] Toolbar integration ------------------- Make sure `django_statsd` is installed:: INSTALLED_APPS = ( .. 'django_statsd', ) This will show you the statsd timings in the toolbar:: MIDDLEWARE_CLASSES = ( 'debug_toolbar.middleware.DebugToolbarMiddleware', ) + MIDDLEWARE_CLASSES Note: this must go before the GraphiteMiddleware so that we've got the timing data in before we show the toolbar panel. Add in the panel:: DEBUG_TOOLBAR_PANELS = ( ... 'django_statsd.panel.StatsdPanel', ) Set the client:: STATSD_CLIENT = 'django_statsd.clients.toolbar' Finally if you have production data coming into a graphite server, you can show data from that server. If you have one, link it up:: Here's the configuration we use on AMO. Because triggers and counts go to different spots, you can configure them differently:: TOOLBAR_STATSD = { 'graphite': 'https://graphite-phx.mozilla.org/render/', 'roots': { 'timers': ['stats.timers.addons-dev', 'stats.timers.addons'], 'counts': ['stats.addons-dev', 'stats.addons'] } } The key is added on to the root. So if you've got a key of `view.GET` this would look that up on the graphite server with the key:: stats.addons.view.GET Front end timing integration ---------------------------- New browsers come with an API to provide timing information, see: http://w3c-test.org/webperf/specs/NavigationTiming/ To record this in statsd you need a JavaScript lib on the front end to send data to the server. You then use the server to record the information. This library provides a view to hook that up for different libraries. First, make sure you can record the timings in your Django site urls. This could be done by pointing straight to the view or including the URL for example:: from django_statsd.urls import urlpatterns as statsd_patterns urlpatterns = patterns('', ('^services/timing/', include(statsd_patterns)), ) In this case the URL to the record view will be `/services/timing/record`. Second, hook up the client. There is a un-sophisticated client called `stick` included in the static directory. This requires no configuration on your part, just make sure that the file `django_statsd/static/stick.js` is in your sites JS. Then call it in the following manner:: stick.send('/services/timing/record'); We also include support for `boomerang`, a sophisticated client from Yahoo: http://yahoo.github.com/boomerang To hook this up, first add in boomerang to your site, make sure you use the web timing enabled version, as discussed here: http://yahoo.github.com/boomerang/doc/howtos/howto-9.html When the script is added to your site, add the following JS:: BOOMR.init({ beacon_url: '/services/timing/record' }).addVar('client', 'boomerang'); Once you've installed either boomerang or stick, you'll see the following keys sent:: window.performance.timing.domComplete 5309|ms window.performance.timing.domInteractive 3819|ms window.performance.timing.domLoading 1780|ms window.performance.navigation.redirectCount 0|c window.performance.navigation.type.reload 1|c There's a couple of options with this you can set in settings:: STATSD_RECORD_KEYS (optional) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A list of the keys you want to record, there's quite a few in the timing api and you likely don't want to record them all. Here's the default:: STATSD_RECORD_KEYS = [ 'window.performance.timing.domComplete', 'window.performance.timing.domInteractive', 'window.performance.timing.domLoading', 'window.performance.navigation.redirectCount', 'window.performance.navigation.type', ] Override this to get different ones. STATSD_RECORD_GUARD (optional) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There's only limited ways to stop people posting junk to your URLs. By defining a this a function you can do some work to allow requests to your needs. If the function returns None, the request is allowed through. If you don't want to allow the request, return any valid Django HTTP response. For example to deny everyone not in INTERNAL_IPS:: from django.http import HttpResponseForbidden def internal_only(request): if request.META['REMOTE_ADDR'] not in INTERNAL_IPS: return HttpResponseForbidden() STATSD_RECORD_GUARD = internal_only Logging errors ~~~~~~~~~~~~~~ If you want to log a count of the errors in your application to statsd, you can do this by adding in the handler. For example in your logging configuration:: 'handlers': { 'test_statsd_handler': { 'class': 'django_statsd.loggers.errors.StatsdHandler', }, } Testing ======= You can run tests with the following command: DJANGO_SETTINGS_MODULE='django_statsd.test_settings' nosetests Nose ==== There is also a nose plugin. If you use nose, then run tests, you'll get output in your tests. To use run tests with the following:: --with-statsd Contributors ~~~~~~~~~~~~ * streeter * crankycoder * glogiotatidis * tominsam * youngbob * jsatt * youngbob * jsocol * janfabry * tomchristie * diox * frewsxcv * fud * ftobia * jawnb * fgallina See: https://github.com/andymckay/django-statsd/pulls?direction=desc&page=1&sort=created&state=closed Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search`