Hi,
I'm Tom.

I'm a Python web developer living in Shanghai. This is my personal website and blog.

Blog  ·  Github  ·  Twitter
Recent (Full archive →)
  1. Build a simple protocol over TCP
  2. Cassandra: A Journey of Upgrade
  3. Ramble on Java & Session
  4. Cassandra: Create a cluster on your local machine
  5. Simple Guide to Install StatsD and Graphite

Site designed by @orourkedesign.

Simple Guide to Install StatsD and Graphite

I’ve been playing around with StatD and Graphite lately. It took me quite a while to set this stack up. So I think it worthwhile to write a post to walk through the whole installing and configuring process for future reference.

Here is a checklist of all the softwares I’m gonna use:

I’m setting up all these stuff inside a Ubuntu precise32 Vagrant box. If you are also using a Vagrant box, add these settings in your Vagrant file.

1
2
 config.vm.network :forwarded_port, guest: 80, host: 8080
 config.vm.network :forwarded_port, guest: 8125, host: 8125, protocol: 'udp'

Let’s ssh into vagrant and change to root first

1
2
 vagrant ssh
 sudo su -

Install Graphite

1
2
3
4
5
6
 apt-get install git python-virtualenv python-dev
 virtualenv /opt/graphite
 source /opt/graphite/bin/activate
 pip install https://github.com/graphite-project/ceres/tarball/master
 pip install whisper
 pip install carbon

Install Graphite Web

1
2
3
4
5
6
7
8
 apt-get install libcairo2-dev
 cd /opt/graphite
 git clone https://github.com/graphite-project/graphite-web.git
 cd graphite-web
 git checkout 0.9.12
 python setup.py install
 pip install -r requirements.txt
 django-admin.py syncdb --settings=graphite.settings --pythonpath=/opt/graphite/webapp

Graphite includes a wsgi file in its installation. Just copy it for later deployment

1
 cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/webapp/wsgi.py

Configure Carbon

1
2
3
4
5
6
7
 cd /opt/graphite/conf/
 cp carbon.conf.example carbon.conf
 cat > storage-schemas.conf << EOF
[stats]
pattern = ^stats.*
retentions = 10s:6h,1min:6d,10min:1800d
EOF

The storage schema is copied from StatsD. Tweak it to meet your needs.

Install statsd

1
2
3
4
5
6
7
8
9
10
11
12
 apt-get install nodejs
 cd /opt/
 git clone https://github.com/etsy/statsd.git
 cat > statsd/config.js << EOF
{
  graphitePort: 2003
, graphiteHost: "127.0.0.1"
, port: 8125
, backends: [ "./backends/graphite" ]
, legacyNamespace: false
}
EOF

Install Grafana

1
2
3
 cd /opt/
 git clone https://github.com/grafana/grafana.git
 cp grafana/src/config.sample.js grafana/src/config.js

Change Permissions

1
2
 adduser tom
 chown tom:tom -R /opt/graphite /opt/statsd /opt/grafana/

Manage process with Supervisord

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
 apt-get install supervisor
 cat > /etc/supervisor/conf.d/gunicorn.conf << EOF
[program:gunicorn]
command = /opt/graphite/bin/gunicorn -b 127.0.0.1:8080 -w 2 --pythonpath /opt/graphite/webapp/ wsgi:application
directory = /opt/graphite/webapp/
user = tom
autostart = true
autorestart = true
redirect_stderr = true
EOF
 cat > /etc/supervisor/conf.d/statsd.conf << EOF
[program:statsd]
command = /usr/bin/node stats.js config.js
directory = /opt/statsd/
user = tom
autostart = true
autorestart = true
redirect_stderr = true
EOF
 cat > /etc/supervisor/conf.d/carbon.conf << EOF
[program:carbon]
command = /opt/graphite/bin/carbon-cache.py start --debug
user = tom
autostart = true
autorestart = true
redirect_stderr = true
EOF
 supervisorctl reload

Set up Nginx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
server {
        listen   80;

        location / {
                add_header Access-Control-Allow-Origin "*";
                proxy_pass http://127.0.0.1:8080;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location /grafana/ {
                alias /opt/grafana/src/;
                index index.html;
        }
}
1
2
3
4
 apt-get install nginx
 ln -s /etc/nginx/sites-available/graphite /etc/nginx/sites-enabled/
 rm /etc/nginx/sites-enabled/default
 /etc/init.d/nginx restart

Now go to http://127.0.0.1:8080/ for Graphite and http://127.0.0.1:8080/grafana/ for Grafana.