DevOps Zone is brought to you in partnership with:

Kris Buytaert is a long time Linux and Open Source Consultant. He's one of instigators of the devops movement, currently working for Inuits He is frequently speaking at, or organizing different international conferences and has written about the same subjects in different Books, Papers and Articles He spends most of his time working on bridging the gap between developers and operations with a strong focus on High Availability, Scalability , Virtualisation and Large Infrastructure Management projects hence trying to build infrastructures that can survive the 10th floor test, better known today as the cloud while actively promoting the devops idea ! Kris is a DZone MVB and is not an employee of DZone and has posted 26 posts at DZone. You can read more from them at their website. View Full User Profile

Graphite, JMXTrans, Ganglia, Logster, Collectd, say what?

01.26.2012
| 8028 views |
  • submit to reddit
Given that @patrickdebois is working on improving data collection I thought it would be a good idea to describe the setup I currently have hacked together.

(Something which can be used as a starting point to improve stuff, and I have to write documentation anyhow)

I currently have 3 sources , and one target, which will eventually expand to at least another target and most probably more sources too.

The 3 sources are basically typical system data which I collect using collectd, However I`m using collectd-carbon from https://github.com/indygreg/collectd-carbon.git to send data to Graphite.

I`m parsing the Apache and Tomcat logfiles with logster , currently sending them only to Graphite, but logster has an option to send them to Ganglia too.

And I`m using JMXTrans to collect JMX data from Java apps that have this data exposed and send it to Graphite. (JMXTrans also comes with a Ganglia target option)

Rather than going in depth over the config it's probably easier to point to a Vagrant box I build https://github.com/KrisBuytaert/vagrant-graphite which brings up a machine that does pretty much all of this on localhost.

Obviously it's still a work in progress and lots of classes will need to be parametrized and cleaned up. But it's a working setup, and not just on my machine ..



Source:  http://www.krisbuytaert.be/blog/graphite-jmxtrans-ganglia-logster-collectd-say-what
Published at DZone with permission of Kris Buytaert, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Timo Lihtinen replied on Wed, 2012/03/14 - 11:50am

Thanks for posting all of this, it's helping me understand a lot about the current state of things. I'm tasked with replacing a Nagios 2.5 install that feeds perfdata to Cacti via custom perl code (yeah, I know).

I understand what every piece is doing for you in your diagram above, except for Ganglia. If you have everything sending metrics to Graphite, why duplicate that to Ganglia? Doesn't Graphite usually supplant Ganglia?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.