Tag Archives: graylog

Blog, Lab

Graylog Extractor for Fortigate Firewall

I’ve been using Graylog in production for awhile. It’s a great log analysis tool, backed by elasticsearch. Conceptually, graylog is pretty much like splunk. I consolidate approximately 170-200 million log messages to graylog everyday. So, I need to optimize them well enough.

Few days ago, I started to use Fortigate extractors from a git repo. It uses regex, and it is very slow. So I (have to) write my own extractors. I write Grok pattern, and keep rewriting until I could cover all messages I need to extract. Then, I started to optimize the pattern. The result is quite good though. I could reduce extractor time from more than 100,000 usec to about 100 usec.

extractor

Yes, that’s approximately 1000x times faster. It’s definitely not perfect though, but it works for me. If you think it might be good for you, you can download my Fortigate content pack here.

Enjoy :)

Blog

Quick install graylog on Ubuntu

Yes, yes, I know. There is the official document to guide you, e.g. http://docs.graylog.org/en/2.0/pages/installation/operating_system_packages.html

BUT, RTFM ! You need to install Java, MongoDB, elasticsearch config, and ensure they run well BEFORE you can run Graylog. That would need skills and time to do it right.

For those Ubuntu people, there is another way. Graylog officially (and constantly) releases a single deb package to update the OVA version. This package includes everything you need to run Graylog, so you can use it to deploy Graylog.

You can simply download the package at https://packages.graylog2.org/omnibus, dpkg -i to install, graylog-ctl to configure and run.

DONE.