Monthly Archives: February 2016

Blog, Lab

Graylog Extractor for Fortigate Firewall

I’ve been using Graylog in production for awhile. It’s a great log analysis tool, backed by elasticsearch. Conceptually, graylog is pretty much like splunk. I consolidate approximately 170-200 million log messages to graylog everyday. So, I need to optimize them well enough.

Few days ago, I started to use Fortigate extractors from a git repo. It uses regex, and it is very slow. So I (have to) write my own extractors. I write Grok pattern, and keep rewriting until I could cover all messages I need to extract. Then, I started to optimize the pattern. The result is quite good though. I could reduce extractor time from more than 100,000 usec to about 100 usec.

extractor

Yes, that’s approximately 1000x times faster. It’s definitely not perfect though, but it works for me. If you think it might be good for you, you can download my Fortigate content pack here.

Enjoy :)

Blog, Lab

dnsmasq

dnsmasq is the default DNS resolver if you use NetworkManager. It runs pretty well most of the time. When it doesn’t, you would not be able to access the Internet if you need to resolve names.

I’m not a fan of dnsmasq, and if you – like me – want to disable it, just edit

/etc/NetworkManager/NetworkManager.conf

and remove/comment

dns=dnsmasq

Restart NetworkManager, it will use traditional faithful method of /etc/resolv.conf.

Blog

Quick install graylog on Ubuntu

Yes, yes, I know. There is the official document to guide you, e.g. http://docs.graylog.org/en/2.0/pages/installation/operating_system_packages.html

BUT, RTFM ! You need to install Java, MongoDB, elasticsearch config, and ensure they run well BEFORE you can run Graylog. That would need skills and time to do it right.

For those Ubuntu people, there is another way. Graylog officially (and constantly) releases a single deb package to update the OVA version. This package includes everything you need to run Graylog, so you can use it to deploy Graylog.

You can simply download the package at https://packages.graylog2.org/omnibus, dpkg -i to install, graylog-ctl to configure and run.

DONE.

 

Blog

Remount tmpfs

This is simple, and you can do it without rebooting the machine.

For example, let’s say you have /tmp using tmpfs, and you want to set size to 4 GB:

# mount -o remount,size=4G,noatime /tmp

That’s it.