SEC - simple event correlator

Introduction

SEC is an event correlation tool for advanced event processing which can be harnessed for event log monitoring, for network and security management, for fraud detection, and for any other task which involves event correlation. Event correlation is a procedure where a stream of events is processed, in order to detect (and act on) certain event groups that occur within predefined time windows.

Many traditional event log management systems store events in a database and execute database queries for implementing event correlation. However, such systems are heavyweight solutions and often involve a complex database infrastructure on dedicated hardware.

In contrast, SEC is a lightweight and platform-independent event correlator which runs as a single process. The user can start it as a daemon, employ it in shell pipelines, execute it interactively in a terminal, run many SEC processes simultaneously for different tasks, and use it in a wide variety of other ways.

SEC reads lines from files, named pipes, or standard input, matches the lines with patterns (like regular expressions or Perl subroutines) for recognizing input events, and correlates events according to the rules in its configuration file(s). SEC can produce output by executing external programs (e.g., snmptrap or mail), by writing to files, by sending data to TCP and UDP based servers, by calling precompiled Perl subroutines, etc.

Documentation

If you are new to SEC, "Simple Event Correlator Tutorial" provides a gentle introduction to SEC. Rule examples from the tutorial are also available in tutorial repository.

See SEC manpage for official documentation. The manpage provides an in-depth discussion of all SEC features and presents a number of detailed rule examples.

Also, the SEC FAQ provides answers to some frequently asked questions.

Other reading

For additional information, you can check the following sources:

Here you can find the list of papers from more distant past that are relevant for SEC-2.6 or older (some solutions from these papers can be implemented more efficiently with more recent SEC versions):

Installation information and dependencies

SEC has been primarily tested on Linux and Solaris, but since it is written in Perl and does not use any platform-dependent code, it should work on any modern UNIX platform. SEC is also known to work on Windows, although some of the functionality that is native to UNIX will be disabled.

In order to install SEC, check where your Perl executable is located and change the first line in the sec file accordingly. For example, if your Perl executable is /usr/local/bin/perl, set the first line to #!/usr/local/bin/perl -w. Then copy sec and sec.man (SEC manpage) to appropriate directories, e.g.,
cp sec /usr/local/bin
cp sec.man /usr/local/man/man1/sec.1

Since SEC is not tested against ancient Perl releases, it is recommended to run SEC with at least Perl 5.8 (see https://www.perl.org for the latest stable Perl release). Apart from Perl, SEC does not depend on other software. It uses Perl Getopt, POSIX, Fcntl, Socket, IO::Handle, Sys::Syslog, and JSON::PP modules which are included in the standard installation of Perl (the presence of Sys::Syslog and JSON::PP is optional).

Mailing list

There is a mailing list for SEC users. The purpose of this list is to facilitate discussion between SEC users, so that you can ask questions from more experienced users and share your experience with others. Before posting a question, please search the list - there are some questions that get asked quite frequently and you might find several answers in the mailing list archive.

Download

 sec-2.9.3.tar.gz (November 1 2024)

... or you can visit SEC download pages at GitHub or Sourceforge (have also older versions available).

SEC has also been packaged for a number of Linux and BSD distributions. You can try the following links for finding a package for your platform:
- Fedora and RHEL packages at Koji
- RPM package search at rpmfind.net
- Debian package info
- Ubuntu package info
- SLE and openSUSE package info
- Gentoo package info
- Arch Linux package info
- OpenBSD package info
- FreeBSD package info
(if any of the links are broken, please contact the author).

Author

Risto Vaarandi (firstname d0t lastname at gmail d0t c0m)

Please don't contact the author with SEC usage questions - you should post such questions to the SEC mailing list.

Academic attribution

When you publish research results that have involved the use of SEC, please provide a reference to the following paper in your publication:

Risto Vaarandi, Bernhards Blumbergs and Emin Caliskan, "Simple Event Correlator - Best Practices for Creating Scalable Configurations," Proceedings of the 2015 IEEE CogSIMA Conference, pp. 96-100, 2015

Acknowledgments

The author is grateful to SEB Estonia for supporting this work.

The author also thanks the following people for supplying software patches, documentation fixes, and suggesting new features: Al Sorrell, Brian Mielke, David Lang, James Brown, Jon Frazier, Mark D. Nagel, Peter Eckel, Rick Casey, and William Gertz.

Last but not least, the author expresses his profound gratitude to John P. Rouillard for many great ideas and creative discussions that have helped to develop SEC.