INTRUSION DETECTION, INCIDENT RESPONSE, AND INTEGRITY CONTROL
"The early bird gets the worm, but the patient coyote can have them both" (Anonymous)

    Intrusion detection is the art of detecting inappropriate, incorrect, or anomalous activity. Intrusion Detection Systems (IDS) can operate either as a host-based system (on individual computers, this approach being called HIDS) or operate with overall network data flows (along with routers and firewalls, this approach being called network-based IDS or NIDS).  Many organizations use both, placing HIDS software on strategically valuable computers, and placing NIDS extra hardware in a server room or the like.  An IDS may be used for many purposes, and the theory as well as historical background of intrusion detection is both extensive and fascinating (Bace 2000).  Advocates of IDS usually believe it is the best way to detect a security breach, and others just love it to observe many hours of entertaining reconnaissance.  Many firewall products have evolved into IDS products.  Many IT administrators already do IDS-like work by checking security logs for suspicious activities.  Anyone who works with antivirus scanner logs, in effect, does some kind of IDS-like work.  However, IDS work usually classifies what it studies as an "event" (e.g., impersonation, password cracking, protocol attacks, buffer overflows, installation of rootkits, rogue commands, vulnerability exploits, malicious code, data manipulation, unauthorized file access, DoS attacks).  It is also customary to distinguish between "attack" and "misuse" where the term attack is sometimes used to describe malicious intent originating from outside the organization, and misuse is sometimes used to describe an attack that originates from inside the internal network. However, most people don't draw such distinctions, and Bragg et. al. (2004) take real exception to such distinctions, arguing instead that "attack" should strictly refer to when some specially crafted code is involved.  The most common approaches to IDS are statistical anomaly detection and pattern-matching detection (explained shortly), and it should be noted that IDS takes care of almost all kinds of threats to the upper layers (layers 3 to 6) of the network stack according to the OSI model.

    Intrusion detection systems (IDS) do more than just detect.  They can prevent, analyze, entrap, log, and put down the threat in various ways.  Some of them can engage in behavior of questionable legality, such as counterattack (reactive countermeasures) by reverse-infecting the attacking machine, sending a complaint message, or shutting down the attacker's connection to the Internet.  Network-based IDS (NIDS), which are the more popular, can analyze network packets speeding by at gigabit speed.  Sometimes an IDS will lock up with heavy loads (just from logging all the protocol attacks), but the technology is getting better at this, and storage space is a must anyway.  About 50% of NIDS available come in hardware form as an appliance one hooks up in the DMZ or at the border (any choke point on the network will do), and the other 50% of NIDS come in the form of software (which usually but not necessarily needs to run on its own server, an IDS server or set of servers, sometimes affectionately called a Tower of Babel).  A common, freely-available, open-source NIDS called Snort is often hailed as the de facto standard for intrusion detection/prevention.  Snort can perform real-time traffic analysis as well as protocol analysis as well as detect a variety of attacks and probes as well as more.  There are many good books written on Snort (see Printed Resources), and it is a good product for beginners to start with.  There are certification programs out there to be had (see CarnegieMellon's CERT-certified Computer Security Incident Handler), and professional experience with IDS is practically essential in any organization which has a computer security incident response team (CSIRT).  IDS also often allows evidence to be collected in a forensically sound manner which sometimes holds up in court.  Modern IDS systems can produce statistical reports, such as rudimentary correlation and regression.

A REVIEW OF NETWORK PACKET ANALYSIS

    IDS relies upon pre-existing knowledge of packet analysis, specifically the network protocols, like TCP/IP and hundreds of others.  Put simply, data transmissions are sent in packets -- little packages of electronic bits (1's and 0's) framed in various formats as defined by the network protocol in use.  Packets are also called datagrams when they contain payloads intended for use by higher-order applications (e.g., parsed webpages and/or emails containing attachments).  The most common Internet protocol in use is TCP/IP running over Ethernet, and these formats usually mandate that transmission packets be sent in fragments or parts, not as wholes.  When packets arrive at their destination, they are reassembled (if needed) and the content is handed off to the destination application.  Malicious network protocol attacks are quite common.  The IP protocol handles routing of a packet from source to destination, and the TCP protocol ensures reliable delivery.  The IP packet header will usually indicate (via what is called a fragmentation flag) whether the packet is part of a larger group of packets needing reassembly, and all packets, in general, contain a whole lot of header information about networks, devices on the network, addresses on the network, etc.  It is the TCP protocol, however, which establishes the infamous three-way "handshaking session" in the form of SYN, ACK/SYN, ACK, which makes the actual connection and monitors the start, establishment, and disconnection of a communication's session state.  Because of this, TCP is called a "stateful" protocol.  IDS also uses a stateful approach, but allows the administrator to ignore the thousands of relatively uninteresting handshaking signals, as opposed to monitoring/capturing other, more important kinds of information such as deliberately malformed packets intended for specific port numbers like those above 1,024 which, by itself, raises suspicion.  Hackers typically, for example, use port scanners to send an ACK flag without the initial SYN flag, or send a SYN flag without ever sending a FIN flag (which gracefully ends sessions).  These "flag exploits" often get past firewalls, and it is up to an IDS to detect them.  Another type of attack involves fragment reassembly where the hacker will adjust the fragment offset value, which, in effect, tells the destination host exactly how to reassemble the packets.  

    IDS also monitors/captures connectless (or stateless) protocol traffic, such as that using the UDP, DNS, DHCP, and SNMP protocols.  Such traffic ordinarily does not have flags, and packet reassembly is not ordinarily required.  Data communication along these lines are especially appealing to hackers for "fingerprinting" what kinds of devices exist on a network.  Such "probes" are worthy of monitoring since hackers will use all sorts of tricks to try and fool an IDS (see A Look at whisker's anti-IDS tactics).  Higher level applications (like Instant Messaging, chat sessions, and certain popup features or browser plug-ins) which utilize stateless protocol traffic (and/or stateful at the same time) are a favorite target of hackers because the TCP/IP part of the session can look normal, but the UDP part can be manipulated to deliver payloads of malicious content, like the forced installation of spyware, malware, or rootkits.  Encoding schemes also exist, as they do in the world of firewalls.

THE VARIETY OF INTRUSION DETECTION SYSTEMS

    There are two (2) types of IDS and two (2) types of detection models.  The two types of IDS are presented first:

• Host-based IDS -- HIDS can be installed on any server, any workstation, or any network device, and will function like a daemon (background process launched at boot time), and are an excellent choice for application layer monitoring since many of them, like Tripwire will do file integrity checks (sometimes called snapshot or checksum IDS), while others like Cisco and McAfee products will look for and react to suspicious behaviors such as keystroke patterns involving hidden registry or critical binary files, and while others such as Internet Security Systems' most recent incarnation of Blackice will trace intruders back to their source, and yet while others such as Honeyd will set up so-called "honeypots" or "sinkholes" (network equivalents of honeypots) which keep hackers busy for hours figuring things out before they realize they're not on the "real" network, but an imitation or mimicked system.  Honeypots have been controversial in their development, but are seen as a great tool against spammers.   

• Network-based IDS -- NIDS can protect all devices within a network or just the server farm, but to work on all the network, each device must have its network card or driver set to "promiscuous" mode, enabling the sniffing of packet traffic which whizzes by on the fly, including unicast traffic (intended for one device only) and broadcast or multicast traffic (intended for two or more devices).  Snort is a common NIDS, as are Intrusion products, the difference being that with the latter, sensors or taps can be placed on strategic network segments to be monitored.  Sensors or taps are often the only solution around a complex network configuration of switches and routers, and if one can't resolve a switch segmentation problem, one can set the switches to enable port mirroring, or redirection, to the sensor or tap. 

    The two types of detection models are as follows:

• Anomaly detection -- Dorothy Denning's 1987 paper, An Intrusion Detection Model (rtf), started it all, and today, modern AD systems are not only real-time (as Denning wanted) but can monitor the overall health of the Internet.  A statistical baseline is taken of such stats as CPU utilization, disk activity, user logins, and file activity, and then an alert is triggered when a statistically significant deviation occurs from this baseline. The number of events happening within a given time interval are compared to a monitored metric; e.g., someone logging in with an incorrect password too many times.   

• Signature detection -- a upgradeable database is kept of the patterns for every known hacker attack, so that, for example, any string such as the pattern /cgi-bin/phf? triggers an alert.  Signatures are byte sequences that may contain a sample of virus code, a combination of keystrokes, or text indicating probes in certain directories.  Signatures and alert rules are put together in databases called rulesets.

    Signature detection IDS operates on almost the exact opposite of anomaly detection.  It works by using databases or rule sets which contain samples of malicious keystrokes or code which represent suspicious behaviors or patterns.  It is possible to customize the database, as should be done, to offset false positives, along with other tweaking such as using wildcard (*) characters to catch variants where a script-kiddie has just changed a symbol or two in a virus code.  Signature detection IDS is popular among some people (some would say the most popular kind of IDS) because professional groups and newsletters exist which alert to new behaviors and patterns (see IETF working group on IDS).  In fact, all sorts of newsgroups exist where IT administrators talk shop about IDS.  A visit to DShield, for example, is enlightening, where one can see the Internet Storm report on "most wanted" abusive IP addresses.

    Most IDS systems are concerned mainly with remote, external intrusions.  For example, an IDS automatically monitors packets on a network wire and attempts to discover if a break-in or denial of service (DoS) attack is occurring.  A typical setup involves installing a standalone IDS computer system next to the firewall or near the server.  A variety of setup configurations are possible, all designed to mislead intruders.  One can either integrate the IDS with the other machines (services stack) where it can be put in service and will monitor its own traffic, or you can run it independently where it watches all promiscuous traffic.  Promiscuous mode is a common Ethernet technology that companies like 3Com and Cisco make available.  There are generally two more features on an IDS deserving of comment: system integrity verifiers (SIV), which monitor for changes and well known signatures in system files; and log file monitors (LFM), which look for patterns in log files.  An additional feature on most IDS is the ability to set up decoys, honey pots, or do fishbowling to catch intruders.  Some IDS systems can be set to automatically reconfigure firewalls or double-check misconfigured firewalls. 

    One of the first things an IDS does is check your system for any "out of the box" default non-secure modes that exist.  Most systems are shipped to customers with default, easy-to-use configurations. Unfortunately, "easy-to-use" means "easy-to-break-in."  The most common bug in newly released software is the buffer overflow problem, where software programmers have set aside 256 characters to hold a login username.  Surely, the programmer thinks, nobody will have a longer name than that.  But a hacker knows that if they enter a false username longer than that, say 300 characters, including code that will be executed by the program (upon return of an error message or system crash), they can gain access.  How do hackers find this code?  Well, for one thing, the source code for a lot of programs is freely available on the net.  They routinely look thru these for encoded command lines, and there are many such programs.  Next, they try to overflow every place the program has an input field. The buffer overflow problem may have been fixed at the login screen, but there are plenty of other places that might have been overlooked.  Programming fixes to the buffer overflow problem are very difficult in C++, not so difficult in Java.  Almost all application programs are vulnerable to combinations of unexpected input and unexpected commands.

INCIDENT RESPONSE

    Intrusions are any suspicious computer activity.  Incidents are intrusions (attacks) that interrupt or threaten to interrupt normal operating procedures.  Detection is the first reactive step, following preparation, and occurring before a series of six steps that make up a larger process called INCIDENT RESPONSE:

• Preparation -- having policies and procedures in place

• Detection -- becoming aware of suspicious activity and identifying attacker

• Investigation -- containment and collection of digital evidence

• Eradication -- isolation and deletion

• Recovery -- reinstall and reconfiguration

• Follow-up -- reporting, and recording lessons learned

    Detection of suspicious activity can come about thru a variety of means -- service slowdowns or malfunctions, web defacements, anonymous tips, disgruntled employee reports, system audits, or a call to the help desk that says "My computer's being controlled, but not by my keyboard or mouse" -- but usually, the process is automated through one of two methods: firewalls or IDS (Intrusion Detection Systems).  Once the detection phase is over, one should enter the investigation phase, which determines the who, what, when, where, and how of the incident.  Establishing the "who" or identity of the attacker is usually the first step, although some organizations prefer a focus on "how" and how to fix it.  In the response phase, the goal is to isolate-and-contain, so that the attacker is unable to continue compromising the system.  One way of isolating an attacker is to setup a "victim" machine on your network, or on a target subnet, to better monitor the activities of the intruder; i.e., fishbowling or setting up a honey pot.  Recovery can occur at one's own pace, checking subnets, system files, and back doors.  Reporting and recording should be something done throughout every phase of incident response.  Because many of the principles of incident response are drawn from the world of firewalls, a review of those security devices is in order.

FIREWALL FORENSICS

    A firewall is hardware or software that filters the Internet, transferring data you want, and blocking data you don't want.  Firewalls work at the transport (TCP, UDP) and network (IP) layers, and their primary purpose is to block incoming traffic.  This is in contrast to a proxy server, which works at the application (HTTP) layer, and whose primary purpose is to "cache" and re-serve outgoing requests. Some proxy servers can be made to act like firewalls.  Almost all routers can be made to act like firewalls.  Firewalls are your last line of defense. They will keep out the script-kiddies, but not sophisticated hackers.

    Since every data packet contains the address of its sender (the ACK bit), it's very easy for firewalls to identify and block individual senders.  With most web servers, however, it's impractical to block all strangers.  Even if you did try to restrict your web site, a hacker could easily IP spoof their access by impersonating the IP acknowledgement bit of one of your trusted users.  More effective use of a firewall is to set it to analyze data packets in detail and block any activities (e.g. port probes or scans) you decide are suspicious.  You do this by establishing rulesets, by which you can set how lax or "paranoid" you want your firewall to be.  With the exception of only a few personal firewalls and home-use firewall programs, most products allow customized rulesets by using a slider bar that can be set from trusting to paranoid.  Some of these home security programs have features that approximate the power of commercial packages.  A secondary firewall is the name for a program that blocks outgoing traffic, which comes in handy when dealing with insider attacks. 

    Besides rulesets that do the actual blocking, firewalls also provide logging capabilities. Logging is the process by which a firewall stores information about who has attempted access and whether they were let in or turned away.  Computer forensics, or "penetration" specialists commonly analyze firewall logs for patterns of attack or security holes.  Also, it is important to understand that when two computers are talking to one another (connected), they share not only IP addresses (sender-destination) but a pair of PORT numbers.  Ports are pathways where computers accept or reject (open-closed) incoming information.  The destination port number often indicates the type of service being connected to. When a firewall blocks a connection, it will save the destination port number to its logfile.  The following describes some of the meanings attached to port numbers:

• The most well-known ports are those from 0 to 1023. These are tightly bound to services, and any traffic on those ports clearly indicates the protocol for that service.  For example, port 80 always indicates HTTP traffic.

• Registered ports are those from 1024 to 49151. These are loosely bound to services, which means that some services are bound to these ports while others are used for many purposes. Most systems start handing out dynamic ports starting around 1024.

• Dynamic and/or private ports are those from 49152 to 65535. In theory, no services should be assigned to these ports.  [The makers of BlackIce have an excellent Port Numbers Database with hyperlinks about common exploits on each port]

    Just because a firewall detects a suspicious activity doesn't mean an attack is underway.  There are lots of false alarms. The military uses the term firecall to refer to procedures for determining seriousness by assembling a team.  The idea is that collective judgment is better than a single person's judgment.  Once a serious intrusion is detected, a backup is made on any critical systems (if not done already), anti-hacking toolkits are enabled, and personnel are put on alert for any possible social engineering. 

    You will have to learn to use your own judgment in determining what suspicious activities are deserving of closer investigation.  Typical INDICATORS OF AN INTRUSION include:

• an alarm or other indication from your Intrusion Detection System

• suspicious access to UNIX root without going thru normal sequence

• unsuccessful logon attempts

• unexplained, new user accounts

• unfamiliar, new file names

• unexplained modifications to file lengths, or dates, especially in system executable files

• unexplained modification or deletion of data

• denial of service or inability of one or more users to login

• system crashes 

• system slowdown (somebody notices a network service takes longer)

• "door knob rattling" (use of probes, scans, or social engineering)

• unusual time of user usage (early in morning or very late at night)

• unusual user usage patterns (compiling of programs by a user who does not know how to program)

    Blocked data packets by your firewall, however, remain the number one indicator of intrusion. These will typically consist of port scans and probes.  It would take too long to list all the services and exploits on each port, and Gibson Research Corporation has a good listing of all the important ports as well as an excellent "Shields-Up" checker), and a selective few are listed here:

    The steps of Intrusion Detection and Incident Response often go together. In fact, they work well together if you are doing things manually. The following table illustrates this:

INTEGRITY CONTROL

    Integrity control refers to a number of tedious tasks -- backups, change (version) control, patch management, antivirus policies, and redundant systems.  Let's begin with BACKUP, which is the procedure for having storage in case of need for data (or disaster) recovery or system restore.  BACKUPS should not only be for important data and documents, but for configuration files as well.  Most large organizations will separate these into what are called "cold sites" and "hot sites."  What is called a "full" or "archival" backup backs up everything, regardless of whether anything has changed or not.  An "incremental" or "differential" backup stores files that have changed since the last backup, when and where files have had their archiving feature turned off and back on again, indicating a change has been made.  Incremental in this sense means multiple backups are kept; e.g., Sunday, Monday, Tuesday, Wednesday, etc.  A technique known as "Grandfather-Father-Son" makes backups monthly, weekly, and daily.  Another technique, known as the "Tower of Hanoi" uses multiple, different types of media for storage.  Data recovery refers to the process of salvaging data from damaged, failed, wrecked or inaccessible storage media, and typically for forensic purposes.  Disaster recovery is essentially the same as a (business) continuity plan, and may involve some data recovery, but more typically involves purchases of replacement hardware.  System restore refers to a way of rolling back the clock to a time when everything worked right.  Some companies outsource their backup services, and such outsourcing may be done online (e.g., Boomarang) and typically involves some physically offsite location (called data-vaulting or using a remote backup service). 

    Change (version) control, traditionally, has been accomplished via setup of three different environments -- production environment (the secured IT resources people use everyday), test environment (where hardware is tested mainly), and development environment (the place where new, or beta versions of software are tested).  Modern change control procedures require this, and more, such as auditing and documentation.  When a new system is planned to be put into operation, this is called "migration" and when new versions of hardware/software are put in place, this is called "upgrade."  With migration and upgrade, the most encountered problems are going to be incompatibilities.  Evaluation and auditing software exist (such as Ecora) which may assist with these problems if the build-configuration is managed in advance.  If management software is not used, it is vital, nonetheless, that some kind of checklist or forms be used. 

    Patch management is a necessity for all systems.  Unpatched software means that a hotfix, service pack, or critical upgrade has not been installed, and estimates are that 95% of compromises on systems are due to misconfigurations or unpatched systems (Bragg et. al. 2004).  Also, several Internet worms and viruses in the wild are known to take advantage of unpatched systems.  For the personal user, patching is pretty much a simplified process -- one either manually visits Windows Update or uses Automatic Update.  For large organizations, shutdowns may be needed because certain running processes might not patch unless the files are inactive.  Also, large organizations tend to have many more systems, utilities, applications, devices and drivers which need patching.  Keeping up with exactly what needs patching can be a full time job.  Certain websites (like BugTraq, CERT, NTBugTraq, and XForce) help with this, and it may be important to get the proper help because alone (and unassisted), many an IT administrator (as well as user) have been fooled into downloading and installing a Trojan horse pretending to be a patch upgrade.  The idea of using a worm (using evil for good) to handle all the update/patch requirements of the Internet has been advocated by some.

    Antivirus software is only as good as it is recent and updated.  Enterprise-wide solutions exist (see Microsoft List of Antivirus Partners) which can greatly simply the process of trying to keep a network virus-free.  By the way, Microsoft's Security Antivirus Center also posts a rewards program for information leading to the arrest and conviction of whomever released the Blaster, Sobig, and MyDoom viruses.  Opinions vary about how good the rating systems are that antivirus vendors use to rank the severity and prevalence of viruses (most of them are heavily dependent upon what information their customers send them), but just about every vendor has some such rating or alert system.  The Big Three are Symantec, McAfee, and Trend Micro, which together, account for about 89% of the global market in antivirus vending.  Opinions vary, and some people prefer to use one of the many other companies other than the Big Three.

    Redundant systems are usually what the hackers have, what with their RAID (multiple hard drive) configurations, so it makes sense for an organization to have something similar.  The most common solution involves having a "standby system" (either full or partial), where if the main system falls, the standby system kicks in (also called a high availability system).  "Clustering" is a variant of this where the network is segmented or partitioned in some way to enable certain clusters to serve as "active standby" to be routed to when necessary.  There is also a procedure called "hot swapping" where certain hardware components (like drives, memory sticks, and cards) can be replaced without shutting anything down.  More and more computer components nowadays, however, are being built to be fault tolerant, which means they can recover from error quickly or survive in spite of error.

INTERNET RESOURCES
Abuse.net Domain Name Lookup Database
Anton Chuvakin's Blog
Bleeding Edge Snort
CERT Incident Notes
Honeypots, Intrusion Detection and Incident Handling
ISS AdvICE Database
LogAnalysis.org
SANS Intrusion Detection FAQ
SecurityFocus IDS Page
Talisker Intrusion Detection Page
The HoneyNet Project
U.S. Navy Computer Incident Response Guidebook
Wikipedia: Backup
Wikipedia: Data Recovery
Wikipedia: Intrusion Detection System

PRINTED RESOURCES
Amoroso, Edward. (1999). Intrusion Detection. NY: IntrusionNet Books [sample pages]
Bace, R. (2000). Intrusion Detection. Indianapolis: Sams Publishing.
Beale, J. & Caswell, B. (2004). Snort 2.1 Intrusion Detection, 2e. NY: Syngress.
Bejtlich, R. (2004). The Tao of Network Security Monitoring: Beyond Intrusion Detection. Boston: Addison-Wesley.
Bragg, R., Rhodes-Ousley, M. & Strassberg, K. (2004). Network Security: The Complete Reference. NY: McGraw Hill.
Cooper, Mark et al. (2001). Intrusion Signatures and Analysis. NY: New Riders [sample pages]
Escamilla, Terry. (1998). Intrusion Detection: Security Beyond the Firewall. NY: Wiley [sample pages]
Caswell, B., Beale, J. & Baker, A. (2006). Snort Intrusion Detection and Prevention Toolkit. NY: Syngress.
Gerg, C. & Cox, K. (2004). Managing Security with Snort and IDS Tools. Cambridge, MA: O'Reilly Media.
Mandia, K. & Prosise, C. (2001). Incident Response: Investigating Computer Crime. NY: McGraw Hill.
McClure, S., J. Scambray & G. Kurtz. (2005). Hacking Exposed: Network Security Secrets and Solutions. Berkeley: Osborne Press. Companion web site at http://www.hackingexposed.com/
Media, Sylvia et al. (2000). Hack Proofing Your Network. NY: Syngress Media, Inc. [sample pages]
Northcutt, S. & Novak, J. (2002). Network Intrusion Detection, 3e. Indianapolis: Sams Publishing.
Northcutt, S., Zeltser, L., Winters, S., Fredrick, K. & Ritchey, R. (2005). Inside Network Perimeter Security: The Definitive Guide to Firewalls, Virtual Private Networks, Routers, and Intrusion Detection Systems, 2e. Indianapolis: Sams Publishing.
Scott, C., Wolfe, P. & Hayes, B. (2004). Snort for Dummies. Indianapolis: Alpha Books.

Last updated: Aug 13, 2006
Not an official webpage of APSU, copyright restrictions apply, see Megalinks in Criminal Justice
O'Connor, T.  (Date of Last Update at bottom of page). In Part of web cited (Windows name for file at top of browser), MegaLinks in Criminal Justice. Retrieved from http://www.apsu.edu/oconnort/rest of URL accessed on today's date.