-
D
A
R
P
A
I
N
< br>T
R
U
S
I
O
N
D
E
T
E
C
T
I
O
N
E
V
A
L
U
A
T
< br>I
O
N
DARPA Intrusion Detection Evaluation
Intrusion detection systems monitor
network state looking for unauthorized usage,
denial of service, and anomalous
behavior.
Such systems have never been
formally evaluated ... until now.
The
Cyber Security and Information
Sciences
Group (formerly the
Information Systems
Technology Group)
of MIT Lincoln Laboratory, under Defense Advanced
Research Projects
Agency
(
DARPA
ITO
)
and
Air
Force
Research
Laboratory
(AFRL/SNHS)
sponsorship,
has
collected
and
distributed
the
first
standard
corpora
for
evaluation
of
computer
network
intrusion
detection
systems.
We
have
also
coordinated,
with
the
Air
Force
Research
Laboratory, the first formal,
repeatable, and statistically significant
evaluations of intrusion
detection
systems. Such evaluation efforts have been carried
out in 1998 and 1999. These
evaluations
measure probability of detection and probability
of false alarm for each system
under
test.
These evaluations are
contributing significantly to the intrusion
detection research field
by providing
direction for research efforts and an objective
calibration of the current technical
state of the art. They are of interest
to all researchers working on the general problem
of
workstation
and
network
intrusion
detection.
The
evaluation
is
designed
to
be
simple,
to
focus
on
core
technology
issues,
and
to
encourage
the
widest
possible
participation
by
eliminating security and
privacy concerns and providing data types that are
used commonly
by the majority of
intrusion detection systems.
?
?
?
?
Data Sets
?
Documentation
?
Publications
?
Evaluation
Results
?
美国国防部高级研究计划局入侵检测评估
入侵检测系统监视网络状态，发现未经授权使用、拒绝服务和反常的行为。
直到现在，这个系统从来没有正式评估
……
。
麻省理工学院的林肯实验室的网络安全与信息科学组织
(
原信息系统技术组织
),
< br>在国防
高级研究计划局
(DARPA
ITO)
和空军研究实验室
(AFRL
/
SNHS)
赞助之下
,
收集和分发计算
机网络入侵检测系统的第一次标准
评价。
我们也跟空军研究实验室协调了第一个正式的、
可
重复的
,
统计上显著的入侵检测系统评价。这
样的评价工作已经在
1998
年和
19
99
年实现。
在此测试下，这些评估能够测量检测率和误告警率
。
这些评估对入侵检测研究领域贡献显著，
< br>为研究工作提供了方向，
并且客观校准了当前
技术状态。
他们对所有致力于解决工作站和网络入侵检测存在的普遍问题的研究人员感兴趣。
评价是设计简单
,
专注于核心技术问题
,
并鼓励尽可能广泛的参与通过消除安全与隐私问题
< br>和提供数据类型通常使用的大多数入侵检测系统。
1999 DARPA Intrusion Detection
Evaluation Data Set
1999 DARPA
Intrusion Detection Evaluation Data Set Overview
There were two parts to the 1999 DARPA
Intrusion Detection Evaluation: an off-line
evaluation and a realtime evaluation.
Intrusion detection systems were tested
in the off-line evaluation using network traffic
and audit logs collected on a
simulation network. The systems processed this
data in batch
mode and attempted to
identify attack sessions in the midst of normal
activities.
Intrusion detection systems
were delivered to AFRL for the realtime
evaluation. These
systems
were
inserted
into
the
AFRL
network
testbed
and
attempted
to
identify
attack
sessions in the midst of normal
activities, in realtime.
Intrusion
detection systems were tested as part of the off-
line evaluation, the realtime
evaluation or both.
Training
Data
Three weeks of training data were
provided for the 1999 DARPA Intrusion Detection
off-line evaluation.
The
first and third weeks of the training data
do not
contain any attacks.
This data was
provided to facilitate
the training of anomaly detection systems.
The second week of the training data
contains a select subset of attacks from the 1998
evaluation in addition to several new
attacks. The primary purpose in presenting these
attacks was to provide examples of how
to report attacks that are detected.
Note:
In 1999, Intrusion
detection systems were trained using the data from
both the 1998
and the 1999 evaluations.
The following files are provided for
each day in the training set:
?
?
?
?
?
?
?
Outside
sniffing data ( Tcpdump format )
Inside
sniffing data ( Tcpdump format )
BSM
audit data ( From pascal )
NT audit
data ( From hume )
Long listings of
directory trees ( From pascal, marx, zeno, and
hume )
Dumps of selected directories (
From pascal, marx, zeno, and hume )
A
Report of file system inode information ( From
pascal )
BSM Configuration
[tar/gzip]
First Week
of
Training Data (Attack Free)
Second
Week
of Training Data (Contains Labled
Attacks)
Third Week
of
Training Data (Attack Free)
Testing
Data
Two weeks of network based attacks
in the midst of normal background data. The forth
and fifth weeks of data are the
10/1/1999.
There
are
201
instances
of
about
56
types
of
attacks
distributed
throughout
these two weeks.
Further
information about the attack instances, where they
are located in week 4 and
5 data is
found in the
Documentation
page.
Fourth Week
of Test
Data
Fifth Week
of Test
Data
1999 Training Data - Week 1
The simulation network normally
collected data twenty-two hours a day. The
tcpslice
program was used to examine
the outside tcpdump data files and the actual
times of the first
and
last
packet
were
extracted.
These
times
are
shown
below.
During
the
first
week
of
training data the simulation network
did not experience any unscheduled down time.
First Packet
Time
Mon
Mar 1
08:00:02
Tue
Mar
2
08:00:02
Last
Packet Time
Mar 2
06:00:02
Tue
Wed
Mar 3
06:00:01
Thu
Mar 4
06:00:01
Fri
Sat
Mar 5
06:00:02
Mar 6
06:00:02
Wed
Mar 3
08:00:03
Thu
Mar 4
08:00:03
Fri
Mar 5
08:00:02
Monday
outside tcpdump
data
inside tcpdump
data
159,432 kb gzipped
165,264 kb gzipped
Solaris
BSM audit data
NT audit
data
5,559 kb gzipped
4,501
kb tarred &gzipped
Selected directory
dumps
3,118 kb tarred &gzipped
File system listing & inode
record
6,851 kb tarred
&gzipped
Tuesday
outside
tcpdump data
inside tcpdump
data
155,620 kb gzipped
163,009 kb gzipped
Solaris
BSM audit data
NT audit
data
2,869 kb gzipped
82,712
kb tarred &gzipped
Selected directory
dumps
2,482 kb tarred &gzipped
File system listing & inode
record
6,347 kb tarred
&gzipped
Wednesday
outside
tcpdump data
inside tcpdump
data
180,585 kb gzipped
186,631 kb gzipped
Solaris
BSM audit data
NT audit
data
2,238 kb gzipped
306 kb
tarred &gzipped
Selected directory
dumps
3,057 kb tarred &gzipped
File system listing & inode
record
6,358 kb tarred
&gzipped
Thursday
outside
tcpdump data
inside tcpdump
data
Solaris BSM audit data
NT
audit data
Selected directory
dumps
File system listing & inode
record
Friday
outside tcpdump data
inside
tcpdump data
Solaris BSM audit
data
NT audit data
Selected
directory dumps
File system listing &
inode record
Errata.
None.
244,683 kb
gzipped
267,904 kb gzipped
2,789 kb gzipped
2,012 kb
tarred &gzipped
3,116 kb tarred
&gzipped
6,288 kb tarred &gzipped
141,562 kb gzipped
150,301
kb gzipped
2,074 kb gzipped
10,546 kb tarred &gzipped
3,117 kb tarred &gzipped
2,732 kb tarred &gzipped
1999 Training Data - Week 2
The simulation network normally
collected data twenty-two hours a day. The
tcpslice
program was used to examine
the outside tcpdump data files and the actual
times of the first
and last packet were
extracted. These times are shown below. During the
second week of
training data the
simulation network was brought down early ( 3:00
AM ) during Tuesday's
run for extended
unscheduled maintenance.
First Packet Time
Last Packet
Time
Mon
Mar 8
08:00:01
Tue
Tue
Mar 9
08:00:01
Wed
Wed
Mar 10
08:00:03
Thu
Thu
Mar 11
08:00:03
Fri
Fri
Mar 12
08:00:02
Sat
Monday
outside
tcpdump data
inside tcpdump
data
Solaris BSM audit data
NT
audit data
Selected directory
dumps
File system listing & inode
record
Tuesday
outside tcpdump data
inside
tcpdump data
Solaris BSM audit
data
NT audit data
Selected
directory dumps
File system listing &
inode record
Wednesday
outside tcpdump data
inside
tcpdump data
Solaris BSM audit
data
NT audit data
Selected
directory dumps
File system listing &
inode record
Mar 9
06:00:49
Mar 10
02:59:59
Mar 11
06:00:01
Mar 12
06:00:00
Mar 13
06:00:00
167,536 kb gzipped
185,368
kb gzipped
2,789 kb gzipped
10,482 kb tarred &gzipped
3,269 kb tarred &gzipped
10,597 kb tarred &gzipped
196,205 kb gzipped
206,995
kb gzipped
3,086 kb gzipped
10,481 kb tarred &gzipped
2,966 kb tarred &gzipped
6,991 kb tarred &gzipped
68,267 kb gzipped
78,986 kb
gzipped
3,182 kb gzipped
10,481 kb tarred &gzipped
3,416 kb tarred &gzipped
6,993 kb tarred &gzipped
Thursday
outside tcpdump
data
inside tcpdump
data
165,445 kb gzipped
172,389 kb gzipped
Solaris
BSM audit data
NT audit
data
5,884 kb gzipped
2,497
kb tarred &gzipped
Selected directory
dumps
3,315 kb tarred &gzipped
File system listing & inode
record
6,563 kb tarred
&gzipped
Friday
outside
tcpdump data
inside tcpdump
data
142,087 kb gzipped
151,229 kb gzipped
Solaris
BSM audit data
NT audit
data
3,875 kb gzipped
440 kb
tarred &gzipped
Selected directory
dumps
3,442 kb tarred &gzipped
File system listing & inode
record
6,944 kb tarred
&gzipped
Errata.
None.
1999 DARPA Intrusion Detection
Evaluation
The official guidelines for
the 1999 DARPA evaluation. Numerous things were
changed
from the 1998 evaluation.
?
?
?
?
?
?
Evaluation
Schedule
Off Line
1999 Evaluation Plan
Labeled
Attacks from 2nd Week of Training Data
Off-line Simulation Network
[
GIF
]
[
PPT
]
List
of Simulation Network
Hosts (Names and IP addresses)
Other
documents about the 1999 evaluation are available.
A Summary of the 1998 Evaluation with a
Brief Outline of Changes for the 1999
Evaluation is available in
PDF Format
.
A
table of stealthy U2R attack
instances
, showing how each attack
instance was made
to be stealthy with
respect to the network sniffer based Intrusion
Detection systems.
An
attack
database
is now available online. This
attack taxonomy is based on the
1998 -
1999 training data and incorporates attack
descriptions from Kris Kendall's thesis. The
database includes attacks considered
The Master's Thesis of Kris Kendall
contains descriptions of all the attacks used in
the
1998 evaluation and a useful
taxonomy of attacks. The thesis is available on
the publications
page.
Detection Scoring Truth
-
List of all attack instances in the 1999 test
data.
Identification Scoring
Truth
- Identification alert entries
for all attack instances in the
1999
test data.
1999 Analysis of Windows NT
Attacks
In early 2000 work was done to
further analyze the detect-ability of all attacks
run
against the Windows NT host in the
1999 Windows NT event log auditing test data. We
have
compiled a table of all such
attacks and the detection results in 1999 and
provided a perl
script that
automatically locates the specific implementations
of these attacks used in 1999.
?
?
?
T
able
of NT
attack instances and detection results in 1999.
A
Perl script
for
locating the 1999 NT attacks in the audit logs.
List
of hosts and operating
systems used in this scenario.
2000
Dataset One
Future Evaluations and
Datasets
Plans for future Intrusion
Detection Evaluations have been discussed. A
significant effort
is being made to
step-back and ensure that evaluations of intrusion
detection technology are
appropriately
designed
and
scaled
to
respond
to
the
needs
of
DARPA
and
the
research
community. Early stages of planning
were carried out in the Spring of 2000. Those
helping
with
this
early
planning
included
DARPA,
Principle
Investigators
in
the
DARPA
Strategic
Information
Assurance
(SIA)
program,
the
Sandia
Red-Team,
and
Lincoln
Laboratory.
A
planning workshop, entitled the
Wisconsin. Slides from the Wisconsin
meeting are available on a Schafer website.
The outcome of this meeting was that in
the current year, Lincoln Laboratory was
tasked to produce much needed off-line
intrusion detection datasets. These datasets will
provide researchers with extensive
examples of attacks and background traffic. The
Hawaii PI
meeting presentation given at
the SIA PI meeting gives the goals of and a
detailed plan for
producing the 2000
datasets.