模式识别中英文

The Science of Pattern Recognition

Achievements and Perspectives

Robert P.W. Duin1 and El

˙

zbieta P_

ekalska2

1 ICT group, Faculty of Electr.

Eng., Mathematics and Computer Science

Delft

University

of

Technology, The

Netherlands

@

2

School

of

Computer

Science,

University of Manchester,

United

Kingdom

pekalska@

Summary.

Automatic pattern recognition is usually considered as an

engineering area which focusses on the development and evaluation of

systems that imitate or assist humans in their ability of recognizing

patterns. It may, however, also be considered as a science that studies

the faculty of human beings (and possibly other biological systems) to

discover, distinguish, characterize patterns in their environment and

accordingly identify new observations. The engineering approach to

pattern recognition is in this view an attempt to build systems that

simulate this phenomenon. By doing that, scientific understanding is

gained of what is needed in order to recognize patterns, in general.

Like

in

any

science

understanding

can

be

built

from

different,

sometimes

even

opposite

viewpoints.

We

will

therefore

introduce

the

main

approaches

to

the

science

of

pattern

recognition

as

two

dichotomies

of

complementary

scenarios.

They

give

rise

to

four

different

schools,

roughly

defined

under

the terms of expert systems, neural networks, structural pattern

recognition and statistical pattern recognition.

We will briefly describe what has been achieved by these schools, what

is common and what is specific, which limitations are encountered and

which perspectives arise for the future. Finally, we will focus on the

challenges

facing

pattern

recognition

in

the

decennia

to

come.

They

mainly

deal with weaker assumptions of the models to make the corresponding

procedures for learning and recognition wider applicable. In addition,

new formalisms need to be developed.

Introduction

We

are

very

familiar

with

the

human

ability

of

pattern

recognition.

Since

our early years we have been able to recognize voices, faces, animals,

fruits or inanimate objects. Before the speaking faculty is developed,

an

object

like

a

ball

is

recognized,

even

if

it

barely

resembles

the

balls

seen before. So, except for the memory, the skills of abstraction and

generalization

are

essential

to

find

our

way

in

the

world.

In

later

years

we

are

able

to

deal

with

much

more

complex

patterns

that

may

not

directly

be based on sensorial observations.

For

example,

we

can

observe

the

underlying

theme

in

a

discussion

or

subtle

patterns in human relations. The latter may become apparent, e.g. only

by

listening

to

somebody’s

complaints

about

his

personal

problems

at

work

that

again

occur

in

a

completely

new

job. Without

a

direct

participation

in the

events, we are able to see both analogy and similarity in examples as

complex as social interaction between people. Here, we learn to

distinguish the pattern from just two examples.

The pattern recognition ability may also be found in other biological

systems:the cat knows

the

way home,

the

dog recognizes his

boss from the

footsteps

or

the

bee

finds

the

delicious

flower.

In

these

examples

a

direct

connection can be made to sensory experiences. Memory alone is

insufficient; an important role is that of generalization from

observations which are similar,although not identical to the previous

ones. A scientific challenge is to find out how this may work.

Scientific questions may be approached by building models and, more

explicitly,

by

creating

simulators,

i.e.

artificial

systems

that

roughly

exhibit

the

same

phenomenon

as

the

object

under

study.

Understanding

will

be

gained

while

constructing

such

a

system

and

evaluating

it

with

respect

to

the

real

object.

Such

systems

may

be

used

to

replace

the

original

ones

and may even improve some of their properties. On the other hand, they

may

also

perform

worse

in

other

aspects.

For

instance,

planes

fly

faster

than

birds

but

are

far

from

being

autonomous.

We

should

realize,

however,

that what is studied in this case may not be the bird itself, but more

importantly, the ability to fly.

Much can be learned about flying in an attempt to imitate the bird, but

also when differentiating from its exact behavior or appearance. By

constructing fixed wings instead of freely movable ones, the insight in

how to fly grows.

Finally, there are engineering aspects that may gradually deviate from

the

original

scientific

question.

These

are

concerned

with

how

to

fly

for

a long time, with heavy loads, or by making less noise, and slowly shift

the point of attention to other domains of knowledge.

The

above

shows

that

a

distinction

can

be

made

between

the

scientific

study

of pattern recognition as the ability to abstract and generalize from

observations and the applied technical area of the design of artificial

pattern recognition devices without neglecting the fact that they may

highly profit from each other. Note that patterns can be distinguished

on many levels,starting from simple characteristics of structural

elements like strokes, through features of an individual towards a set

of

qualities

in

a

group

of

individuals,to

a

composite

of

traits

of

concepts

and their possible generalizations. A pattern may also denote a single

individual as a representative for its population, model or concept.

Pattern recognition deals, therefore, with patterns,

regularities,characteristics

or

qualities

that

can

be

discussed

on

a

low

level of sensory measurements (such as pixels in an image) as well as on

a high level of the derived and meaningful concepts (such as faces in

images).

In

this

work,

we

will

focus

on

the

scientific

aspects,

i.e.

what

we

know

about

the

way

pattern

recognition

works

and,

especially,

what

can

be learned from our attempts to build artificial recognition devices.

A number of authors have already discussed the science of pattern

recognition based on their simulation and modeling attempts. One of the

first, in the beginning of the sixties, was Sayre [64], who presented a

philosophical study on perception, pattern recognition and

classification. He made clear that classification is a task that can be

fulfilled with some success, but recognition either happens or not. We

can

stimulate

the

recognition

by

focussing

on

some

aspects

of

the

question.

Although we cannot set out to fully recognize an individual, we can at

least start to classify objects on demand. The way Sayre distinguishes

between recognition and classification is related to the two subfields

discussed in traditional texts on pattern recognition, namely

unsupervised and supervised learning. They fulfill two complementary

tasks. They act as automatic tools in the hand of a scientist who sets

out to find the regularities in nature.

Unsupervised

learning

(also

related

to

exploratory

analysis

or

cluster

analysis)

gives

the

scientist

an

automatic

system

to

indicate

the

presence

of

yet

unspecified

patterns

(regularities)

in

the

observations.

They

have

to

be

confirmed

(verified)

by

him.

Here,

in

the

terms

of

Sayre,

a

pattern

is recognized.

Supervised

learning

is

an

automatic

system

that

verifies

(confirms)the

patterns

described

by

the

scientist

based

on

a

representation

defined

by

him. This is done by an automatic classification followed by an

evaluation.

In

spite

of

Sayre’s

discussion,

the

concepts

of

pattern

recognition

and

classification are still frequently mixed up. In our discussion,

classification is a significant component of the pattern recognition

system,

but

unsupervised

learning

may

also

play

a

role

there.

Typically,

such

a

system

is

first

presented

with

a

set

of

known

objects,

the

training

set, in some convenient representation. Learning relies on finding the

data descriptions such that the system can correctly characterize,

identify

or

classify

novel

examples.

After

appropriate

preprocessing

and

adaptations, various mechanisms are employed to train the entire system

well.

Numerous

models

and

techniques

are

used

and

their

performances

are

evaluated and compared by suitable criteria. If the final goal is

prediction, the findings are validated by applying the best model to

unseen data. If the final goal is characterization, the findings may be

validated by complexity of organization (relations between objects) as

well as by interpretability of the results.

Fig. 1 shows the three main stages of pattern recognition systems:

Representation,

Generalization

and

Evaluation,

and

an

intermediate

stage

of

Adaptation[20].

The

system

is

trained

and

evaluated

by

a

set

of

examples,

the Design Set. The components are:

?

Design

Set.

It

is

used

both

for

training

and

validating

the

system.

Given the background knowledge, this set has to be chosen such that it

is representative for

the

set of

objects to

be recognized by

the trained

are

various

approaches

how

to

split

it

into

suitable

subsets

for training,validation and testing. See e.g. [22, 32, 62, 77] for

details.

?

Representation.

Real world objects have to be represented in a

formal way in order

to be

analyzed

and compared

by mechanical means

such

as a computer. Moreover, the observations derived from the sensors or

other formal representations have to be integrated with the existing,

explicitly formulated knowledge either on the objects themselves or on

the

class

they

may

belong

to.

The

issue

of

representation

is

an

essential

aspect of pattern recognition and is different from classification. It

largely influences the success of the stages to come.

?

Adaptation.

It

is

an

intermediate

stage

between

Representation

and

Generalization,in

which

representations,

learning

methodology

or

problem

statement are adapted or extended in order to enhance the final

step

may

be

neglected

as

being

transparent,

but

its

role

is may reduce or simplify the representation, or it may

enrich it by emphasizing particular aspects, e.g. by a nonlinear

transformation of features that simplifies the next stage. Background

knowledge may appropriately be (re)formulated and incorporated into a