Use common sense when it comes to outliers

NETWORK WORLD NEWSLETTER: M. E. KABAY ON SECURITY
06/14/05
Today's focus: Use common sense when it comes to outliers

Dear security.world@gmail.com,

In this issue:

* Unusual activity can be a security tip
* Links related to Security
* Featured reader resource
_______________________________________________________________
This newsletter is sponsored by Ciena

Extending disk mirroring and tape backup applications outside of
The data center has become critical to the survivability of
enterprises who need 24x7 access and protection of their
critical enterprise systems. Download this white paper to learn
about MAN and WAN connectivity options and techniques for
reducing operational costs.
http://www.fattail.com/redir/redirect.asp?CID=106337
_______________________________________________________________
Explore your best options in local- and wide-area wireless data
services ...

... discover strategies that implement mobile applications
effectively ... and develop a framework for implementing
enterprisewide mobile-networking. Qualify and you can attend
Wireless & Mobility: Commanding Broadband Everywhere FREE - it's
the new Network World Tech Tour and Expo event coming to *
Boston - June 21 and * Denver June 23.
http://www.fattail.com/redir/redirect.asp?CID=106288
_______________________________________________________________

Today's focus: Use common sense when it comes to outliers

By M. E. Kabay

I recently came across a story in Risks that reminded me that
some computer failures can be spotted using outlier analysis or
even just common sense.

Back in November 2003, Risks contributor Danny Burstein sent in
a report about a medical testing equipment failure that
illustrates a common failing among computer users: not using
common sense.

It seems that "about 3,000 people got opposite results when they
were tested for gonorrhea and chlamydia over an 18-month period.
Because of a faulty diagnostic machine in Cranbrook
(southeastern British Columbia), positive and negative test
results for the two sexually transmitted diseases were
reversed."

Peter Neumann's summary continues, "About 3,000 people were
tested. The 83 that were positive were incorrectly told they
were clean. The 2,900 or so that were negative were told they
were positive and were given the standard treatments."

Burstein and Neumann correctly note, "One Would Have Thought
that someone in the medical office or the lab or the insurance
or the pharmacy or somewhere..., looking at 3,000 test results,
would have quickly noticed that instead of finding a positive
rate of 3% these tests were coming back at 97%." [Risks 23.19]

The case is a reminder that system and network managers must
analyze outliers. Outliers are unusual events.

Examples include the biggest users of network bandwidth, the
user with the highest rate of growth in network disk storage,
the department with the highest number of calls per capita to
the help desk, and the workgroup with the sharpest inflection
point (change in slope) in their total mainframe CPU utilization
growth curve.

In research, it is a truism that once the basic model has been
tested and currently available alternative explanations for
observations have been disproved, the next phase of work is to
analyze "residuals." Residuals are the deviations from
expectations based on the current model. Residuals are the veins
of observation in which we can mine additional insights into
reality.

The people who were processing the reversed data in the Canadian
medical-equipment case should have been interested in the
unusual ratio of infected vs. uninfected patients. Even the
first dozen cases or so should have alerted a responsible
supervisor that there was a problem. For example, if the
expected occurrence rate of infection was normally 3%, the
non-infection rate was 97%. So the likelihood of having 10
uninfected people in the first 10 results would be (0.97)^10 =
74%. Looked at another way, the likelihood of having at least
one infected person in the first group of 10 results would be
26% (1 - 0.74 = 0.26). The likelihood of having two or more
infected results out of 10 would be only 0.72%. (The derivation
is left as an exercise for the reader. Hint: Calculate the
probability of at least one infection out of nine patients and
then multiply the probability that a 10th patient is infected).

So an alert statistician would have seen by the second
"infection" in the series that there was something odd about the
results and possibly saved more than 2,900 people from being
treated for diseases they didn't have - and would have gotten
quicker treatment to people who were really sick.

I remember one Monday morning 20 years ago when I was checking
the weekly status reports for clients at the service bureau
where I was director of technical services in the 1980s. I
notice a sharp inflection in the disk space utilization for one
of our clients over the last week: they were increasing their
usage about 10 times faster than ever before and much faster
than anyone else on the system. Investigation revealed that a
programmer had REMmed (commented) out the PURGE commands for
hundreds of temporary files used in the nightly batch programs
as part of a diagnostic run - and then forgotten to take out the
REMs. There were now thousands of these files accumulating in
the client's account for no good reason, costing them money and
putting our disk capacity at risk. So one simple question,
"What's causing this outlier?" saved us a great deal of trouble.

Don't ignore outliers.

RELATED EDITORIAL LINKS

Outsourced security called battle tested
Network World, 06/13/05
http://www.networkworld.com/nlsec2587

Two cases of lost data shine light on risks
Network World, 06/13/05
http://www.networkworld.com/news/2005/061305-ezboard.html?rl

Tivoli exec tackles security
Network World, 06/13/05
http://www.networkworld.com/news/2005/061305-tivoli.html?rl
_______________________________________________________________
To contact: M. E. Kabay

M. E. Kabay, Ph.D., CISSP, is Associate Professor in the
Division of Business and Management at Norwich University in
Northfield, Vt. Mich can be reached by e-mail
<mailto:mkabay@norwich.edu> and his Web site
<http://www2.norwich.edu/mkabay/index.htm>.

A Master's degree in the management of information assurance in
18 months of study online from a real university - see
<http://www3.norwich.edu/msia>
_______________________________________________________________
This newsletter is sponsored by Concord Communications
Network World Executive Guide: The Evolution of Management
Technologies

Network and systems technologies have an important new role -
helping networked systems live up to new business realities.
With growing and shifting demands, network executives are
balancing business goals with prioritizing IT projects. Read
about the 'Future of Management', 'IT Service Management',
'Managing Security', and 'Best Practices'.
http://www.fattail.com/redir/redirect.asp?CID=106671
_______________________________________________________________
ARCHIVE LINKS

Archive of the Security newsletter:
http://www.networkworld.com/newsletters/sec/index.html

Security Research Center:
http://www.networkworld.com/topics/security.html

Instant sign-up for Security News Alert:
http://www.networkworld.com/isusecna

Instant sign-up for Virus & Bug Patch Alert:
http://www.networkworld.com/isubug
_______________________________________________________________
FEATURED READER RESOURCE
VoIP SECURITY

For the latest in VoIP security, check out NW's Research Center
on this very topic. Here you will find a collection of the
latest news, reviews, product testing results and more all
related to keeping VoIP networks secure. Click here for more:
<http://www.networkworld.com/topics/voip-security.html>
_______________________________________________________________
May We Send You a Free Print Subscription?
You've got the technology snapshot of your choice delivered
at your fingertips each day. Now, extend your knowledge by
receiving 51 FREE issues to our print publication. Apply
today at http://www.subscribenw.com/nl2

International subscribers click here:
http://nww1.com/go/circ_promo.html
_______________________________________________________________
SUBSCRIPTION SERVICES

To subscribe or unsubscribe to any Network World e-mail
newsletters, go to:
<http://www.nwwsubscribe.com/Changes.aspx>

To change your e-mail address, go to:
<http://www.nwwsubscribe.com/ChangeMail.aspx>

Subscription questions? Contact Customer Service by replying to
this message.

This message was sent to: security.world@gmail.com
Please use this address when modifying your subscription.
_______________________________________________________________

Have editorial comments? Write Jeff Caruso, Newsletter Editor,
at: <mailto:jcaruso@nww.com>

Inquiries to: NL Customer Service, Network World, Inc., 118
Turnpike Road, Southborough, MA 01772

For advertising information, write Kevin Normandeau, V.P. of
Online Development, at: <mailto:sponsorships@nwfusion.com>

Copyright Network World, Inc., 2005