Examining statistical process control as a method of temporal data mining
by Brown, Herbert Earle Mathias, Ph.D., NOVA SOUTHEASTERN UNIVERSITY, 2007, 135 pages; 3259566

Abstract:

A methodology based on statistical process control was examined for the data mining problem of anomaly detection. This methodology does not suffer from many of the limitations of other data mining techniques often proposed for anomaly detection. This research demonstrated statistical process control has sound theoretical backing, has a linear time complexity, is accurate in classifying anomalies, and is able to identify novel information. Furthermore, it was shown that the contemporaneous use of numerous univariate statistical process control charts can address the prevalent problem of class imbalance. This research found that statistical process control based techniques are an effective method of temporal anomaly detection.

Statistical process control based algorithms were developed and tested on a food industry complaint database of both frequent and infrequent Poisson distributed events. In applications of statistical process control, Shewhart charts are often used in conjunction with either exponential weighted moving average or cumulative sum charts for detecting both large and small shifts in a process quickly. This research compared exponentially weighted moving average charts and cumulative sum charts, each used with Shewhart charts, for the purpose of data mining. For the trial database considered, the cumulative sum based method was preferred finding significantly more events of interest.

Considerations for the design, setup, and maintenance of statistical process control based anomaly detection algorithms were also examined. The relationships between a confusion matrix, often used for binary classification, and typical measures of statistical process control performance, e.g. average run length and average time to signal, were derived. In addition, a general process of adapting statistical process control charts for a data mining task was developed.

 
AdviserSumitra Mukherjee
SchoolNOVA SOUTHEASTERN UNIVERSITY
SourceDAI/B 68-04, p. , Aug 2007
Source TypeDissertation
SubjectsStatistics; Computer science
Publication Number3259566
Adobe PDF Access the complete dissertation:
 

» Find an electronic copy at your library.
  Use the link below to access a full citation record of this graduate work:
  http://gateway.proquest.com/openurl%3furl_ver=Z39.88-2004%26res_dat=xri:pqdiss%26rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation%26rft_dat=xri:pqdiss:3259566
  If your library subscribes to the ProQuest Dissertations & Theses (PQDT) database, you may be entitled to a free electronic version of this graduate work. If not, you will have the option to purchase one, and access a 24 page preview for free (if available).

About ProQuest Dissertations & Theses
With over 2.3 million records, the ProQuest Dissertations & Theses (PQDT) database is the most comprehensive collection of dissertations and theses in the world. It is the database of record for graduate research.

The database includes citations of graduate works ranging from the first U.S. dissertation, accepted in 1861, to those accepted as recently as last semester. Of the 2.3 million graduate works included in the database, ProQuest offers more than 1.9 million in full text formats. Of those, over 860,000 are available in PDF format. More than 60,000 dissertations and theses are added to the database each year.

If you have questions, please feel free to visit the ProQuest Web site - http://www.proquest.com - or call ProQuest Hotline Customer Support at 1-800-521-3042.