Digital Signal Processing 17 (2007) 617–633
www.elsevier.com/locate/dsp
Feasibility of impact-acoustic emissions for
detection of damaged wheat kernels
Tom C. Pearson a,∗, A. Enis Cetin b, Ahmed H. Tewfik c,
Ron P. Haff d
a USDA-ARS, Manhattan, KS, USA
b Bilkent University, Ankara, Turkey
c University of Minnesota, Minneapolis, MN, USA
d USDA-ARS, Albany, CA, USA
Available online 2 September 2005
Abstract
A non-destructive, real time device was developed to detect insect damage, sprout damage, and
scab damage in kernels of wheat. Kernels are impacted onto a steel plate and the resulting acoustic
signal analyzed to detect damage. The acoustic signal was processed using four different methods:
modeling of the signal in the time-domain, computing time-domain signal variances and maximums
in short-time windows, analysis of the frequency spectrum magnitudes, and analysis of a derivative
spectrum. Features were used as inputs to a stepwise discriminant analysis routine, which selected a
small subset of features for accurate classification using a neural network. For a network presented
with only insect damaged kernels (IDK) with exit holes and undamaged kernels, 87% of the former
and 98% of the latter were correctly classified. It was also possible to distinguish undamaged, IDK,
sprout-damaged, and scab-damaged kernels.
© 2005 Elsevier Inc. All rights reserved.
Keywords: Neural network; Spectral analysis; Insect damage kernels; Sorting; Acoustic emissions
* Corresponding author. Fax: +785 537 5550.
E-mail address: tpearson@gmprc.ksu.edu (T.C. Pearson).
1051-2004/$ – see front matter © 2005 Elsevier Inc. All rights reserved.
doi:10.1016/j.dsp.2005.08.002
618 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–6331. Introduction
In most years, many types of wheat kernel damage occur in limited quantities from
isolated growing areas. Three common types of damage that impact flour quality and/or
safety are fungal damage, sprout damage, and insect damage. Several fungi can infect
kernels before and after harvest. The most important of these is Fusarium graminearum,
which creates “scab” damage and may lead to toxins known to cause cancer [1]. Scab-
damaged kernels generally have a white, chalky appearance covering most of the exterior
surface [2]. Sprout damage occurs when the moisture content at harvest time is too high,
causing germination of the seed. The result is a sprout shoot emerging from the kernel
and/or a discolored germ. Enzymes which convert starch to sugar are activated during
sprouting and can impact bread quality if present in the flour. Internal insect infestation
degrades the quality and value of wheat and is one of the most difficult defects to detect.
Infestation causes grain loss by consumption, contaminates the grain with excrement and
fragments, causes nutritional losses, and degrades end-use quality of flour [3]. Levels of
insect infestation are therefore a major factor in the grading of wheat quality. USDA stan-
dards allow up to 31 insect damaged kernels (IDK) per 100 grams (about 3000 kernels) of
wheat for grades U.S. No. 1–U.S. No. 5 [4]. Samples with more than 31 instances of IDK
are graded as U.S. sample grade. Food and Drug Administration standards consider wheat
infested at the same limit of 32 or more instances of IDK [5]. However, wheat millers usu-
ally specify that wheat loads must contain less than five IDK per 100 g. Inspecting for IDK
is labor intensive and may overlook most of the infested kernels where an immature insect
has not emerged from the kernel.
Wheat kernels become infested when an adult female bores a small hole (roughly
0.05 mm) into the kernel, deposits an egg, and seals the hole with a mucilaginous plug.
The plug is the same color as the wheat surface, making it difficult to detect by external ex-
amination. When the egg hatches, the larvae develops and feeds, forming tunnels inside the
kernel. Upon maturity, the insect exits the kernel, leaving an exit hole. Inspectors counting
IDK in a sample look for this exit hole. However, kernels which contribute the most to
reduced flour quality are those containing immature insects, which are generally not de-
tected by inspectors. These kernels containing hidden infestations are not part of the IDK
classification. Not only do these insects mature and lay more eggs, allowing the infestation
to spread, but also contribute to the insect fragments in the flour.
Sprout, scab, and IDK are identified in the grading process by manual inspection of
small samples of grain. However, normal procedures for handling and transporting lead to
mixing of damaged with undamaged kernels. Hence, the ratio between the two is gener-
ally small, requiring large samples for damage to be accurately quantified. To date, rapid
and economical methods to quantify IDK and hidden insect infestations in wheat sam-
ples have not been developed. The need for rapid inspection of individual kernels makes
development of such a method problematic. Because the occurrence of IDK and hidden
infestations is very small as a fraction of the total number of kernels to be evaluated, a low
false-positive error rate (error in classifying good kernels as damaged) is essential to avoid
discarding large amounts of good product. A variety of methods have been attempted to
detect internal insect damage, and others are under development. These include cracking
and flotation [6], X-ray imaging [7–9], acoustic detection of larval movement and chewing
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 619Fig. 1. Schematic of pistachio sorter based on acoustic emissions.
[10–13], carbon dioxide measurement [14], and immunological methods [15]. However,
most of these methods have only achieved limited implementation either because they are
slow, labor intensive, expensive, and can only detect specific insect species, or they cannot
quantitatively measure insect infestation levels. More recently, an image-analysis program
has been developed to automatically scan X-ray images for insect infestation [7]. Other
researchers have investigated use of NIR spectroscopy to detect hidden insects in wheat
kernels [16,17]. Finally, a digital signal-processing method has been developed to work in
conjunction with a Perten single-kernel characterization system (SKCS) to detect insect-
damaged kernels [18]. This method further processes the crush force signal produced by
the SKCS as it crushes wheat kernels. As expected, insect damaged kernels produce differ-
ent crush force signals than un-damaged kernels. X-ray, NIR methods, and SKCS methods
suffer from high false-positive error rates and are cost prohibitive for many commercial
applications. Thus, no economically viable equipment utilizing these technologies has be-
come available for detection of internal insects.
Sprout, scab, and insect damage may be detectable by impact acoustic emissions since
the physical and/or structural properties of these kernels are different compared with un-
damaged kernels. Sprouting causes conversion of starch to sugars, and emergence of the
shoot is accompanied by splitting of the kernel. Fungi infecting scab-damaged kernels can
make the endosperm porous and the kernel shrink and lose weight [19]. Finally, the insect
tunnels alter the structure of infested kernels. Impact-acoustic emissions has been success-
fully used as the basis for a device that separates pistachio nuts with closed shells from
those with split shells [20–22]. In that system, shown in Fig. 1, pistachio nuts impact a
steel plate and the resulting acoustic emissions are analyzed in real time. When a closed
shell is detected, an air valve is activated and the nut diverted. Classification accuracy is ap-
proximately 97%, with a throughput rate near 40 nuts per second. The device is economical
(approximately $5000), reliable in a food processing environment, and requires minimal
620 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633maintenance or expertise. Simple features such as amplitude of the sound and decay rate
of the amplitude are extracted from the acoustic signal as a basis for sorting.
1.1. Non-destructive techniques using acoustic emissions
Acoustic-emission techniques are widely used to monitor structures and machine parts,
especially bearings and gears, to determine if failure is imminent [23]. Common signal
processing includes thresholding, counting points above or below a threshold, modeling
signals in the time-domain with damped sinusoids, using a combination of techniques,
such as covariance analysis, principal component analysis, differential timing analysis and
neural networks to detect and classify acoustic emissions in the presence of strong back-
ground sound signals [24] and [25], and detection of repetitive pulses [26]. Monitoring of
machine parts usually allows capture and analysis of several periods of a repeated signal.
For example, certain types of slow-moving bearings emit a “tick” on every rotation when
failure is imminent. For sorting of agricultural products in real time on a processing line,
there is limited time (a few milliseconds) to acquire and process data, classify the object,
and trigger a rejection mechanism.
Historically, the fields of oil exploration and seismic activity due to earthquakes have
produced the most extensive research of acoustical signal processing methods. In the case
of oil exploration, a charge is detonated and the acoustic signals of echoes at plate bound-
aries and oil reservoir surfaces are analyzed [27]. Early studies in this field led to the
development of the cepstrum, defined as the inverse discrete Fourier transform (DFT) (or
inverse discrete cosine transform) of the log DFT of the time domain signal [28]. The
cepstrum characterizes variations in the spectrum due to overtones of the underlying funda-
mental resonating frequency. Portions of the cepstrum are now commonly used as feature
vectors for representing human speech and musical signals. Cetin et al. [22] developed
an improved classification scheme for the pistachio-sorting device developed by Pearson
[20] through use of cepstrum computations. The sound-feature vector, used to distinguish
closed from split shells, consists of mel-cepstral coefficients along with principal compo-
nents of the magnitude of acoustic-impact emission signals. This technique is common in
speech-recognition systems [21].
Other acoustical methods have been studied to detect firmness in fruits. Several re-
searchers have found a good correlation between fruit firmness and resonant frequency
[29,30]. These acoustical systems typically involve tapping the food with a plunger, record-
ing the sound generated, and digitally processing the signal to extract dominant frequency
bands or other features correlated with firmness. Younce and Davis [31] developed such a
system to measure firmness of cherries using impact acoustics. Sugiyama et al. [32] devel-
oped an acoustical firmness tester for melons that measured sound-transmission velocity.
This technique eliminated some error caused by size and shape variations among the fruits.
While these methods are able to measure certain fruit properties, they are not readily adapt-
able to high-speed inspection. A more likely candidate for high-speed inspection of wheat
would be to capture and analyze the signal generated when the kernels are dropped and
impact a steel plate, as in the case of the pistachio-sorting device described above.
The objective of this study was to demonstrate the feasibility of using signal processing
algorithms to separate damaged and undamaged wheat kernels impacting a steel plate at
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 621high speed in real time. Emphasis was given to detecting IDK and hidden infestations,
which are among the more difficult types of damage to quantify. The overall goal of the
project was to detect, classify, and remove contaminated and damaged kernels in real time
using the sound generated when a kernel strikes a steel plate (impact acoustics) as the basis
for sorting. This mode of acoustic excitation is easily adapted to high-throughput sorting
systems.
2. Materials and methods
Rice Weevils [Sitophilus oryzae (L)], a common stored-grain insect, were reared in jars
of hard red winter (HRW) wheat. The wheat, grown in central Kansas, Oklahoma, and
Colorado in 2001 and 2002 was a mixture of Pioneer 2137, 2163, 2174, Karl and Jagger
varieties. Selected kernels were frozen to kill the insects and larvae, held at room temper-
ature for six weeks to allow moisture to equilibrate (10.5%), then X-rayed on film at an
exposure of 18 kV, 3 mA, for 2 min exposure. Kernels with interior insect damage at the pu-
pal stage were identified by examination of the X-ray films. Pupae were differentiated from
less mature larvae by their larger size and the presence of a snout and/or limbs. These im-
ages represent hidden insects at the most advanced stage before exiting the kernels. Since
the immediate objective of this study was to demonstrate feasibility, less mature infesta-
tions were not included. Figure 2 shows a radiograph of undamaged kernels (left), kernels
with pupae present (center), and kernels in which the adult insect has exited the kernel
(IDK) (right). Wheat from the same source was hydrated for three days to allow germina-
tion to begin and cause sprout damage. Scab-damaged HRW kernels of unknown variety
were provided by the Federal Grain Inspection Service (FGIS). From the wheat sources
described, 900 undamaged kernels were collected along with 500 each of pupae-stage in-
fested kernels, IDK, scab-damaged, and sprout-damaged kernels. This gave a total of 2900
kernels, 2000 damaged and 900 undamaged, to be used for classification experiments.
2.1. Experimental apparatus
A schematic of the experimental apparatus for singulating wheat kernels, dropping them
onto the impact plate, then collecting the acoustic emissions from the impact is shown in
Fig. 2. Radiographs of wheat kernels with different stages of insect damage. From left to right, undamaged
kernels, kernels with damage at the pupal stage, and IDK where the insect has matured and exited the kernel.
622 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633Fig. 3. Schematic of experimental apparatus for collecting acoustic emissions from wheat kernels.
Fig. 3. The bulk kernels were transformed into a single-file stream using a vibration feeder
(F-TO-C, FMC-Syntron, Homer City, PA), which consisted of a vibrating bulk hopper
and a steel, V-shaped, trough. The impact plate was a polished block of stainless steel
approximately 7.5×5.0×10 cm. The mass of a single wheat kernel is negligible compared
to the mass of the impact plate, hence the possibility of vibrations from the plate interfering
with acoustic emissions from kernels was minimized. The drop distance from the feeder
to the impact plate was 40 cm and the plate was inclined at 30◦ above the horizontal.
This angle was determined by trial and error. Flatter angles of incline produced a stronger
signal but kernels would tend to bounce twice before falling off of the plate. This was not
conducive to high speed sorting. The angle of 30◦ was found to produce adequate signal
and project the kernel off of the plate after the first bounce. Larger drop distances also
produce a stronger signal; however, during larger drops, the kernel trajectory would not
be as consistent. For example, kernels would fall into a one centimeter diameter spot for
a 40 cm drop but that increased to 3 cm for an 80 cm drop. To reliably divert kernels, the
location where they drop needs to be consistent so a drop of 40 cm was selected.
Two microphones (4939 L with 2669 L pre-amp, Bruel and Kjaer North America, Nor-
cross, GA), sensitive to frequencies up to 100 kHz, were used to capture audible and
ultrasonic acoustic emissions from the impact of the wheat kernels with the steel plate.
The end of each microphone was placed 25 mm from the point where kernels impacted
the plate. Kernels were usually aligned lengthwise before dropping off of the end. How-
ever, during dropping, they often tumbled so their orientation was quite random when they
striked the impact plate. This caused a great discrepancy in the amplitude of the acoustic
signals emitted. Preliminary experiments showed that an order of magnitude difference
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 623in the intensity of the acoustic emissions among kernels was common. Thus, one micro-
phone was amplified (2690 NEXAS, Bruel and Kjaer North America, Norcross, GA) at
1 V/Pa and the other at 10 V/Pa to ensure the system had the required dynamic range
to capture the acoustic emissions. Microphone signals were digitized using a sound card
(Waveterminal 192X, Ego Sys, Seoul, South Korea) at a sampling frequency of 192 kHz
with 16-bit resolution. This sound card does not have a 20 kHz low pass filter that most
consumer sound cards do use. The data acquisition was triggered using an optical sen-
sor (QS30LLPC, Banner Engineering Corp. Minneapolis, MN). Upon receipt of a trigger
signal, the computer acquired 2000 data points from the microphone signal. A high-pass
filter was applied to the acquired signal using a single-pole recursive filter with a cutoff
frequency of 9600 kHz. This filtering effectively eliminated 60 Hz noise, any DC offset,
and eliminated effects from audible ambient sounds.
2.2. Data processing
Features from the 10 V/Pa-amplified microphone signal were extracted if less than six
sampled points were saturated; otherwise, features extracted from the 1 V/Pa-amplified
microphone signal were used. The signal processing and feature extraction were identical
for each of the two amplified signals. Extracted features, along with the microphone gain,
were presented to a neural network for classification. Four different methods were used for
the feature extraction: (1) modeling of the signal in the time-domain; (2) computing time-
domain signal variances in short-time windows; (3) analysis of the frequency spectrum
magnitudes; and (4) differential spectral analysis. Discussion of each method follows.
2.2.1. Time-domain signal modeling
Typical signals from an undamaged kernel and IDK are shown in Fig. 4. The maxi-
mum amplitude of the signals are quite variable and are not useful for distinguishing IDK
from undamaged kernels. However, IDK’s have a consistent tendency to resonate for a
longer time than undamaged kernels. This phenomenon is due to the insect tunnel within
the body of the kernel. Similar acoustical response have been observed while grinding dif-
Fig. 4. Typical signals from an undamaged kernel (left) and IDK (right). Note that duration of the IDK signal is
longer.
624 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633Fig. 5. Maximum-value filtered signals fitted with a Weibull curve for an undamaged kernel (left) and IDK (right).
ferent types of harder and softer wheat kernels [33]. Preliminary attempts to use damped
sinusoids [26] to model the signals were not successful for distinguishing IDK and un-
damaged kernels. IDK acoustic emissions often, but not always, changes from an initially
high frequency to a lower frequency before the signal diminishes. Thus, modeling with
a damped sinusoid often showed that IDK signals diminished more rapidly than they ac-
tually did. Signal modeling was performed using the following three-step transformation:
(1) rectifying the signal by taking the absolute value at all points; (2) non-linear filtering
of the signal by replacing the center data point with the maximum value in a seven-point
window in order to accentuate the peak in each wave period; and (3) non-linear estimation
of the four parameters of the Weibull function, given by Eq. (1), which has a shape similar
to that of the processed time-domain signal.
Y =
cb
a
[
(x − x0)
a
](b−1)
{
e−[
(x−x0)
a ]
b}
, (1)
where x > x0, otherwise Y = 0.
Parameters of the Weibull equation, a, b, c and x0, were iteratively estimated by a non-
linear regression dynamic-link library (DLL) (NLREG, Phil Sherod, Brentwood, TN). Y is
the magnitude of the Weibull function at data point x. Parameter a controls the width of the
curve while b controls the shape; c reflects the magnitude of the signal; and x0 determines
where the Weibull curve begins. For values of b approaching 3.5, the curve approximates
a Gaussian; when b is near 2, the curve approximates a log-normal curve; and when b
approaches 1, the curve approximates an exponential decay. Thus, parameters a and b are
potentially useful features for distinguishing IDK from undamaged kernels irregardless of
the signal magnitude. Additionally, the proportion of variation explained by the model,
R2, can also aid in the identification of IDK. Figure 5 shows sample signals that have been
processed as described above, and Table 1 shows average parameter estimates.
2.2.2. Short-time variance window processing
Variances and maximums of the signals were computed in short-time windows of the
high-pass filtered time domain signal. While the Weibull function captures the shape of
the recorded signal globally, the short-time variance information models the local time-
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 625Table 1
Average Weibull parameter estimates
Parameter Undamaged Insect-damaged (IDK)
Avg Std dev Avg Std dev
a 15.01 8.06 27.99 19.37
b 2.75 1.19 1.59 0.78
R2 0.85 0.17 0.80 0.13
Fig. 6. Average normalized variances from short-time windows of time-domain signals.
domain variations in the signal. For variance computations, the short-time windows were
50 points in duration and incremented in steps of 30 points so that each window overlapped
by 20 points. The first window began 40 points in front of the maximum-signal magnitude.
A total of eight short-time windows were computed to cover the entire duration of all
signals. After all variances were computed, they were normalized by the sum of all eight
variances as described by the equation
σ 2ni =
σ 2i
∑8
i=1 σ
2
i
, (2)
where σ 2ni and σ 2i are the normalized and computed variances from window i with i = 1
being the first window and i = 8 being the last. This method captures the increased duration
of the signals from insect-damaged kernels. Figure 6 shows the average normalized vari-
ance for IDK and undamaged kernels. IDK and undamaged kernels can be distinguished
by the fact that the normalized variance of the third window is greater for IDK than for
undamaged kernels, and the slope between the third and fourth normalized variance differs
between the two.
For maximum-magnitude computations, the short-time window consisted of ten points
with no overlap between the windows. Five windows were used, the first beginning at the
maximum magnitude for the entire signal. As in Wiebull modeling, this computation relates
to the duration of the signal ringing, but it also captures more localized variations than in
626 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633Fig. 7. Maximum rectified signal magnitudes of 10 point, short-time windows after the global signal maximum
magnitude. Note the difference between IDK and undamaged kernels in the fourth window.
the Wiebul model. For this computation, the maximum magnitude of the rectified signal
was computed and recorded, producing five points that describe the general magnitude of
the signal in the 50 points immediately following the signal maximum magnitude. Figure 7
shows the average of these magnitudes for undamaged kernels and IDK. As found for the
short-time variance windows, the values differ between the third and fourth windows, with
the value of the magnitude increasing substantially for undamaged kernels. Otherwise,
maximum magnitudes for undamaged and IDK are very similar.
2.2.3. Frequency domain processing
A 256-point DFT was computed from each signal using a Hamming window. The 256-
point window started 80 data points before the signal maximum slope, corresponding to
the moment of impact, and covered the duration of the sound generated by the impact.
The magnitude of each spectrum was computed, and a 20-tap FIR low-pass filter was
applied to remove jagged spikes in the spectrum. The filter cutoff frequency was π/4 in
the normalized DFT domain. Figure 8 shows the frequency spectrum of an IDK, revealing
a sharper peak at a lower frequency than for undamaged kernels. It should be noted that the
DFT of the two kernels shown in Fig. 8 are obviously quite different but it emphasizes the
major difference found in the DFT of IDK and undamaged kernels. Higher resolution DFTs
have shown that IDK in fact have two peaks. However, this feature is often lost in noise.
Peaks of the spectra of undamaged kernels and IDK are clearly distinguishable. However,
there are significant overlaps in many cases, which are mainly due to the tumbling of
the wheat kernel as it strikes the metal plate. The frequency corresponding to the peak
magnitude in the frequency spectrum was saved as a potential discriminating feature. In
addition, the 15 magnitude values before the peak and 15 points after the peak were saved
and normalized to the peak magnitude. For undamaged kernels, the average frequency
corresponding to the peak DFT magnitude was 34.5 kHz with a standard deviation of
9.1 kHz. In contrast, IDK had an average peak frequency of 26.3 kHz with a standard
deviation of 9.6 kHz.
2.2.4. Differential spectral analysis
Preliminary testing using cepestral coefficients and mel-cepstral coefficients as theywere applied to pistachio impact emissions by Cetin et al. [22] did not add any discrimi-
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 627Fig. 8. Example frequency-spectrum magnitudes for an undamaged kernel and IDK.
Fig. 9. Average magnitudes of the differential spectrum. Note the undamaged kernels have two distinct peaks
compared with one for IDK, emphasizing the smaller oscillations observed in spectra from undamaged kernels
of the time-domain signal.
nating power to the other features discussed thus far. As noted earlier, spectra from IDK
had a tendency for one or two sharper peaks. In contrast, spectra from undamaged kernels
tend to have one larger peak and many smaller oscillations between the frequencies of 33
and 81 kHz. Thus, the first derivative of the DFT was computed between these frequencies
to emphasize these smaller oscillations and help deconvolute other peaks. This is a tech-
nique commonly used in near-infrared spectroscopy to extract peaks from smoothed data
[34]. The derivative of the DFT magnitude, comprised of the 64 points between 33 and
81 kHz, was computed numerically using equation 3 followed by a 64-point DFT (on the
resulting derivative spectrum) to produce 32 magnitude values of the differential spectrum.
The magnitudes of this spectrum are shown in Fig. 9.
S′n = Sn−1 − Sn+1, (3)
where Sn is the magnitude of the spectrum at frequency index n. The computation was per-
formed for all n between 43 and 107, which corresponded to 33 and 81 kHz, respectively.
628 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–6332.2.5. Classification
The following features were pooled together as potential discriminating features:
Weibull curve-fit parameters a, b, and R2; normalized variances from the eight short-time
windows; maximum values from five short-time windows; the frequency corresponding to
the peak DFT magnitude; 30 normalized DFT magnitudes centered about the frequency
corresponding to the peak DFT magnitude; the entire 128 DFT magnitudes; and the 32
magnitudes of the differential spectrum. Stepwise discriminant analysis (significance for
entry and exit = 0.1) was used to select a maximum of 20 of these features for classification
by a neural network. This relatively low number of features was used to guard against the
possibility of over-fitting the neural network. As an additional guard against over-fitting,
the data sets were evenly divided into three randomly selected data sets henceforth called
training, testing, and validation. Feature selection with stepwise discriminant analysis [35],
followed by neural network training, was performed on the training set. Training was
stopped when classification error difference between the training and test sets exceeded
3%. The final trained neural network was applied to the validation set and these classi-
fication accuracies are reported. Two neural network calibrations were performed. One
neural net was trained to discriminate IDK from undamaged kernels only. This network
was then applied to all other damaged kernels (scab damage, sprouted, and kernels dam-
aged by an insect at the pupae stage) to determine how other damaged kernels would be
classified when presented to an IDK-detection neural net. The second neural network was
trained to discriminate undamaged kernels from IDK plus a third group comprising scab,
sprouted, and hidden insect-damaged kernels. Commercial neural network software was
used (Alyuda Neurointelligence 2.1, Alyuda Research Corp., Los Altos, CA). The conju-
gate gradient algorithm [36] was used as the neural network training method. This method
does not require specification of momentum or learning rate as back propagation methods
do. The network architecture contained one hidden layer, and the number of hidden nodes
was determined using an exhaustive search from one to 60 input nodes. As the number of
features to be extracted has an impact on the processing time for real time sorting, the net-
works were tested using various combinations of the most significant features to determine
how many features were required for maximum accuracy.
3. Results
3.1. Discrimination between IDK and undamaged kernels (first network)
The stepwise discriminant analysis procedure selected 19 features from the pool of
all potential features. The six features selected as most significant were two magnitudes
on each side of the peak of the DFT spectrum, the DFT magnitudes at 19.5, 2.25, and
40.5 kHz, and the fourth magnitude in the differential spectrum. The selection of the DFT
magnitudes near the peak of the DFT spectrum as the two most significant features is not
surprising as this characterizes the sharpness of the peak in the frequency spectrum. The
selection of DFT magnitudes at 19.5, 2.25, and 40.5 kHz simply show that there were sig-
nificant differences in the general shape of the DFT between IDK and undamaged kernels.
Undamaged kernels tended to have more energy at higher frequencies than IDK. Finally,
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 629Table 2
Validation set classification accuracies for subsets of features
Features Percent accuracy
Undamaged IDK
Weibull a, b, and R2 87 71
Short-time window maximums 88 72
Short-time window variances and maximums 86 76
DFT peak area information 92 85
Complete DFT 91 77
Differential spectrum 78 85
Short-time window variances and maximums + DFT peak information 94 86
Top 6 selected features 96 87
19 selected parameters from all features 98 89
Table 3
Classification of other types of damaged kernels by inputting the
19 features selected to distinguish IDK from undamaged kernels
only to the neural network
Actual damage category Percent accuracy
Undamaged IDK
Scab 96 4
Sprout 95 5
Pupae 95 5
as pointed out earlier, the differential spectrum helps to identify smaller oscillations in the
DFT. For undamaged kernels, whose DFT’s tend to have a single broad peak accompanied
by smaller oscillations, the differential spectra show one peak at a low frequency and an-
other at a higher frequency (Fig. 9). In contrast, differential spectra from IDK tend to have
one peak at a moderate frequency as DFT from IDK tend to have one or two sharp peaks.
Thus, the stepwise discriminant procedure selected the fourth point, which tends to be the
peak for IDK and a local minimum for undamaged kernels. The Neural network was found
to perform best with 19 input nodes and 33 hidden nodes. Table 2 shows the classification
accuracies obtained when various combinations of the selected features were input into the
network.
The best results, 89% of IDK detected with 2% false positive, were obtained using all 19
features. Results using only the top six features as input vectors were only slightly worse,
with 96% of IDK detected and 4% false positives. The number of features used in relation
to the size and diversity of the training set indicates that over-fitting of the neural network
was not likely. Results gradually degraded as fewer input vectors were used.
Table 3 shows the classification results for other types of damage when presented to the
neural network trained to discriminate IDK from undamaged kernels. These results indi-
cate other types of damage are unlikely to be classified as IDK using this neural network.
A maximum of 6% of kernels with other types of damage were classified as IDK. While
this may be desirable, depending on the sorting criteria, there is the disadvantage that this
network was not able to detect hidden insect infestations.
630 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633Table 4
Classification results for network trained to distinguish undamaged and IDK, plus a third group including all other
damage
Actual damage category Percent accuracy
Undamaged IDK Scab, sprouted, or pupae
Undamaged 91 2 7
IDK 7 75 18
Scab 19 11 70
Sprouted 21 9 70
Pupae 33 22 45
The classification results obtained for discriminating IDK from undamaged kernels
compare favorably with X-ray imaging and NIR methods. Human examination of X-ray
films is a more accurate method of detecting infested kernels at all maturity levels but can
have false-positive errors of 1.0% or higher [8]. Computer algorithms to automatically
scan X-ray images have similar recognition rates as the SKCS for insect-infested ker-
nels but have higher false-positive rates, about 7.4% [9]. NIR methods also suffer from
false-positive errors and, additionally, kernel-orientation problems [37]. X-ray imaging
equipment is very expensive, and software to segment images of wheat kernels that are
touching each other leads to further errors. X-ray imaging likely would not be able to de-
tect scab-damaged or sprouted kernels either. NIR can detect approximately 90% of IDK
and scab-damaged kernels [6], with approximately 95% accuracy on undamaged kernels.
Thus, the accuracy of NIR for damaged kernels is slightly higher at the expense of slightly
higher false-positive errors. While NIR has been correlated with falling number tests with
reasonable accuracy on bulk samples, no study has yet shown good accuracy with de-
tecting sprout condition and alpha-amylase activity in single kernels. Thus, the acoustic
method may have some additional usefulness in this application given the lack of another
non-destructive method to detect sprout damaged kernels. NIR is more accurate than the
acoustic method to detect kernels with insect pupae [17]. However, automated NIR instru-
ments currently can have a throughput rate of approximately 1 kernel/s [38], while the
acoustic method proposed here should be feasible at rates of 40 kernels/s or higher.
3.2. Discrimination between undamaged, IDK, and other damage (second network)
The stepwise discriminant procedure selected 20 features to a three way classification
of kernels as undamaged, IDK, or having other damage (scab, sprout, pupae). As with
IDK training, the two most significant features were two magnitudes near the peak of the
DFT spectrum, followed by the DFT magnitude at 19.5 kHz, the fourth magnitude in the
differential spectrum, the maximum value in the fourth short-time window, and the DFT
magnitude at 67.5 kHz. Table 4 displays a confusion matrix of the classification results.
The results indicate that it is possible to distinguish undamaged kernels from IDK and
kernels with other types of damage with reasonable accuracy for IDK, scab and sprouted
kernels; but, poor accuracy for kernels infested with insects at the pupae level. For exam-
ple, a very poor quality sample comprising 3000 kernels (∼100 g) might contain 30 IDK,
30 scab damaged, 30 sprouted, 30 pupae and the remaining 2880 kernels would be un-
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 631damaged. Thus, this hypothetical sample would comprise 96% undamaged kernels and 1%
from each of the four damage types. A system could first classify kernels as IDK and di-
vert those one way, then classify kernels as having “other damage” and divert those another
direction, while undamaged kernels would go a third direction. This would produce an un-
damaged stream containing 2645 kernels, of which 99.1% (2621 kernels) are undamaged,
0.08% (2 kernels) are IDK, 0.23% (6 kernels) are scab damaged, 0.23% (6 kernels) are
sprouted, and 0.37% (10 kernels) have pupae damage. The “other damage” stream would
comprise less than 10% of the original sample, 263 kernels, with 76.8% of these being
undamaged, 2% being IDK, 8% being scab damaged, 8% being sprouted, and 5 having pu-
pae damage. Given the potential high-throughput rate for an acoustic-based sorter, a large
sample could be processed quickly and inspectors need only examine the reject streams.
Using the algorithm presented here, the signal processing required 20 ms/kernel to per-
form, with the Weibull curve fitting taking about half this time. If Weibull features are not
used, then computational time is less than 10 ms/kernel on a Windows-based PC system
using an AMD Athlon 3000 processor. If an air valve is used to divert damaged kernels,
it will cause a disturbance of approximately 15 ms where acquiring data on a new kernel
cannot be made. Thus, processing and removing kernels requires approximately 25 ms for
a kernel throughput rate of 40 kernels/s.
4. Conclusions and future work
This work shows that use of impact-acoustic emissions as a basis for detection of IDK,
sprout damage, and scab damage is a feasible and promising method. The computational
cost of classifying a kernel using this technique is very low, allowing inspection of large
numbers of wheat kernels very rapidly, ∼40 kernels/s. Grain inspectors usually use a 100 g
(3000 kernel) sample to inspect for IDK. This takes an inspector approximately 20 min to
do manually but can be accomplished in about 75 s with an acoustic system.
Further experimentation and development is needed to expand this method, using a
wider range of wheat varieties grown under different climatic conditions. Additionally,
wheat is stored at a range of moisture contents, typically between 9 and 13%, so data from
a range of moisture contents will be needed to determine its effect. Moisture content from
a bulk sample may need to be added to the neural network so that its influence can be
accounted for. Fortunately, moisture content can be easily measured and entered. Quite
likely, further signal processing and a more elaborate neural network will be needed to ob-
tain high classification accuracies on more varied data. Additionally, more study is needed
to improve accuracy on kernels infested with insects that have not yet emerged from the
kernels.
References
[1] C.M. Christensen, R.A. Meronuck, Quality Maintenance in Stored Grains and Seeds, University of Min-
nesota Press, Minneapolis, MN, 1986.
[2] T. Herman, R.K. Bequette, P.J. McClusky, J.R. Pedersen, J.G. Ponte, J.P. Shroyer, Quality Oriented Market-
ing of Hard Winter Wheat, Cooperative Extension Service, Kansas State University, Manhattan, KS, 1995.
632 T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633[3] J. Pederson, Insects: Identification, damage, and detection, in: D.B. Sauer (Ed.), Storage of Cereal Grains
and Their Products, fourth ed., American Association of Cereal Chemists, St. Paul, MN, 1992.
[4] USDA. Electronic code of federal regulations. Title 7 (Agriculture), Chapter VIII (Federal Grain Inspec-
tion Service), Part 810 (Official United States Standards for Grain), available at: www.usda.gov/gispa/
lawsandregs/lawsregs.htm, accessed 5 January 2004.
[5] FDA. Adulteration by insect and rodent filth. Compliance policy guide CPG7104.03, Section 578.300, avail-
able at: www.fda.gov/ora/compliance_ref/cpg/cpgfod/cpg578.300.htm, accessed 5 January 2004.
[6] G.E. Russell, Evaluation of four analytical methods to detect weevils in wheat: Granary weevil, Sitophilus
granarius (L), in soft white wheat, J. Food Protect. 51 (7) (1988) 547–553.
[7] R.P. Haff, X-ray Inspection of Wheat for Granary Weevils, PhD dissertation, University of California, Davis,
CA, 2001.
[8] R.P. Haff, Real-time X-ray inspection of wheat for infestation by the granary weevil, Sitophilus granarius
(L), Trans. ASAE 47 (2) (2004) 531–537.
[9] C. Karunakkaran, D.S. Jayas, N.D.G. White, Soft X-ray inspection of wheat kernels infested by Sitophilus
oryzae, Trans. ASAE 46 (3) (2003) 739–745.
[10] K.W. Vick, J.C. Webb, B.A. Weaver, C. Litzcow, Sound detection of stored-product insects that feed inside
kernels of grain, J. Econ. Entomol. 81 (5) (1988) 1489–1493.
[11] R.W. Mankin, D. Shuman, J.A. Coffelt, Acoustic counting of adult insects with differing rates and intensities
of sound production in stored wheat, J. Econ. Entomol. 90 (4) (1997) 1032–1038.
[12] D. Shuman, J.A. Coffelt, K.W. Vick, R.W. Mankin, Quantitative acoustical detection of larvae feeding inside
kernels of grain, J. Econ. Entomol. 86 (3) (1993) 933–938.
[13] D.W. Hagstrum, Paul W. Flinn, Comparison of acoustical detection of several species of stored grain beetles
(Coleoptera: Curculionidae, Tenebrionidae, Bostrichidae, Cucujidae) over a range of temperatures, J. Econ.
Entomol. 86 (4) (1993) 1271–1278.
[14] W.A. Bruce, W.M. Street, R.C. Semper, D. Fulk, Detection of Hidden Insect Infestations in Wheat by In-
frared Carbon Dioxide Gas Analysis, ARS bulletin AAT-S-26, July 1982.
[15] T.F. Schatzki, E.K. Wilson, G.B. Kitto, P. Behrens, I. Heller, Determination of hidden Sitophilus granarius
(Coleoptera: Curculionidae) in wheat by myosin ELISA, J. Econ. Entomol. 86 (5) (1993) 1584–1589.
[16] E.B. Maghirang, F.E. Dowell, J.E. Baker, J.E. Throne, Automated detection of single wheat kernels contain-
ing live or dead insects using near-infrared reflectance spectroscopy, Trans. ASAE 46 (4) (2003) 1277–1282.
[17] F.E. Dowell, J.E. Throne, J.E. Baker, Automated nondestructive detection of internal insect infestation of
wheat kernels by using near-infrared reflectance spectroscopy, J. Econ. Entomol. 91 (4) (1998) 899–904.
[18] T.C. Pearson, Automated detection of internal insect infestations in whole wheat kernels using a Perten
SKCS 4100, Appl. Eng. Agricul. 19 (6) (2003) 727–733.
[19] S.R. Delwiche, Classification of scab- and other mold-damaged wheat kernels by near-infrared reflectance
spectroscopy, Trans. ASAE 46 (3) (2003) 731–738.
[20] T.C. Pearson, Detection of pistachio nuts with closed shells using impact acoustics, Appl. Eng. Agricult.
17 (2) (2001) 249–253.
[21] A.E. Cetin, T.C. Pearson, A.H. Tewfik, Classification of closed- and open-shell pistachio nuts using voice-
recognition technology, Trans. ASAE 47 (2) (2004).
[22] A.E. Cetin, T.C. Pearson, A.H. Tewfik, Classification of closed- and open-shell pistachio nuts using impact
acoustical analysis, 2004, presented in ICASSP.
[23] D.B. Board, Incipient failure detection in high speed machinery, in: Proceedings of the 10th Symposium on
NDE, San Antonio, 1975.
[24] V. Emamian, M. Kaveh, A.H. Tewfik, Z. Shi, L. Jacobs, J. Jarzynski, Robust clustering of acoustic emission
signals using neural networks and signal subspace projections, EURASIP J. Appl. Signal Process. 2003 (3)
(2003) 276–286.
[25] C. Chen, C. Mo, A method for intelligent fault diagnosis of rotating machinery, Digital Signal Process. 14
(2004) 203–217.
[26] T.J. Holroyd, Acoustics Emissions and Ultrasonic Monitoring Handbook, first ed., Coxmoor Publishing
Company, Oxford, UK, 2000.
[27] B.B. Hubbard, in: A.K. Peters (Ed.), The World According to Wavelets: The Story of a Mathematical Tech-
nique in the Making, second ed., Natick, MA, 1998.
T.C. Pearson et al. / Digital Signal Processing 17 (2007) 617–633 633[28] J.W. Tukey, B.P. Bogert, M.J.R. Healy, in: M. Rosenblatt (Ed.), The frequency analysis of time series for
echoes: Cepstrum, pseudo-autocovariance, cross-cepstrum, and saphe-cracking, in: Proceedings of the Sym-
posium on Time Series Analysis, Wiley, New York, 1963, pp. 209–243.
[29] F. Duprat, M. Grotte, E. Pietri, D. Loonis, The acoustic impulse-response-method for measuring the overall
firmness of fruit, J. Agricult. Eng. Research 66 (4) (1997) 251–259.
[30] M.L. Stone, P.R. Armstrong, D.D. Chen, G.H. Brusewitz, N.O. Maness, Peach firmness prediction by mul-
tiple location impulse testing, Trans. ASAE 41 (1) (1998) 115–119.
[31] F.L. Younce, D.C. Davis, A dynamic sensor for cherry firmness, Trans. ASAE 38 (5) (1995) 1467–1476.
[32] J. Sugiyama, T. Katsurai, J. Hong, H. Koyama, K. Mikuriya, Melon ripeness monitoring by a portable
firmness tester, Trans. ASAE 41 (1) (1998) 121–127.
[33] D. Massie, D. Slaughter, J. Abbott, W. Hruschka, Acoustic, single-kernel wheat hardness, Trans. ASAE
36 (5) (1993) 1393–1398.
[34] W.R. Hruschka, Multivariate calibration and data compression, in: P. Williams, K. Norris (Eds.), Near-
Infrared Technology in the Agricultural and Food Industries, first ed., American Association of Cereal
Chemists, St. Paul, MN, 1987.
[35] C.J. Huberty, Applied Discriminant Analysis, Wiley, New York, 1994.
[36] C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, Oxford, MA, 1995.
[37] A.R. Ghaedian, R.L. Wehling, Discrimination of sound and granary weevil-larva-infested wheat kernels by
near-infrared diffuse reflectance spectroscopy, J. AOAC Internat. 80 (5) (1997) 997–1005.
[38] F.E. Dowell, E.B. Maghirang, Accuracy and feasibility of measuring characteristics of single kernels using
near-infrared spectroscopy, in: Proceedings of the ICC Conference 2002 “Novel Raw Materials, Technolo-
gies, and Products — New Challenge for Quality Control,” 2002, Budapest, pp. 313–320.
Tom C. Pearson received his PhD in engineering from University of California, Davis, in 1998.
He is currently a lead scientist/engineer at the United States Department of Agriculture—Agricultural
Research Service in Manhattan, Kansas. He is also an adjunct faculty at Kansas State University,
Department of Biological and Agricultural Engineering. His current research interests are in the
development of instrumentation and signal processing methods for rapid inspection and sorting of
agricultural products. Specific applications include use of imaging, acoustics and near infrared spec-
troscopy for detecting defects in grains.
A. Enis Cetin received his PhD in systems engineering in 1987 from the Moore School of
Electrical Engineering at University of Pennsylvania in Philadelphia. He is currently a Professor
in Electrical and Electronics Engineering at Bilkent University in Ankara, Turkey. His current re-
search interests are in the development of robust voice recognition systems in the presence of noise,
motion detection and tracking in video streams for security systems, and applications of digital signal
processing for improved inspection accuracy of agricultural products.
Ahmed H. Tewfik received his PhD in electrical engineering and computer science from the
Massachusetts Institute of Technology in 1987. He is currently the E.F. Johnson Professor of Elec-
tronic Communications, Department of Electrical Engineering, University of Minnesota in Min-
neapolis. His current research interests include genomics, datanomic computing, wireless personal
area communications, and food inspection.
Ron P. Haff received his PhD in engineering from University of California, Davis, in 2001. He
is currently a lead scientist/engineer at the United States Department of Agriculture—Agricultural
Research Service in Albany California. His current research interests are in the development of in-
strumentation and signal processing methods for rapid inspection and sorting of agricultural products.
Specific methods include X-ray imaging and physical methods for detecting defects in fruits, nuts,
and grains.