ITMF, ICCTM
Sti ki
W
ki
Stickiness Working group
Measurements based on counts :
variability and methods of analysis
Gozé E., Gourlot J.-P., Lassus S., ,
Annual cropping systems research unit, CIRAD
Measurements based on counts :
i bili
d
h d
f
l i
variability and methods of analysis
Useful measurement must be
1 correlated to practical properties of the 1 correlated to practical properties of the
analyzed material 2 reproduciblep
Measurements are variable V i bilit i k i d i i
Variability => risk in decisions
need of criteria to measure variability : e.g. CV=5% Is a single figure of standard deviation or a CV useful for count data? Is a 5% CV a good benchmark ?g
Outline
Outline
• Variability of some defects counts on yarn and
fiber : evidence and characterization of a mean
to variance relationship
• How to analyze calibration and round test
experiments
experiments
Research directions : extra sources of variability
• Research directions : extra sources of variability
Outline
Outline
• Variability of some defects counts on yarn and
fiber : evidence and characterization of a mean
to variance relationship
• How to analyze calibration and round test
experiments
experiments
Research directions : extra sources of variability
• Research directions : extra sources of variability
200% Neps on yarn 5 s a m p l e s o f 2 0 0 m p e r y a r n (Cirad laboratory) S t a n d a r d D e v i a t i o n 1 1 1 2 9 1 0 1 1 7 8 9 5 6 7 3 4 M e a n 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0
200% Neps on yarn 5 s a m p l e s o f 2 0 0 m p e r y a r n (Cirad laboratory) C V ( % ) 4 0 3 0 2 0 1 0 0 M e a n 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0
200% Neps on yarn 5 s a m p l e s o f 2 0 0 m p e r y a r n (Cirad laboratory) L o g N ( 1 + V a r i a n c e ) 5 4 4 3 2 L o g N ( 1 + M e a n ) 2 . 6 2 . 8 3 . 0 3 . 2 3 . 4 3 . 6 3 . 8 4 . 0 4 . 2 4 . 4
200% Neps on yarn 5 s a m p l e s o f 2 0 0 m p e r y a r n (Cirad laboratory) L o g N ( 1 + V a r i a n c e ) 5 4 Poisson distribution : Variance=mean 4 3 2 L o g N ( 1 + M e a n ) 2 . 6 2 . 8 3 . 0 3 . 2 3 . 4 3 . 6 3 . 8 4 . 0 4 . 2 4 . 4
Afis n ASTM D 5866 r e p e a t a b i l i t y w i t h i n l a b o r a t o r i e s C V ( % ) 6 0 4 0 5 0 3 0 4 0 2 0 0 1 0 M e a n 0 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0
Afis n ASTM D 5866 r e p e a t a b i l i t y w i t h i n l a b o r a t o r i e s L o g N ( 1 + V a r i a n c e ) 8 7 Variance mean Overdispersion = 5 6 L ( di i ) 4 5 ≈ Log(overdispersion) 3 2 1 2 3 4 5 6 7 L o g N ( 1 + M e a n )
Trash count ( C i r a d l a b o r a t o r y ) L o g N ( 1 + V a r i a n c e ) 6 5 4 3 2 1 1 2 3 4 L o g N ( 1 + M e a n )
Landmark probability distributions
Landmark probability distributions
I d d l l d d f i h i l
• Independently located defects in a homogeneous material
Poisson distribution
σ² = µ
• Patchy located defects in a homogeneous material
Neyman type A distribution
σ² = µ(1+φ)
• Independantly located defects in an heterogeneous material : density varies randomly : compound distribution
Log normal density :
Log-normal density :
Poisson-lognormal distribution σ² = µ (1+µφ)
G
d
it
Gamma density
• Distributions are only landmarks
• Addition of multiple effects : operator,
p
p
,
laboratory, calibration yields a more
complex compound distribution
complex compound distribution
• Any of the observed mean-to-variance
relationships could be fitted with an
relationships could be fitted with an
overdispersed negative binomial, where
lv Poisson = 1 etat=mixed Appareil=H2SD lv Poisson = 2.19 etat=mixed Appareil=H2SD 5 6 7 5 6 7 2 3 4 2 3 4 1 2 l 1 2 3 4 5 1 2 l 1 2 3 4 5 lm lm lv 7 Neg. binomial 1/k=0.0228 etat=mixed Appareil=H2SD lv 7 Neg. binomial 1/k=0.0228 = 1.44 etat=mixed Appareil=H2SD 5 6 7 5 6 7 2 3 4 2 3 4 1 2 lm 1 2 3 4 5 1 2 lm 1 2 3 4 5
Mean-to-variance relationship summary o v e r d i s p e r s i o n 4 3 ø 1/k Yarn-Neps repeat 1 0 2 AFIS-n repeat 3.07 0 AFIS-n reprod 3.95 0 Trashm repeat 1.09 0.0507 H2SD i d 1 44 0 0228 1 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0 H2SD mixed 1.44 0.0228 i n s t r _ p r e p A F I S - n r e p e a t A F I S - n r e p r o d H 2 S D m i x e d T r a s h m r e p e a t m e a n H 2 S D m i x e d T r a s h m r e p e a t Y a r n - N e p r e p e a t
Outline
Outline
• Variability of some defects counts on yarn and
fiber : evidence and characterization of a mean
to variance relationship
• How to analyze calibration and round test
experiments
experiments
Research directions : extra sources of variability
• Research directions : extra sources of variability
Calibration and round test experiments
Regression and analysis of variance are not appropriate p
• Regression and analysis of variance are not appropriate for count data, since the distribution of errors are not
“normal” (gaussian), and worse not of constant variance. • This does not always appear clearly, as diagnostics are
not displayed as standard in most softwares and not displayed as standard in most softwares and spreadsheet toolkits; and also because the
measurements conditions are not variable enough to display theses defects clearly
display theses defects clearly.
• We present here an experiment where the measuring p p g conditions have been set on purpose in a way they
strongly influence the results of a counting device : this caricatural situation will make more clear the need for alternative methods
Model for a two-way Anova: Yijk= # of SCT sticky spots i = cotton
Yijk = m + ai + bj + (ab)ij + Eijk j = measurement conditions k = replicate
M t i l ti hi U b ll lik i t ti l t
Mean to variance relationship Umbrella-like interaction plot :
Cotton and conditions effects are multiplicative
Two-way Anova : evidence of a
cotton x measurement conditions interaction
Non-additive effects of measurement conditions => cannot propose a calibration coefficient
=> cannot propose a calibration coefficient valid for all cottons
Analysis of variance on the SCT # sticky dots, cubic root transformed to stabilize variance Sum of
Source DF Squares Mean Square F Value Pr > F Model 65 194.2782433 2.9888961 35.11 <.0001 Model 65 194.2782433 2.9888961 35.11 <.0001
Error 132 11.2378639 0.0851353
Corrected Total 197 205.5161072
R-Square Coeff Var Root MSE rac3_col Mean 0.945319 9.573093 0.291780 3.047913
Source DF Type I SS Mean Square F Value Pr > F HUM 5 30.5636193 6.1127239 71.80 <.0001 coton 10 156.3502085 15.6350208 183.65 <.0001 HUM*coton 50 7.3644155 0.1472883 1.73 0.0072
Generalized linear model replaces analysis of variance :
Log( µijk ) = m + ai + bj + (ab)ij + Eijk
E[Yijk] = µ
V(Yijk ) (P i di t ib ti ) V(Yijk ) = µ (Poisson distribution)
or V(Yijk ) = µ ø (1+µ/k) (Overdispersed distribution) or V(Yijk ) = µ ø (1+µ/k) (Overdispersed distribution)
N
i t
ti
lib ti
ffi i t
No more interactions : calibrations coefficients
are now valid regardless of the particular cotton
being tested
The mean-to variance relationship is enough
The mean to variance relationship is enough
to provide means of analysis
of round tests and calibration data
of round tests and calibration data
with a generalized linear model,
where usual regression and Anova fail.
where usual regression and Anova fail.
The results are sound and intelligible
e esu ts a e sou d a d te g b e
Though now standard in medical and insurance
g
research, the generalized linear model
is not part of the toolkit
p
of the engineers and university graduates,
(except when specialized in Statistics).
Outline
Outline
V i bilit
f
d f t
t
• Variability of some defects counts on yarn
and fiber : evidence and characterization
of a mean to variance relationship
• How to analyze calibration and round test
experiments
experiments
• confidence intervals + research directions
on extra sources of variability
The mean-to variance relationship is not enough to provide means of calculating exact confidence to provide means of calculating exact confidence
intervals and litigation risks. Approximate results are available only when the count numbers are high
enough.
Checks of distributions have to be made on cottons Checks of distributions have to be made on cottons, especially in the region of maximum litigation risks.
The generalized linear model can be applied to more complicated designs to allow the estimation of
complicated designs to allow the estimation of different flavours of reproducibility :
changing week, month, technician, device, labs, …g g , , , , , Different scales of observation (specimen, sample, bale, module…) mean different sources of variability adding up. The resulting distribution can be seen as a continous mixture of Poisson distribution of various
continous mixture of Poisson distribution of various expectations, and the resulting distribution can be calculated by integration.y g
Funded by CSITC project, some investigations are in i i i bili i hi h
progress to investigate variability within the coton production in Africa, I hope my African colleagues and I will show you the results in two years time and I will show you the results in two years time.
Measurements based on counts :
i bili
d
h d
f
l i
variability and methods of analysis
Useful measurement must be Useful measurement must be
1 correlated to practical properties of the analyzed material
2 reproducible, i.e. its reproducibility has been measured with 2 reproducible, i.e. its reproducibility has been measured with proper indexes.
Apparent paradoxes displayed by classical variability Apparent paradoxes displayed by classical variability indexes such as standard deviation or CV on counts can be overcome by considering a simple relation can be overcome by considering a simple relation between mean and variance.
This leads to a general way of measuring the This leads to a general way of measuring the
variability, from repeatability to different flavours of reproducibility.
Is there a threshold beyond which an instrument should be rejected ?
A high overdispersion does not imply the instrument is not reliable (see e.g. AFIS-n). However the customers ot e ab e (see e g S ) o e e t e custo e
should be informed about the number of replicates needed to achieve a given precision. Commercial norms for the number of replicates should be derived from these numbers.
The customer should also be informed
of the procedure to calibrate his instrument and p
Trash area ( C i r a d l a b o r a t o r y ) L o g N ( 1 + V a r i a n c e ) 0 . 1 4 0 . 1 5 0 1 0 0 . 1 1 0 . 1 2 0 . 1 3 0 . 0 7 0 . 0 8 0 . 0 9 0 . 1 0 0 0 3 0 . 0 4 0 . 0 5 0 . 0 6 0 . 0 0 0 . 0 1 0 . 0 2 0 . 0 3 L o g N ( 1 + M e a n ) 0 . 0 0 0 . 0 5 0 . 1 0 0 . 1 5 0 . 2 0 0 . 2 5 0 . 3 0 0 . 3 5 0 . 4 0
