Proficiency Testing and Error Rates Flashcards

(70 cards)

1
Q

how long is the initial training for FIOs

A

9 weeks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

where do FIOs do their initial training

A
  • Ontario Police College (OPC) in Aylmer, Ontario
  • Canadian Police College (CPC) in Ottawa, Ontario
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

how long is the initial training for FIAs

A

2-3 weeks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

how long is the initial training for SOCOs

A

2-3 weeks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

what does SOCO stand for

A

scenes of crime officer

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

how often do FIOs in ontario need to recertify after their initial training

A

every 3 years

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

who runs the recertification test

A

ontario police college

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

how often does the RCMP run their own proficiency test for FIOs

A

every 2 years

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

what other proficiency testing is done outside of the RCMP and ontario police college

A
  • specific to the agency the FIO works at or administered through,
  • via the CPC, OPC or RCMP
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

how long is the OPC test and what is it on

A
  • 3 hours
  • mainly question based
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

how long is the RCMP test and what is it on

A
  • 5 hours
  • mainly casework exams
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what is the objective of the RCMP proficiency testing

A

to assess fingerprint examiners and their ability to reach correct conclusions (opinions) using the ACE-V process

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what happens in the CPC expert witness testimony course

A
  • 1 week course
  • Submit a real fingerprint case that resulted in an identification of a suspect/individual
  • Write a report and submit a CV
  • Complete a mock trial on that case
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

which fingerprint error rate study has errors in it

A

Miami-Dade

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

true/false there is an error rate study which replicates the entire fingerprint comparison process

A

false

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Each error rate is subject to what

A

the conditions of the experiment from which it was derived

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

error rate studied should be used to do what (according to CanFRWG)

A
  • validate or invalidate a discipline
  • not attempt to estimate the error rate in practice
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

in the miami-dade study, what was the false pos error rate

A
  • 1 in 18 (5.6%)
  • buuut not calculated properly
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

what does false pos identification mean

A

reporting an individual is the source of the print when they are not

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

what does false neg identification mean

A

reporting an individual is not the source of the print when they are

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

how do you calculate the error rate of an examiner’s casewor

A

of actual cases of error/# of cases in which fingerprint comparison done

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

finding the error rate of an examiner tells you what

A
  • the rate for each
  • not the ground truth for cases
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

what does ground truth mean

A
  • fundamental truths, facts
  • info known to be true
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

how do we find ground truths

A
  • direct observation
  • measurement
  • not inference
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
**true/false** ground truths can be provided via inference
false
26
what did Itiel Dror (2006) look at and what was the result
- **cognitive bias** - whether experts will be misled by extraneous info like context or not - had experts look at prints theyve already looked at 5 years earlier in real crim investigations to see if they have the same result - originally were identifications, added context that could bias to make them a non-match - only 5 participants from various countries - *most of them made* **different** *judgements*
27
describe the results of the Itiel Dror study (2006) but like more in depth
- 1 kept the same opinion *(inconclusive)* - 4 changed *(ie not a match)*
28
what was a big limitation of the Itiel Dror (2006) study
sample size (only 5 people)
29
the accuracy of examiners is the interest of who
the head/director of fingerprint unit
30
the likelihood FPR (or FNR) is the interest of who
trier of fact
31
describe the Glenn Langenburg (2015) study
- **verification** - inconclusive and exclusion decisions are *not* verified, as recommended by OSAC - and often done in a non-blind manner - they used AFIS to search exclusion and inconclusive decisions to help reduce wrong exclusions - also increases efficiency of fingerprint unit - can be applied to cold case investigations - buuuut AFIS isnt foolproof and doesn't replace human factor
32
why aren't inconclusive and exclusion decisions verified
1. limited personnel 2. limited resources 3. limited caseload
33
______ *(blind/ non-blind)* verifications would reduce the number of examiner errors
this is unknown
34
why cant we blindly use AFIS
- not foolproof *(per Mayfield case)* - and doesn't replace human factor
35
what is the false pos error rate across various studied
less than 1%
36
what is the false negative error rate across various studies
2.2-7.9%
37
what was the false neg error rate in the FBI/Noblis "black box study"
7.5%
38
should we focus on limiting false neg or false pos
We typically focus a lot on false positives cause we dont wanna send the wrong person, but in reality the false neg is higher so maybe it should be focused on more
39
what does FBI stand for
federal bureau of investigation
40
what is noblis
- **independent** non profit science, tech, strategy org - brings the best of scientific thought, management and eng expertise - in an environment of independence and objectivity
41
where is noblis located
washington DC
42
when are black box studied done
- to measure the reliability of methods that rely mainly on human judgement *rather than lab instruments* - used to evaluate reliability of forensic methods, measure error rates, and give courts info on admissibility
43
what was the point of the FBI/Noblis study
to measure the accuracy of fingerprint examiners conclusions
44
what was *not* considered in the FBI/Noblis study
how they reached those conclusions
45
what factors were all addressed as a single entity in the FBI/Noblis study
- education - experience - tech - procedure
46
how many fingerprint examiners were in the FBI/Noblis study
169
47
how many prints were the examiners looking at in the FBI/Noblis study
- compared 100 pairs of latent to exemplar - *randomly assigned** - selected from a pool of 744 pairs
48
who developed the software in the FBI/Noblis study
noblis
49
when was the FBI/Noblis study done
2011
50
why did the FBI/Noblis study come about
due to the daubert standard
51
when did daubert happen
1993
52
what is the daubert standard factors used for
what a trial judge may consider when determining whether to admit scientific testimony in court
53
what are the daubert standard factors
1. whether the theory or technique in question can be + has been tested *yes by langford (?)* 2. Whether it has been subjected to peer review and publication 3. The degree of its known or potential error rate *has been found* 4. The existence and maintenance of standards controlling its operation *if someone has a question on how to use ACE-V, would they have somewhere to go* 5. Whether it has attracted widespread acceptance within a relevant scientific community
54
what is the canadian equivilant of daubert
mohan
55
what came about in the case of US v Mitchell
- 2004 appeals court - recommended that prosecutors seek to shoe *individual* error rates of experts and not that of the forensic discipline in *general* - sooo practitioner error rates vs discipline error rates - this caused confusion as some didnt get the difference in such an error rate
56
does atkinson think practitioner or discipline error rates are better
- **discipline** - cause if one person just hasnt messed up yet, then thats a 0% error rate, which is misleading
57
what was the false pos rate in the FBI/Noblis Study
0,1%
58
what was the false neg rate in the FBI/Noblis Study
7.5%
59
what is the impact of the FBI/Noblis Study
- after publication, was introduced into court almost immediately - accepted by scientific community - report has been downloaded 70k times - among top 5% of all research outputs in terms of impact online - research team published 15 more papers related to latent print exams - *atkinson said if you cant understand this study, you have no business being on the stand talking about prints*
60
what made the FBI/Noblis Study effective
- FBI partnered w outside, independent researchers - FBI brought expertise in latent print exam, and noblis brought reputation for objective analysis - big sample size *169 experts from various agencies* - study was double blind, open set, and random - participants didnt know the ground truth - researchers didnt know exam identities, org. affiliations and decisions - study had diverse range of quality and complexity - challenging comparisons were intentionally included so that the error rate measured would represent an upper limit for errors in real casework - brought attention to the need to review exclusions of examiners
61
in the FBI/Noblis Study, how many examiners made at least one false neg error
85%
62
where did the case study on fabricating fingerprint evidence happen
- new york state police - identification unit of troop c in sidney, NY
63
when did the case study on fabricating fingerprint evidence happen
1984-1992
64
how many cases were affected in the case study on fabricating fingerprint evidence
30 (at least)
65
who was charged in the case study on fabricating fingerprint evidence
- 5 state troopers - 4 found guilty, 1 acquitted - *criminal trial*
66
what was done in the case study on fabricating fingerprint evidence
they would wait until suspect was identified in the case and then pretend to have found their fingerprint at the scene
67
how was it discovered in the case study on fabricating fingerprint evidence and what was the effect
- one of the troopers interviewed for a job w the CIA - they asked if he would commit a crime for national security, meaning like break into a house or use a wire - he explained how he fabricated evidence to convict people he felt were guilty of a crime - framing people wasn't what they meant - **effect:** cases had to be reviewed, some were overturned, lawsuits
68
how was the crime done in the case study on fabricating fingerprint evidence
- suspect would handle items while being booked at troop C headquarters - prints were lifted from these items and attached to evidence cards, claiming to be from the scene of murder - orrrr sometimes they were photocopies and said to have been lifted off an item
69
what new procedures are implemented after the case study on fabricating fingerprint evidence
* All fingerprints must be photographed before being lifted * At least two investigators must co-sign crime scene reports * At least two supervisors must verify the validity of all fingerprints discovered at crime scenes - *highlights why documentation matters: cause the print cant have been from the scene cause you never wrote of finding it there*
70